Artificial Neural Networks (ANN)

 Artificial Neural Networks (ANN):- 

Artificial Neural Networks are computational models inspired by the structure and function of the human brain's neural networks. ANNs consist of interconnected nodes (neurons) organized in layers, including an input layer, one or more hidden layers, and an output layer. They are used for various machine learning tasks, including classification, regression, clustering, and dimensionality reduction. ANN possess a large number of processing elements called nodes/neurons which operate in parallel. Neurons are connected with others by connection link. Each link is associated with weights which contain information about the input signal. Each neuron has an internal state of its own which is a function of the inputs that neuron receives- Activation level .In short , An artificial neural network consists of a pool of simple processing units which communicate by sending signals to each other over a large number of weighted connections. A set of major aspects of a parallel distributed model include:

• a set of processing units (cells).
• a state of activation for every unit, which equivalent to the output of the unit.
• connections between the units. Generally each connection is defined by a weight.
• a propagation rule, which determines the effective input of a unit from its external inputs.
• an activation function, which determines the new level of activation based on the effective input and the current activation.
• an external input for each unit.
• a method for information gathering (the learning rule).
• an environment within which the system must operate, providing input signals and _ if necessary _ error signals.

The working of ANN is as follows:-

Input Layer:-

• The input layer consists of neurons representing the input features of the data. Each neuron corresponds to one feature, and the number of neurons in the input layer equals the dimensionality of the input data.

Hidden Layers:-

• Hidden layers are intermediate layers between the input and output layers. Each hidden layer consists of multiple neurons, and the number of hidden layers and neurons in each layer can vary depending on the complexity of the problem.
• Neurons in hidden layers perform computations on the input data through a process called forward propagation.

Output Layer:-

• The output layer consists of neurons representing the outputs or predictions of the network. The number of neurons in the output layer depends on the type of task (e.g., binary classification, multi-class classification, regression).
• The activation function of the output neurons depends on the type of task. For example, the sigmoid or softmax function is often used for binary or multi-class classification, while linear activation is used for regression.

Connections and Weights:-

• Each neuron in a layer is connected to every neuron in the subsequent layer through weighted connections. These connections represent the parameters (weights) learned by the network during training.
• The weights determine the strength of the connections between neurons and are adjusted during the training process to minimize the error between the predicted and actual outputs.

Activation Functions:-

• Activation functions introduce non-linearity into the network, enabling it to learn complex patterns in the data. Common activation functions include sigmoid, tanh, ReLU (Rectified Linear Unit), and softmax.
• The activation function of each neuron determines its output based on the weighted sum of its inputs.

Forward Propagation:-

• During forward propagation, input data is fed into the input layer, and activations are computed sequentially through the hidden layers until the output layer.
• Each neuron in the network computes a weighted sum of its inputs, applies the activation function to the sum, and passes the result to the neurons in the next layer.

Training (Backpropagation):-

• Training an ANN involves adjusting the weights of the connections to minimize the difference between predicted and actual outputs.
• Backpropagation is the training algorithm used to compute the gradients of the loss function with respect to the weights and update the weights accordingly.
• This process iteratively adjusts the weights using optimization techniques such as stochastic gradient descent, mini-batch gradient descent, or adaptive learning rate methods.

Loss Function:-

• The loss function measures the difference between the predicted and actual outputs of the network. Common loss functions include mean squared error (MSE) for regression tasks and cross-entropy loss for classification tasks.

Training Epochs:-

• Training proceeds through multiple iterations or epochs, where the entire dataset is passed through the network, and the weights are updated based on the computed gradients.
• The number of epochs, as well as other hyperparameters such as learning rate and regularization strength, are tuned to optimize the network's performance.

Prediction:-

• Once trained, the ANN can be used to make predictions on new, unseen data by performing forward propagation with the learned weights.
• The output of the output layer represents the predicted values or probabilities for the given inputs.

ANNs learn complex patterns and relationships in data through the iterative adjustment of weights during training. They are versatile models capable of handling a wide range of tasks, including classification, regression, and more complex tasks such as image recognition and natural language processing.

Advantages:-

• Ability to learn complex and non-linear relationships in data.
• Can automatically extract features from raw data, reducing the need for manual feature engineering.
• Suitable for large-scale datasets and high-dimensional feature spaces.
• Can be trained using techniques like stochastic gradient descent, mini-batch gradient descent, and adaptive learning rate methods.

Disadvantages:-

• Requires a large amount of labeled data for training, which may be challenging to obtain in some domains.
• Complex architectures with many parameters can lead to overfitting, especially with insufficient regularization.
• Black-box nature makes it difficult to interpret and understand the learned representations.
• Computationally intensive during training, especially for deep architectures and large datasets.