▎Common Deep Learning Terms

1. Neural Network: A computational model inspired by the human brain, consisting of interconnected nodes (neurons) organized in layers.

2. Layer: A collection of neurons that process input data in a neural network; common types include input layers, hidden layers, and output layers.

3. Activation Function: A mathematical function applied to the output of each neuron, introducing non-linearity into the model; common examples include ReLU, sigmoid, and tanh.

4. Forward Propagation: The process of passing input data through the network to obtain an output prediction.

5. Backpropagation: An algorithm used to update the weights of a neural network by calculating the gradient of the loss function with respect to each weight.

6. Epoch: One complete pass through the entire training dataset during the training process.

7. Batch Size: The number of training examples used in one iteration of model training; affects memory usage and training speed.

8. Learning Rate: A hyperparameter that controls how much to change the model's weights during training based on the gradient of the loss function.

9. Dropout: A regularization technique that randomly sets a fraction of neurons to zero during training to prevent overfitting.

10. Convolutional Neural Network (CNN): A specialized type of neural network designed for processing grid-like data, such as images, using convolutional layers.

11. Recurrent Neural Network (RNN): A type of neural network designed for sequential data, allowing information to persist across time steps; often used in natural language processing.

12. Long Short-Term Memory (LSTM): A specific type of RNN architecture that can learn long-term dependencies by using memory cells and gates.

13. Generative Adversarial Network (GAN): A framework consisting of two neural networks (generator and discriminator) that compete against each other to generate new data samples.

14. Transfer Learning: A technique where a pre-trained model is fine-tuned on a new, often smaller dataset to leverage learned features.

15. Loss Function: A measure of how well the model's predictions match the actual outcomes; commonly used functions include mean squared error and categorical cross-entropy.

16. Optimizer: An algorithm used to adjust the weights of a neural network during training to minimize the loss function; examples include Adam, SGD, and RMSprop.

17. Gradient Descent: An optimization algorithm used to minimize the loss function by iteratively updating model parameters in the direction of the steepest descent.

18. Overfitting: A modeling error that occurs when a neural network learns noise and details from the training data too well, resulting in poor performance on unseen data.

19. Underfitting: A situation where a neural network fails to capture the underlying trend in the training data, leading to poor performance on both training and test datasets.

20. Data Augmentation: Techniques used to artificially increase the size of a training dataset by creating modified versions of existing data points (e.g., rotating, flipping images).