Deep Learning Specialization / AI credentials overview: key terms

Flashcards for this exam Deep Learning Specialization / AI credentials overview guide

• Activation FunctionA nonlinear function applied to a neuron's output that lets networks learn complex patterns.Foundational
• AdamAdaptive Moment Estimation: an optimizer combining momentum with per-parameter adaptive learning rates.Intermediate
• BackpropagationThe algorithm that computes gradients of the loss with respect to every weight by applying the chain rule backward through the network.Intermediate
• Batch NormalizationNormalizing each layer's inputs across the mini-batch to stabilize and speed up training.Intermediate
• Batch SizeThe number of training examples processed together before the model's parameters are updated.Foundational
• Convolutional Neural Network (CNN)A neural network that uses convolutional filters to detect spatial patterns, primarily for images.Intermediate
• Cross-Entropy LossA loss that measures the difference between predicted class probabilities and the true labels; standard for classification and LLMs.Intermediate
• Data AugmentationExpanding a training set by creating modified copies of existing examples.Intermediate
• Deep Learning (DL)A subset of machine learning that uses multi-layer neural networks to learn hierarchical representations directly from raw data.Foundational
• DropoutA regularization technique that randomly deactivates a fraction of neurons during each training step.Intermediate
• Early StoppingHalting training when validation performance stops improving to prevent overfitting.Intermediate
• EpochOne complete pass of the training algorithm through the entire training dataset.Foundational
• Exploding GradientA training problem where gradients grow uncontrollably large, causing unstable updates and NaN losses.Intermediate
• Gradient DescentAn optimization algorithm that iteratively adjusts parameters in the direction that most reduces the loss.Intermediate
• HyperparameterA configuration value set before training that controls the learning process, rather than being learned from data.Foundational
• Image ClassificationA computer vision task that assigns a label to an entire image.Foundational
• KerasA high-level deep learning API for defining and training neural networks; Keras 3 runs on TensorFlow, JAX or PyTorch.Intermediate
• L2 RegularizationAdds a penalty proportional to the squared weights, shrinking all weights toward zero (Ridge, weight decay).Intermediate
• Learning RateThe hyperparameter that sets how large a step the optimizer takes when updating weights.Foundational
• Long Short-Term Memory (LSTM)An RNN variant with gates that control what information to keep, forget and output, enabling long-range memory.Intermediate
• Loss FunctionA function that measures how far a model's predictions are from the true values; training minimizes it.Intermediate
• Multiclass ClassificationClassification where each input is assigned to exactly one of three or more classes.Intermediate
• Multilayer Perceptron (MLP)A feedforward neural network with one or more hidden layers of fully connected neurons.Intermediate
• Neural Network (NN)A model made of layers of interconnected artificial neurons that learn weighted transformations of input data.Foundational
• NormalizationMin-max scaling that rescales a feature to a fixed range, usually 0 to 1.Foundational
• OptimizerThe algorithm that updates model weights using computed gradients, such as SGD, Adam or AdamW.Intermediate
• OverfittingWhen a model learns training data too closely, including noise, and performs poorly on new data.Foundational
• Recurrent Neural Network (RNN)A neural network that processes sequences by passing a hidden state from one step to the next.Intermediate
• RegularizationTechniques that constrain model complexity to reduce overfitting.Intermediate
• ReLURectified Linear Unit: an activation that outputs the input if positive and zero otherwise, f(x)=max(0,x).Foundational
• Residual ConnectionA skip connection that adds a layer's input to its output, easing gradient flow in deep networks.Advanced
• Sigmoid FunctionAn S-shaped activation that maps any real number to the range 0 to 1.Foundational
• SoftmaxA function that converts a vector of raw scores (logits) into a probability distribution summing to 1.Intermediate
• Stochastic Gradient Descent (SGD)Gradient descent that updates parameters using one example or a small mini-batch at a time.Intermediate
• TensorA multi-dimensional array of numbers, the core data structure in deep learning frameworks.Foundational
• TensorFlowAn open-source machine learning framework from Google, with Keras as its high-level API.Foundational
• Transfer LearningReusing a model trained on one task as the starting point for a related task.Intermediate
• UnderfittingWhen a model is too simple to capture underlying patterns and performs poorly on both training and new data.Foundational
• Vanishing GradientA training problem where gradients shrink toward zero in early layers, so they learn very slowly or not at all.Intermediate
• WeightsLearned numeric parameters that scale the strength of connections between neurons.Foundational