NVIDIA-Certified Professional: Generative AI LLMs: key terms

Flashcards for this exam NVIDIA-Certified Professional: Generative AI LLMs guide Class dates

• AdamAdaptive Moment Estimation: an optimizer combining momentum with per-parameter adaptive learning rates.Intermediate
• AI AlignmentThe effort to ensure AI systems pursue goals and behave in accordance with human intentions and values.Intermediate
• Attention MechanismA technique that lets a model weight the relevance of different input elements when producing each output.Intermediate
• Autoregressive GenerationProducing output one token at a time, with each new token conditioned on all previously generated tokens.Intermediate
• Base ModelA pretrained model before instruction tuning or alignment, which continues text rather than following instructions.Intermediate
• BenchmarkA standardized dataset and metric used to compare model capabilities.Intermediate
• BERTScoreA text-similarity metric that compares contextual embeddings of generated and reference text.Advanced
• BF16Brain Floating Point 16: a 16-bit format with FP32's exponent range but less mantissa precision.Intermediate
• BLEUBilingual Evaluation Understudy: a metric scoring generated text by n-gram precision overlap with reference text.Intermediate
• Byte-Pair Encoding (BPE)A subword tokenization algorithm that iteratively merges the most frequent adjacent symbol pairs into new tokens.Advanced
• Catastrophic ForgettingWhen fine-tuning on new data causes a model to lose previously learned capabilities.Advanced
• Continued PretrainingFurther self-supervised training of a foundation model on large unlabeled domain data.Advanced
• Continuous BatchingAn LLM serving technique that adds and removes requests from the running batch at each generation step.Advanced
• Cross-EncoderA model that scores a query and document together in one pass, giving accurate relevance at higher cost.Advanced
• Cross-Entropy LossA loss that measures the difference between predicted class probabilities and the true labels; standard for classification and LLMs.Intermediate
• Data ParallelismDistributed training where each GPU holds a full model copy and processes a different slice of each batch.Intermediate
• Decoder-Only ArchitectureA transformer that uses only the decoder stack with causal (masked) attention to generate text one token at a time.Intermediate
• Decoding StrategyThe method used to choose each next token from the model's probability distribution.Intermediate
• Direct Preference Optimization (DPO)A preference-tuning method that optimizes a model directly on preferred vs. rejected responses without a separate reward model.Advanced
• Encoder-Decoder ArchitectureA transformer with an encoder that reads the input and a decoder that generates output while attending to the encoder.Intermediate
• Encoder-Only ArchitectureA transformer that uses only the encoder stack with bidirectional attention, suited to understanding tasks.Intermediate
• Evaluation HarnessA framework that runs a model or application over test cases and computes metrics automatically and repeatably.Advanced
• FAISSFacebook AI Similarity Search: Meta's open-source library for efficient vector similarity search, with GPU support.Advanced
• Fine-TuningFurther training a pretrained model on a smaller task- or domain-specific dataset to adapt its behavior.Intermediate
• FlashAttentionAn IO-aware exact attention algorithm that reduces GPU memory traffic to speed up attention.Advanced
• FP1616-bit half-precision floating point, halving memory versus FP32 with a narrower numeric range.Intermediate
• FP4A 4-bit floating-point format accelerated by NVIDIA Blackwell Tensor Cores for high-throughput inference.Advanced
• FP8An 8-bit floating-point format supported by NVIDIA Hopper and later Tensor Cores for faster training and inference.Advanced
• Full Fine-TuningUpdating all of a model's weights during fine-tuning.Intermediate
• Fully Sharded Data Parallel (FSDP)A PyTorch training strategy that shards parameters, gradients and optimizer states across data-parallel GPUs.Advanced
• Golden DatasetA curated set of inputs with verified expected outputs used as the reference for evaluation and regression testing.Intermediate
• Gradient AccumulationSumming gradients over several mini-batches before updating weights to simulate a larger batch.Advanced
• GraphRAGRAG that uses a knowledge graph of entities and relationships to retrieve connected information.Advanced
• GroundednessAn evaluation measure of whether a response's claims are supported by the provided context.Intermediate
• High Bandwidth Memory (HBM)Stacked DRAM packaged alongside the GPU that provides very high memory bandwidth.Intermediate
• Hugging FaceA platform and open-source community hosting models, datasets and libraries such as Transformers.Foundational
• Human EvaluationAssessment of model outputs by human raters, often treated as the gold standard for quality.Foundational
• HumanEvalA code-generation benchmark of programming problems scored by whether generated code passes unit tests.Intermediate
• In-Context Learning (ICL)A model's ability to learn a task from instructions and examples in the prompt without weight updates.Intermediate
• Inference ThroughputThe amount of work an inference system completes per unit time, such as tokens or requests per second.Intermediate
• Instruction TuningSupervised fine-tuning on many instruction-response examples so a model follows natural-language instructions.Intermediate
• Knowledge DistillationTraining a smaller student model to reproduce the outputs or behavior of a larger teacher model.Intermediate
• KV CacheStored attention keys and values for previous tokens, reused so each new token does not recompute them.Advanced
• Layer NormalizationNormalizing activations across the features of each individual example rather than across a batch.Advanced
• Learning RateThe hyperparameter that sets how large a step the optimizer takes when updating weights.Foundational
• LLM-as-a-JudgeUsing a language model to grade or compare outputs of another model against a rubric.Intermediate
• LLMOpsOperational practices for deploying and managing large language model applications in production.Intermediate
• Lost in the MiddleThe tendency of some LLMs to use information at the start and end of a long context better than information in the middle.Advanced
• Low-Rank Adaptation (LoRA)A PEFT method that freezes base weights and trains small low-rank matrices added to selected layers.Intermediate
• Masked Language Modeling (MLM)A pretraining objective in which random tokens are hidden and the model predicts them from surrounding context.Advanced
• Mixed Precision TrainingTraining with lower-precision number formats (FP16/BF16/FP8) for most operations while keeping critical values in FP32.Intermediate
• Mixture of Experts (MoE)An architecture where a router activates only a few expert sub-networks per token, increasing capacity without proportional compute.Advanced
• Multi-Head Attention (MHA)Running several attention operations in parallel so the model can attend to different relationships at once.Advanced
• NeMo GuardrailsNVIDIA's open-source toolkit for adding programmable safety, topic and security rails to LLM applications.Advanced
• Next-Token PredictionThe training objective of predicting the next token in a sequence given the preceding tokens.Intermediate
• NVIDIA AI Enterprise (NVAIE)NVIDIA's licensed, supported software suite for developing and deploying production AI, including NIM, NeMo and Triton.Intermediate
• NVIDIA DynamoNVIDIA's open-source distributed inference framework for serving reasoning and generative models across many GPUs.Advanced
• NVIDIA NeMoNVIDIA's framework and microservices for building, customizing, evaluating and guarding generative AI models and agents.Intermediate
• NVIDIA NGCNVIDIA's catalog of GPU-optimized containers, pretrained models, Helm charts and SDKs.Intermediate
• NVIDIA NIMNVIDIA Inference Microservices: prebuilt, optimized containers that serve AI models through industry-standard APIs.Intermediate
• ONNXOpen Neural Network Exchange: an open format for representing models so they can move between frameworks and runtimes.Intermediate
• Open-Weight ModelA model whose trained weights are publicly downloadable for self-hosting and customization.Intermediate
• OptimizerThe algorithm that updates model weights using computed gradients, such as SGD, Adam or AdamW.Intermediate
• PagedAttentionA KV-cache memory technique, introduced by vLLM, that stores cache in fixed-size blocks like virtual memory pages.Advanced
• Parameter-Efficient Fine-Tuning (PEFT)Fine-tuning methods that update only a small set of added or selected parameters while freezing the base model.Intermediate
• PerplexityA language-model metric measuring how well a model predicts text; lower perplexity means better prediction.Intermediate
• Pipeline ParallelismSplitting a model's layers into sequential stages placed on different GPUs, with micro-batches flowing through.Advanced
• PolicyIn reinforcement learning, the strategy that maps states to actions.Intermediate
• Positional EncodingInformation added to token embeddings so a transformer knows the order of tokens.Advanced
• Prefill and DecodeThe two phases of LLM inference: processing the whole prompt in parallel (prefill), then generating tokens sequentially (decode).Advanced
• PretrainingThe initial, compute-intensive training of a model on vast unlabeled data to learn general representations.Intermediate
• Prompt TuningA PEFT method that learns continuous 'soft prompt' embeddings prepended to inputs while the model stays frozen.Advanced
• PruningRemoving less important weights, neurons or layers from a model to make it smaller and faster.Advanced
• PyTorchAn open-source deep learning framework, originally from Meta and now governed by the PyTorch Foundation, known for dynamic computation graphs.Foundational
• QLoRALoRA fine-tuning on top of a base model quantized to 4-bit precision, sharply reducing memory.Advanced
• QuantizationReducing the numeric precision of model weights (and sometimes activations) to shrink memory and speed inference.Intermediate
• RAG EvaluationMeasuring a RAG system's retrieval quality and answer quality, typically with faithfulness, relevance and context metrics.Advanced
• Reasoning ModelAn LLM trained to generate intermediate reasoning before answering, improving performance on complex multi-step tasks.Intermediate
• Reinforcement Learning from Human Feedback (RLHF)Aligning a model by training a reward model on human preference rankings and optimizing the model against it.Intermediate
• RerankingRe-scoring an initial set of retrieved results with a more accurate model to put the most relevant first.Intermediate
• Residual ConnectionA skip connection that adds a layer's input to its output, easing gradient flow in deep networks.Advanced
• Reward FunctionThe signal that tells a reinforcement learning agent how good an action or outcome was.Intermediate
• Reward HackingWhen an AI system exploits flaws in its reward signal to score highly without doing the intended task.Advanced
• Reward ModelA model trained to score outputs according to human or AI preferences, used to guide RL fine-tuning.Advanced
• ROUGERecall-Oriented Understudy for Gisting Evaluation: n-gram and sequence overlap metrics for summarization.Intermediate
• Scaling LawsEmpirical relationships showing model loss improves predictably with more parameters, data and compute.Advanced
• Self-AttentionAttention in which each token in a sequence attends to every other token in the same sequence.Intermediate
• Self-Supervised Learning (SSL)Learning in which the training signal is generated automatically from the data itself, without human labels.Intermediate
• SoftmaxA function that converts a vector of raw scores (logits) into a probability distribution summing to 1.Intermediate
• Speculative DecodingSpeeding up generation by having a small draft model propose tokens that the large model verifies in parallel.Advanced
• Supervised Fine-Tuning (SFT)Fine-tuning a model on labeled input-output pairs, such as instruction-response examples.Intermediate
• SWE-benchA benchmark that measures whether models can resolve real GitHub issues by producing patches that pass the project's tests.Advanced
• Synthetic DataArtificially generated data used to train, fine-tune or evaluate models.Intermediate
• Tensor ParallelismSplitting individual layers' weight matrices across GPUs so each computes part of every operation.Advanced
• TensorRTNVIDIA's SDK for optimizing trained models into high-performance inference engines.Intermediate
• TensorRT-LLMNVIDIA's open-source library for optimized LLM inference on NVIDIA GPUs.Advanced
• Test-Time ComputeAdditional computation spent during inference, such as longer reasoning or multiple samples, to improve answer quality.Advanced
• Time to First Token (TTFT)The latency between sending a request and receiving the first generated token.Intermediate
• TokenizerThe component that converts text to token IDs and back, defined by its vocabulary and splitting rules.Intermediate
• TransformerA neural network architecture based on self-attention that processes all tokens in a sequence in parallel; the basis of modern LLMs.Foundational
• Transformer EngineNVIDIA hardware and software that dynamically manages FP8/FP4 precision to accelerate transformer models.Advanced
• Triton Inference ServerNVIDIA's open-source inference server (now Dynamo-Triton) for serving models from many frameworks on GPUs and CPUs.Intermediate
• Vision Transformer (ViT)A transformer that processes an image as a sequence of fixed-size patches embedded like tokens.Advanced
• Vision-Language Model (VLM)A model that jointly understands images and text, for tasks such as visual question answering and captioning.Intermediate
• vLLMA popular open-source, high-throughput LLM serving engine built around PagedAttention.Advanced