thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

TensorRT-LLM

NVIDIA's open-source library for optimized LLM inference on NVIDIA GPUs.

Definition

TensorRT-LLM provides in-flight batching, paged KV cache, quantization (FP8, INT4, FP4), speculative decoding and multi-GPU tensor and pipeline parallelism. It powers many NIM LLM microservices.

Exam tip

TensorRT-LLM is NVIDIA's optimized engine specifically for LLM inference.

Related terms