thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Triton Inference Server

NVIDIA's open-source inference server (now Dynamo-Triton) for serving models from many frameworks on GPUs and CPUs.

Definition

Triton serves TensorRT, PyTorch, ONNX, TensorFlow, Python and vLLM backends with HTTP/gRPC APIs, dynamic batching, concurrent model execution, model ensembles and Prometheus metrics. NVIDIA now brands it Dynamo-Triton as part of the Dynamo platform.

Exam tip

Dynamic batching and concurrent model instances are Triton's key throughput features.

Related terms