Home/Self-Hosted AI/Triton Inference Server

Triton Inference Server

Name: Triton Inference Server
Rating: 3.6 (7200 reviews)
Author: NVIDIA

by NVIDIA7.2k

NVIDIA's inference serving platform

About

NVIDIA's inference serving platform Provides comprehensive functionality for seamless integration.

docker pull nvcr.io/nvidia/tritonserver

Microsoft

Microsoft's LLM orchestration SDK

High-throughput LLM serving engine

Hugging Face

Hugging Face's LLM deployment solution