AI Tools
Home/Self-Hosted AI/TensorRT-LLM

TensorRT-LLM

by NVIDIA6.8k

NVIDIA's optimized LLM inference

View on GitHub

About

NVIDIA's optimized LLM inference Provides comprehensive functionality for seamless integration.

Installation

pip install tensorrt-llm

Tags

nvidiainferenceoptimized

Details

Category
Self-Hosted AI
Added
Jun 18, 2024
GitHub Stars
6.8k
nvidia/tensorrt-llm

Related Tools

Ollama

Ollama

52.0k

Run large language models locally

localinferenceeasyFeatured

vLLM

vLLM

18.0k

High-throughput LLM serving engine

servingperformanceinferenceFeatured

TGI

Hugging Face

7.5k

Hugging Face's LLM deployment solution

servinghuggingfaceinference
← Back to Self-Hosted AI