AI Tools
Home/Self-Hosted AI/vLLM

vLLM

by vLLM18.0k
Featured

High-throughput LLM serving

View on GitHub

About

High-throughput LLM serving Provides comprehensive functionality for seamless integration.

Installation

pip install vllm

Tags

servingthroughputpaged

Details

Category
Self-Hosted AI
Added
Nov 28, 2024
GitHub Stars
18.0k
vllm-project/vllm

Related Tools

vLLM

vLLM

18.0k

High-throughput LLM serving engine

servingperformanceinferenceFeatured

TGI

Hugging Face

7.5k

Hugging Face's LLM deployment solution

servinghuggingfaceinference

Vespa

Yahoo

4.5k

Big data serving engine for AI applications

vector-searchyahooserving
← Back to Self-Hosted AI