AI Tools
Home/Self-Hosted AI/Ray Serve

Ray Serve

by Anyscale3.2k

Scalable model serving with Ray

View on GitHub

About

Scalable model serving with Ray Provides comprehensive functionality for seamless integration.

Installation

pip install ray[serve]

Tags

servingrayscalable

Details

Category
Self-Hosted AI
Added
Jun 5, 2024
GitHub Stars
3.2k
ray-project/ray

Related Tools

vLLM

vLLM

18.0k

High-throughput LLM serving engine

servingperformanceinferenceFeatured

TGI

Hugging Face

7.5k

Hugging Face's LLM deployment solution

servinghuggingfaceinference

Milvus

Zilliz

26.0k

Open-source vector database for AI

vector-dbscalableopen-source
← Back to Self-Hosted AI