Self-Hosted AI

AWQ quantization for efficient inference

Easy-to-use GPTQ quantization

Tensor library for ML on consumer hardware

File format for LLM weights

Apple's ML framework for Apple silicon

LLM inference optimized for Apple silicon

Rust ML framework with minimalist approach

Fast inference engine for Transformer models

Cross-platform ML inferencing

NVIDIA's optimized LLM inference

Operating LLMs in production

High-throughput LLM serving

Text Generation Inference from Hugging Face

Scalable model serving with Ray

NVIDIA's inference serving platform

Python bindings for llama.cpp

Hugging Face's model library

Easy distributed training and inference

Microsoft's deep learning optimization library

Parameter-efficient fine-tuning methods

Low-Rank Adaptation for fine-tuning

Efficient fine-tuning of quantized LLMs

2x faster LLM fine-tuning

Tool for fine-tuning various AI models