AI Tools
Home/Self-Hosted AI/AutoAWQ

AutoAWQ

by casper-hansen1.4k

AWQ quantization for efficient inference

View on GitHub

About

AWQ quantization for efficient inference Provides comprehensive functionality for seamless integration.

Installation

pip install autoawq

Tags

quantizationawqefficient

Details

Category
Self-Hosted AI
Added
Aug 27, 2024
GitHub Stars
1.4k
casper-hansen/autoawq

Related Tools

llama.cpp

ggerganov

54.0k

Port of Facebook's LLaMA model in C/C++

localcppefficientFeatured

Mistral

Mistral AI

28.0k

Efficient open-source language models

mistralefficientopen-sourceFeatured

Mixtral

Mistral AI

8.5k

Mixture of experts model from Mistral

moemistralefficient
← Back to Self-Hosted AI