LLM Utilities

Fast BPE tokenizer for OpenAI models

Fast tokenizers library from Hugging Face

Unsupervised text tokenizer and detokenizer

Standalone GPT-2 tokenizer implementation

Online token counting for various models

Examples and guides for using OpenAI API

Curated list of ChatGPT prompt examples

Microsoft's tool for building LLM apps

Comprehensive guide to prompt engineering

Repository of LangChain prompts and chains

Open-source LLMOps platform for prompt management

Control panel for AI applications

LLM logging and cost tracking

Semantic cache for LLM queries

State-of-the-art text embeddings

Open-source embedding models

Task-aware text embeddings

Text embeddings for retrieval tasks

BAAI general embedding models

Nomic's open embedding models

Massive text embedding benchmark

Open-source embedding database

Open-source vector search engine

Vector database with extended filtering