AI Tools
Home/LLM Utilities/SentencePiece

SentencePiece

by Google8.2k

Unsupervised text tokenizer and detokenizer

View on GitHub

About

Unsupervised text tokenizer and detokenizer Provides comprehensive functionality for seamless integration.

Installation

pip install sentencepiece

Tags

tokenizergooglesubword

Details

Category
LLM Utilities
Added
Aug 22, 2024
GitHub Stars
8.2k
google/sentencepiece

Related Tools

Google Drive MCP

Anthropic

15.0k

Access and manage Google Drive files and folders

googledrivestorage

YouTube MCP Server

Community

2.0k

YouTube video and channel management

youtubevideogoogle

Firebase MCP Server

Community

2.8k

Google Firebase services integration

firebasegooglebackend
← Back to LLM Utilities