22 июля 2026 г.

GigaToken: ~1000x faster Language model tokenization

rss:Hacker News: Front Page · оригинал

Article URL: https://github.com/marcelroed/gigatoken/ Comments URL: https://news.ycombinator.com/item?id=49010167 Points: 8 # Comments: 0

Заголовок: GigaToken: ~1000x faster Language model tokenization

Кратко: Article URL: https://github.com/marcelroed/gigatoken/ Comments URL: https://news.ycombinator.com/item?id=49010167 Points: 8 # Comments: 0

Детали: Article URL: https://github.com/marcelroed/gigatoken/ Comments URL: https://news.ycombinator.com/item?id=49010167 Points: 8 # Comments: 0