bytetok

0.2.2

A fast, modular and light-weight BPE tokenizer for NLP research and prototyping.

License Sources
SourceLicenseClass
Licensie (detected)
Pending-
PyPI (reported)
MITPermissive

License detection is still in progress for this version.

Loading dependencies…
Versions
3 versions
VersionLicensePublishedStatus
0.2.2 Latest ViewingMIT (Unverified)Mar 3, 2026 Pending
0.2.1 MIT (Unverified)Mar 1, 2026 Pending
0.2.0 MIT (Unverified)Feb 20, 2026 Pending