Licensie / Python
Log in Join Waitlist

kitoken

0.11.0

Fast tokenizer for language models, supporting BPE, Unigram and WordPiece tokenization

License
BSD-2-Clause
Permissive - Unverified

A permissive license that comes in two variants, the BSD 2-Clause and BSD 3-Clause. Both have very minimal requirements about how the software can be redistributed.

Published
May 10, 2026
11d ago
Package Registry
PyPI Source Homepage
README badge Customize →
License Sources
SourceLicenseClass
Licensie (detected)
Pending-
PyPI (reported)
BSD-2-ClausePermissive

License detection is still in progress for this version.

Loading dependencies…
Versions
1 version
VersionLicensePublishedStatus
0.11.0 Latest ViewingBSD-2-Clause (Unverified)May 10, 2026 Pending