tokenizers
ΠΠΏΠΈΡΠ°Π½ΠΈΠ΅
π₯ Fast State-of-the-Art Tokenizers optimized for Research and Production
Π―Π·ΡΠΊΠΈ
- Rust70,5%
- Python20,3%
- Jupyter Notebook4,9%
- TypeScript2,5%
- JavaScript1,2%
- CSS0,3%
- ΠΡΡΠ°Π»ΡΠ½ΡΠ΅0,3%
2 Π³ΠΎΠ΄Π° Π½Π°Π·Π°Π΄
README.md
