Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
OpenMachineAI
/
UnitronX-Tokenizer-32k-v1
like
0
Follow
OpenMachine
2
tokenizers
tokenizer
bpe
byte-level
multilingual
code
unitronx
License:
mit
Model card
Files
Files and versions
xet
Community
main
UnitronX-Tokenizer-32k-v1
Commit History
Upload tokenizer_config.json
9c7e12e
verified
TheOpenMachine
commited on
Nov 6, 2025
Update README.md
111acaf
verified
TheOpenMachine
commited on
Nov 6, 2025
Create README.md
a0c093f
verified
TheOpenMachine
commited on
Nov 6, 2025
Initial release: UnitronX Tokenizer 32k v1
37646f7
verified
TheOpenMachine
commited on
Nov 6, 2025
initial commit
4e64f69
verified
TheOpenMachine
commited on
Nov 6, 2025