Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
toksuite
/
tokenizer-comparison
like
0
Running
App
Files
Files
Community
Fetching metadata from the HF Docker repository...
main
tokenizer-comparison
/
app.py
Commit History
Further improvements
ce07484
Gül Sena Altıntaş
commited on
Sep 20, 2025
Improvements
44cdae3
Gül Sena Altıntaş
commited on
Aug 28, 2025
Small improvement for visualization
199862a
Gül Sena Altıntaş
commited on
Aug 27, 2025
Added support for showing newlines
d9779a0
Gül Sena Altıntaş
commited on
Aug 27, 2025
Fixed tokenmonster issue
f58b113
Gül Sena Altıntaş
commited on
Jul 29, 2025
Add normalization
37a99cb
Gül Sena Altıntaş
commited on
Jun 9, 2025
Added sample texts
452a924
Gül Sena Altıntaş
commited on
Jun 9, 2025
Refactoring, and visual improvements
c02e89e
Gül Sena Altıntaş
commited on
Jun 9, 2025
Added additional tokenizers
3a08f05
Gül Sena Altıntaş
commited on
Jun 9, 2025
Create app.py
6b9ea0b
verified
gsaltintas
commited on
Jun 9, 2025