Logo
Sign up Sign in
forks/ollama
1
0
hosted on Rithub
Fork 0
You've already forked ollama
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
ollama/tokenizer
History
Michael Yang f1373193dc
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
..
testdata
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
bytepairencoding_test.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
bytepairencoding.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
sentencepiece_test.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
sentencepiece.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
tokenizer.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
vocabulary_test.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
vocabulary.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
wordpiece_test.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
wordpiece.go
move tokenizers to separate package (#13825)
2026-02-05 17:44:11 -08:00
English
English
Licenses API
Explore · Agents · Move from GitHub · Rithub vs GitHub · For startups · Terms · Privacy