jzhang38/TinyLlama
View on GitHub ↗ ★ 9,010 Python archived
The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
11 mentions on Hacker News by 11 people: 1 story, 10 comments · first 2023-09-04 · latest 2026-06-19
20082026
-
i have toyed around Tiny Llama 1.1B ( https://github.com/jzhang38/TinyLlama ) for the "conversational brain" of NPCs in a game i was dabbling.... it was fun.
comment by slipwalker · 2026-06-19 · in Ask HN: Whats the best and small open source model? -
…the amount of training data that was needed to make a LLM perform at its potential. [1]: https://github.com/jzhang38/TinyLlama
comment by littlestymaar · 2024-11-06 · in The deep learning boom caught almost everyone by surprise -
…millions of dollars. Though I suppose it all depends on your personal definition of expensive. [0] https://github.com/jzhang38/TinyLlama [1] https://aws.amazon.com/ec2/instance-types/p4/
comment by 2024throwaway · 2024-02-12 · in Nvidia CEO: Every Country Needs Sovereign AI -
GitHub repo with links to the checkpoints: https://github.com/jzhang38/TinyLlama
comment by ronsor · 2024-01-05 · in TinyLlama: An Open-Source Small Language Model -
…to assemble full-fledged massively expensive systems in order to have impact. Check out TinyLlama; https://github.com/jzhang38/TinyLlama Four research students from the Singapore University of Technology and Design are pretraining a 1.1B…
comment by sunpazed · 2023-12-22 · in NLP Research in the Era of LLMs -
…with this. I am looking to run this on lower end systems. I wonder if someone has already tried https://github.com/jzhang38/TinyLlama , could save me some time :)
comment by AMICABoard · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file -
…is trying to do that pushing by training a small 1.1 billion-parameter model on 3 trillion tokens: https://github.com/jzhang38/TinyLlama
comment by yorwba · 2023-10-31 · in RedPajama v2 Open Dataset with 30T Tokens for Training LLMs -
…[1] https://github.com/ornladios/ADIOS2 [2] https://github.com/ggerganov/llama.cpp [3] https://github.com/jzhang38/TinyLlama
comment by godelski · 2023-10-21 · in What every developer should know about GPU computing -
…ttps://twitter.com/gneubig/status/1286731711150280705?s=20 ) TinyLlama is trying to do that for 1.1B: https://github.com/jzhang38/TinyLlama As long as we are not at the capacity limit, we will have a few of these 7B beats 13B (or 7B beats…
comment by joennlae · 2023-10-11 · in Mistral 7B -
…just started a 90 day effort to train a 1.1 billion parameter llama model on 3 trillion tokens. https://github.com/jzhang38/TinyLlama
comment by lumost · 2023-09-05 · in Chinchilla’s death