Lightning-AI/lit-llama
View on GitHub ↗ ★ 6,080 Python
Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
5 mentions on Hacker News by 4 people: 1 story, 4 comments · first 2023-03-28 · latest 2023-06-13
20082026
-
…/huggingface.co/docs/transformers/main/model_doc/llam... [1] https://github.com/turboderp/exllama [2] https://github.com/Lightning-AI/lit-llama EDIT: Or do you mean cuda? Because yeah, it's such a shame AMD's Rocm is so bad even geohot…
comment by theaiquestion · 2023-06-13 · in Llama.cpp: Full CUDA GPU Acceleration -
So is https://github.com/Lightning-AI/lit-llama
comment by akrymski · 2023-05-16 · in Together’s $20M seed funding to build open-source AI and cloud platform -
You can customize a model with a "cheap" 3090 (or maybe a 7900 XTX), see here: https://github.com/Lightning-AI/lit-llama#finetune-the-model What OP did is more intense, but Lora/Adapter still can give excellent results. Pure AI cards for…
comment by brucethemoose2 · 2023-04-24 · in Godot-dodo – Finetuning LLaMA on single-language comment:code data pairs -
This is fabulous. Just want to add that there are efforts to impove training speed, like this: https://github.com/Lightning-AI/lit-llama/issues/62 So the practical cost/dataset size for language finetunes is bound to get better rapidly.…
comment by brucethemoose2 · 2023-04-23 · in Godot-dodo – Finetuning LLaMA on single-language comment:code data pairs -
▲ 158
Show HN: A fully open-source (Apache 2.0)implementation of llamastory by osurits · 2023-03-28 · 52 comments