cmp-nct/ggllm.cpp
View on GitHub ↗ ★ 251 C
Falcon LLM ggml framework with CPU and GPU support
5 mentions on Hacker News by 5 people: 0 stories, 5 comments · first 2023-06-18 · latest 2023-08-17
20082026
-
It doesn't support Falcon right now, but there's a fork that does ( https://github.com/cmp-nct/ggllm.cpp/ ).
comment by gliptic · 2023-08-17 · in How Is LLaMa.cpp Possible? -
Pretty much anything with 32GB (?) total RAM+VRAM: https://github.com/cmp-nct/ggllm.cpp But its going to be slow without even a small Nvidia GPU (a 2060?). CPUs are really slow at prompt ingestion, and that can't be hidden with streaming.
comment by brucethemoose2 · 2023-08-01 · in Alfred-40B, an OSS RLHF version of Falcon40B -
…(I may or may not have a mild local LLM addiction - and video cards cost more then drugs) [1] https://github.com/cmp-nct/ggllm.cpp [2] https://huggingface.co/TheBloke/falcon-7b-instruct-GGML [3] https://huggingface.co/abhinavkulkarni/tiiuae…
comment by mikeravkine · 2023-07-11 · in Show HN: Danswer – Open-source question answering across all your docs -
…(finding models and downloading them) https://github.com/ggerganov/llama.cpp (llama) https://github.com/cmp-nct/ggllm.cpp (falcon) For interactive work (art/chat/research/playing around), things like: https://github.com/oobabooga/text-gener…
comment by horsawlarway · 2023-07-07 · in ChatGPT loses users for first time, shaking faith in AI revolution -
Experimental Falcon inference via ggml (so on CPU): https://github.com/cmp-nct/ggllm.cpp It has problems but it does work
comment by orost · 2023-06-18 · in Falcon LLM – A 40B Model