ridgerchu/matmulfreellm
View on GitHub ↗ ★ 3,101 Python
Implementation for MatMul-free LM.
7 mentions on Hacker News by 7 people: 3 stories, 4 comments · first 2024-06-09 · latest 2024-09-07
20082026
-
Claims 90% memory reduction with OSS code for replication on standard GPUs, https://github.com/ridgerchu/matmulfreellm > ..avoid using matrix multiplication using two main techniques. The first is a method to force all the numbers within…
comment by transpute · 2024-09-07 · in Hardware Acceleration of LLMs: A comprehensive survey and comparison -
Paper: https://arxiv.org/abs/2406.02528 -- always better than a press release. Code: https://github.com/ridgerchu/matmulfreellm --- Like others before them, the authors train LLMs using parameters consisting of ternary digits, or trits,…
comment by cs702 · 2024-06-25 · in Researchers run high-performing LLM on the energy needed to power a lightbulb -
▲ 3
Implementation for MatMul-Free LMstory by cpeterso · 2024-06-10 · 0 comments
-
▲ 4
MatMul-Free LM – architecture that eliminates need for Matrix Multiplication Opsstory by reimertz · 2024-06-10 · 1 comment
-
▲ 2
MatMul-Free LMstory by beefman · 2024-06-09 · 0 comments
-
…new paper seems to be the implementation of optimized and fused kernels using triton, as seen here: https://github.com/ridgerchu/matmulfreellm/blob/master/mmfre... This is quite useful, as this should make training this type of LLMs much…
comment by cpldcpu · 2024-06-09 · in Scalable MatMul-Free Language Modeling -
the github link in the paper: https://github.com/ridgerchu/matmulfreellm it is super easy to try it out, the 2.7B, 1.3B, 0.37B models are on huggingface, and the generate.py example just works if you have triton 2.2 installed
comment by throwaway71271 · 2024-06-09 · in Scalable MatMul-Free Language Modeling