Hacker GitHub

← All repos

MoonshotAI/Attention-Residuals

View on GitHub ↗ ★ 3,570

3 mentions on Hacker News by 3 people: 2 stories, 1 comment · first 2026-03-16 · latest 2026-05-04

Mentions per year 3 mentions · busiest year 2026 (3)
20082026
AllStoriesComments NewestTopOldest
  1. …on static data but also on the capability of handling context in itself. Kimi introduced this https://github.com/MoonshotAI/Attention-Residuals but i'm pretty sure closed labs like Google had something like this for a while.
    comment by AntiUSAbah · 2026-05-04 · in The Road to a Billion-Token Context
  2. ▲ 241
    Attention Residuals
    story by GaggiX · 2026-03-20 · 34 comments
  3. ▲ 17
    Attention Residuals: Rethinking depth-wise aggregation [pdf]
    story by salkahfi · 2026-03-16 · 1 comment