Apoorva Karnik
builder. maker. tinkerer.
I work at the intersection of hardware and software —
co-designing systems where the two meet for best performance.
When I'm not doing that, I'm reverse engineering things for fun.
Writing →
Kimi Delta Attention: How It Works, Step by StepKimi Delta Attention (KDA) is the core linear-attention mechanism introduced in Moonshot AI’s **Kimi Linear** architecture. Its goal is ambitious: keep the inference efficiency of recurrent or linear Transformer Inference MathSystems-level explainer on FLOPs, prefill vs decode, and KV cacheFLOPs CalculatorInteractive prefill & decode FLOPs estimatorKV Cache CalculatorLive memory sizing for MHA / GQA / MQA