MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...
A team led by Cold Spring Harbor Laboratory Assistant Professor Benjamin Cowley has compressed a 60-million-parameter ...