Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Harvard University have developed a method to reduce the memory needed to fine-tune large language models. In reporting published on November 19, 2026, MIT CSAIL said the method, called Low-memory Activation Rank Subspace (LARS), addresses a memory cost that remains even when only a small fraction of a model’s parameters are updated.
Parameter-efficient fine-tuning methods such as LoRA limit parameter updates, but training still stores intermediate calculations known as activations. LARS reduces that storage burden by keeping those calculations in a more compact form. In the researchers’ tests, it reduced memory use by more than 23% on GPUs and about 47% on CPUs compared with leading parameter-efficient fine-tuning methods.
On an 8 GB Raspberry Pi 5, LARS saved 9% more memory than LoRA when fine-tuning at a context length of 128,000; the researchers found that savings grew as context length increased. In other tests, LARS saved more memory than the comparison methods but was slightly less accurate than LoRA. Across question-answering tests on NVIDIA GPUs, it saved about 10 GB more than its leading baseline. In CPU tests using prompts of different lengths, the researchers also observed roughly 22% lower power and energy use than comparable methods.
MIT CSAIL researcher Irene Tenison said the approach could help devices analyze longer stretches of data locally—for example, heart-rate readings on a fitness watch. That is a potential application, not a reported deployment. The researchers presented the LARS paper at the Asia-Pacific Chapter of the Association for Computational Linguistics conference in November.




