Connect with us

Hi, what are you looking for?

Hit The StockHit The Stock

Tech News

Microsoft LASERs away LLM inaccuracies

Microsoft logo
Illustration: The Verge

During the January Microsoft Research Forum, Dipendra Misra, a senior researcher at Microsoft Research Lab NYC and AI Frontiers, explained how Layer-Selective Rank Reduction (or LASER) can make large language models more accurate.

With LASER, researchers can “intervene” and replace one weight matrix with an approximate smaller one. Weights are the contextual connections models make. The heavier the weight, the more the model relies on it. So, does replacing something with more correlations and contexts make the model less accurate? Based on their test results, the answer, surprisingly, is no.

“We are doing intervention using LASER on the LLM, so one would expect that the model loss should go up as we are doing more approximation,…

Continue reading…

You May Also Like

Editor's Pick

Patrick G. Eddington In a fiery 40‐​minute press conference attended by almost a dozen House Freedom Caucus (HFC) members, the group discussed the current state of...

Editor's Pick

Marc Joffe Performance measurement in the public sector can be challenging, and state and local governments too often avoid the task entirely. However, the...

Editor's Pick

Chris Edwards President Biden has released his proposed federal budget for fiscal year 2025 and beyond. The proposal includes a raft of spending increases, including...

Editor's Pick

Colin Grabow Economic strategist and author David P. Goldman is concerned about the state of American manufacturing. Describing the sector as critical to both...