We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| pratyushasharma/laser | canonical | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| prune | Ran | pratyushasharma/laser/src/laser/matrix_utils.py code served (permissive licence) · get_code("dfd3f0f25b0b9ad2") |
| rank | Ran | pratyushasharma/laser/src/laser/matrix_utils.py code served (permissive licence) · get_code("15557c4657639788") |
| sorted_mat | Ran | pratyushasharma/laser/src/laser/matrix_utils.py code served (permissive licence) · get_code("349808d7ef472b90") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Transformer-based Large Language Models (LLMs) have become a fixture in modern machine learning. Correspondingly, significant resources are allocated towards research that aims to further advance this technology, typically resulting in models of increasing size that are trained on increasing amounts of data. This work, however, demonstrates the surprising result that it is often possible to significantly improve the performance of LLMs by selectively removing higher-order components of their weight matrices. This simple intervention, which we call LAyer-SElective Rank reduction (LASER), can be done on a model after training has completed, and requires no additional parameters or data. We show extensive experiments demonstrating the generality of this finding across language models and datasets, and provide in-depth analyses offering insights into both when LASER is effective and the mechanism by which it operates.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2312.13558")
get_code_for_paper("2312.13558")
have("2312.13558")
Connect an agent — have() is free.