We lifted 4 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| NLPJCL/RAG-Retrieval | canonical | 4 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| sigmoid | Ran | NLPJCL/RAG-Retrieval/rag_retrieval/infer/reranker_models/llm_rankers.py code served (permissive licence) · get_code("b8e95809ca2c17c9") |
| pairwise_ranknet | Ran | NLPJCL/RAG-Retrieval/rag_retrieval/train/reranker/ranking_loss.py code served (permissive licence) · get_code("3e96bdcb68881aa5") |
| pointwise_bce | Ran | NLPJCL/RAG-Retrieval/rag_retrieval/train/reranker/ranking_loss.py code served (permissive licence) · get_code("4be0550994765aff") |
| pointwise_mse | Ran | NLPJCL/RAG-Retrieval/rag_retrieval/train/reranker/ranking_loss.py code served (permissive licence) · get_code("b07e6aba62f37f14") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
A crucial component in many deep learning applications, such as Frequently Asked Questions (FAQ) and Retrieval-Augmented Generation (RAG), is dense retrieval. In this process, embedding models transform raw text into numerical vectors. However, the embedding models that currently excel on text embedding benchmarks, like the Massive Text Embedding Benchmark (MTEB), often have numerous parameters and high vector dimensionality. This poses challenges for their application in real-world scenarios. To address this issue, we propose a novel multi-stage distillation framework that enables a smaller student embedding model to distill multiple larger teacher embedding models through three carefully designed losses. Meanwhile, we utilize Matryoshka Representation Learning (MRL) to reduce the vector dimensionality of the student embedding model effectively. Our student model named Jasper with 2 billion parameters, built upon the Stella embedding model, obtained the No.3 position on the MTEB leaderboard (as of December 24, 2024), achieving an average 71.54 score across 56 datasets. We have released the model and data on the Hugging Face Hub (https://huggingface.co/infgrad/jasper_en_vision_language_v1) (https://huggingface.co/datasets/infgrad/jasper_text_distill_dataset), and the training codes are available in this project repository (https://github.com/NLPJCL/RAG-Retrieval).
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2412.19048")
get_code_for_paper("2412.19048")
have("2412.19048")
Connect an agent — have() is free.