SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2412.19048 · 2024

Jasper and Stella: distillation of SOTA embedding models

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 4 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
NLPJCL/RAG-Retrieval canonical 4 of 4
FunctionStatusWhere it lives
sigmoid Ran NLPJCL/RAG-Retrieval/rag_retrieval/infer/reranker_models/llm_rankers.py
code served (permissive licence) · get_code("b8e95809ca2c17c9")
pairwise_ranknet Ran NLPJCL/RAG-Retrieval/rag_retrieval/train/reranker/ranking_loss.py
code served (permissive licence) · get_code("3e96bdcb68881aa5")
pointwise_bce Ran NLPJCL/RAG-Retrieval/rag_retrieval/train/reranker/ranking_loss.py
code served (permissive licence) · get_code("4be0550994765aff")
pointwise_mse Ran NLPJCL/RAG-Retrieval/rag_retrieval/train/reranker/ranking_loss.py
code served (permissive licence) · get_code("b07e6aba62f37f14")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

A crucial component in many deep learning applications, such as Frequently Asked Questions (FAQ) and Retrieval-Augmented Generation (RAG), is dense retrieval. In this process, embedding models transform raw text into numerical vectors. However, the embedding models that currently excel on text embedding benchmarks, like the Massive Text Embedding Benchmark (MTEB), often have numerous parameters and high vector dimensionality. This poses challenges for their application in real-world scenarios. To address this issue, we propose a novel multi-stage distillation framework that enables a smaller student embedding model to distill multiple larger teacher embedding models through three carefully designed losses. Meanwhile, we utilize Matryoshka Representation Learning (MRL) to reduce the vector dimensionality of the student embedding model effectively. Our student model named Jasper with 2 billion parameters, built upon the Stella embedding model, obtained the No.3 position on the MTEB leaderboard (as of December 24, 2024), achieving an average 71.54 score across 56 datasets. We have released the model and data on the Hugging Face Hub (https://huggingface.co/infgrad/jasper_en_vision_language_v1) (https://huggingface.co/datasets/infgrad/jasper_text_distill_dataset), and the training codes are available in this project repository (https://github.com/NLPJCL/RAG-Retrieval).

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2412.19048")
get_code_for_paper("2412.19048")
have("2412.19048")

Connect an agent — have() is free.