Navid Rekabsaz, Markus Frohmann, Mahdiyar Alavi, Elizabeth Lingg
We lifted 16 functions out of this paper's own repositories and ran 14 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| thomsonreuters/presentation-dependence | canonical | 14 of 16 |
| Function | Status | Where it lives |
|---|---|---|
| canonical_method_key | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/analysis/terminology.py code served (permissive licence) · get_code("0b18e643c678e842") |
| detect_surface | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/analysis/amortization.py code served (permissive licence) · get_code("52ffd26e79d3940a") |
| hierarchical_bootstrap_ci | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/analysis/metrics.py code served (permissive licence) · get_code("9d94581fe52f5f71") |
| load_b1_scores | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/copresence_scores.py code served (permissive licence) · get_code("a4d7d5f474a39ce8") |
| load_b20_order_averaged_scores | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/copresence_scores.py code served (permissive licence) · get_code("516a0a73bb992598") |
| model_signature | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/bundle.py code served (permissive licence) · get_code("b7956f205862a59d") |
| normalize_recipe_token | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/analysis/amortization.py code served (permissive licence) · get_code("d052ee9cfc9e1da1") |
| paired_bootstrap_ci | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/analysis/metrics.py code served (permissive licence) · get_code("d0479097f346085d") |
| paper_method_label | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/analysis/terminology.py code served (permissive licence) · get_code("68ba74adfebf2269") |
| published_presentations | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/context_decomposition.py code served (permissive licence) · get_code("cb864446516f5c03") |
| rank_ids_from_scores | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/context_decomposition.py code served (permissive licence) · get_code("12b8497af8a4338e") |
| shard_qids | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/context_decomposition.py code served (permissive licence) · get_code("64a0c08a1d67fc27") |
| stem_of | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/analysis/amortization.py code served (permissive licence) · get_code("5b507438e60abedc") |
| summarize_score_pairs | Ran | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/copresence_scores.py code served (permissive licence) · get_code("75bc657ab9a4ae00") |
| assert_shared_model | Not yet run | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/bundle.py code served (permissive licence) · get_code("6eeaae096d235160") |
| run_bundle | Not yet run | thomsonreuters/presentation-dependence/src/presentation_dependence/eval/bundle.py code served (permissive licence) · get_code("121652f9d430c54a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Rerankers, reward models and multi-document QA scorers score candidate documents or responses in one LLM prompt, so each score depends on their order. Such scorers are selected on ranking quality, but their scores determine a decision: what a score threshold retains, a reader answers, or a preference model selects. However, equal ranking quality does not imply equal decisions: on passage reranking, five trained scorers within 0.010 nDCG@10 retain sets that overlap by only 0.66-0.84 when reordered. A published reranker takes the highest retained-set F1 in our comparison and still overlaps by only 0.667. No prompt-time change we test removes that order dependence: the only one that gains ranking quality leaves all three decisions unchanged. Order-consistency SFT (OC-SFT) attenuates it in the weights, training a candidate's score not to depend on the order. It holds ranking quality and leads every decision-stability measure among trained scorers on all three tasks: it flips the reader's answer on 0.125 of permutation pairs against 0.149-0.164 for three other objectives that target order. It is more stable than order-averaged distillation on 12 base models, and one OC-SFT permutation retains sets that overlap more than ten averaged off-the-shelf permutations. A comparison should therefore report what a threshold retains and a reader answers, not ranking quality alone. Code is available at github.com/thomsonreuters/presentation-dependence.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2608.26762")
get_code_for_paper("2608.26762")
have("2608.26762")
Connect an agent — have() is free.