We lifted 2 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| dannalily/trustscore | canonical | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| remove_matched_string_in_a_list | Ran | dannalily/trustscore/distractor_generator.py code served (permissive licence) · get_code("b79e08d6444a7002") |
| string_matching | Ran | dannalily/trustscore/distractor_generator.py code served (permissive licence) · get_code("d25c90187e2b3ac7") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, prompting a surge in their practical applications. However, concerns have arisen regarding the trustworthiness of LLMs outputs, particularly in closed-book question-answering tasks, where non-experts may struggle to identify inaccuracies due to the absence of contextual or ground truth information. This paper introduces TrustScore, a framework based on the concept of Behavioral Consistency, which evaluates whether an LLMs response aligns with its intrinsic knowledge. Additionally, TrustScore can seamlessly integrate with fact-checking methods, which assesses alignment with external knowledge sources. The experimental results show that TrustScore achieves strong correlations with human judgments, surpassing existing reference-free metrics, and achieving results on par with reference-based metrics.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.12545")
get_code_for_paper("2402.12545")
have("2402.12545")
Connect an agent — have() is free.