Chirag Agarwal, Sriparna Saha, Akash Ghosh, Srivarshinee Sridhar, Raghav Kaushik, Muhsin Muhsin
We lifted 16 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| AikyamLab/clinic | canonical | 0 of 16 |
| Function | Status | Where it lives |
|---|---|---|
| clean_and_extract_choice | Not yet run | AikyamLab/clinic/evaluation/colloquial/colloquial_eval.py code served (permissive licence) · get_code("0bc2a2920d72eb2b") |
| clean_response | Not yet run | AikyamLab/clinic/evaluation/adverserial/adv_eval.py code served (permissive licence) · get_code("eee9780706ca91ac") |
| clean_response | Not yet run | AikyamLab/clinic/evaluation/consistency/consistency_eval.py code served (permissive licence) · get_code("1b68e4a2fe2400a0") |
| clean_response | Not yet run | AikyamLab/clinic/evaluation/disparagement/disp_eval.py code served (permissive licence) · get_code("d3064a9f7b1e0b34") |
| clean_response | Not yet run | AikyamLab/clinic/evaluation/exaggerated_safety/exagsaf.py code served (permissive licence) · get_code("d99cb6304d112c48") |
| get_embeddings | Not yet run | AikyamLab/clinic/evaluation/adverserial/adv_eval.py code served (permissive licence) · get_code("e51b60b9add827b6") |
| get_embeddings | Not yet run | AikyamLab/clinic/evaluation/consistency/consistency_eval.py code served (permissive licence) · get_code("daf8bacd15f6bdea") |
| get_score | Not yet run | AikyamLab/clinic/evaluation/colloquial/colloquial_eval.py code served (permissive licence) · get_code("7fdf75bfa127d457") |
| get_score | Not yet run | AikyamLab/clinic/evaluation/disparagement/disp_eval.py code served (permissive licence) · get_code("a5fb7b464744eb20") |
| get_score | Not yet run | AikyamLab/clinic/evaluation/exaggerated_safety/exagsaf.py code served (permissive licence) · get_code("987ba214da993053") |
| get_score_fct | Not yet run | AikyamLab/clinic/evaluation/hallucinations/hallu_eval.py code served (permissive licence) · get_code("018247f0e18ff6f9") |
| get_similarity_score | Not yet run | AikyamLab/clinic/evaluation/adverserial/adv_eval.py code served (permissive licence) · get_code("4612dca95f28aeaf") |
| gpt_eval | Not yet run | AikyamLab/clinic/evaluation/disparagement/disp_eval.py code served (permissive licence) · get_code("796b7eaaef2c4dd5") |
| process_csv_file | Not yet run | AikyamLab/clinic/evaluation/fairness-preference/fair_pref_eval.py code served (permissive licence) · get_code("ed0ccb3bae64b137") |
| remove_descent_variants | Not yet run | AikyamLab/clinic/evaluation/fairness-preference/fair_pref_eval.py code served (permissive licence) · get_code("d2a53c7af3589e7d") |
| replacements | Not yet run | AikyamLab/clinic/evaluation/fairness-stereotype/fair_stereo_eval.py code served (permissive licence) · get_code("f99ee61006d6c28d") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Integrating language models (LMs) in healthcare systems holds great promise for improving medical workflows and decision-making. However, a critical barrier to their real-world adoption is the lack of reliable evaluation of their trustworthiness, especially in multilingual healthcare settings. Existing LMs are predominantly trained in high-resource languages, making them ill-equipped to handle the complexity and diversity of healthcare queries in mid-and low-resource languages, posing significant challenges for deploying them in global healthcare contexts where linguistic diversity is key. In this work, we present CLINIC, a Comprehensive Multilingual Benchmark to evaluate the trustworthiness of language models in healthcare. CLINIC systematically benchmarks LMs across five key dimensions of trustworthiness: truthfulness, fairness, safety, robustness, and privacy, operationalized through 18 diverse tasks, spanning 15 languages (covering all the major continents), and encompassing a wide array of critical healthcare topics like disease conditions, preventive actions, diagnostic tests, treatments, surgeries, and medications. Our extensive evaluation reveals that LMs struggle with factual correctness, demonstrate bias across demographic and linguistic groups, and are susceptible to privacy breaches and adversarial attacks. By highlighting these shortcomings, CLINIC lays the foundation for enhancing the global reach and safety of LMs in healthcare across diverse languages. The GitHub page for this project can be found in https://github.com/AikyamLab/clinic.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2512.11437")
get_code_for_paper("2512.11437")
have("2512.11437")
Connect an agent — have() is free.