SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2404.09027 · 2024

MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Experts

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 13 functions out of this paper's own repositories and ran 8 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
mediabrain-sjtu/ming canonical 3 of 3
mediabrain-sjtu/medicalgpt-zh pwc_unofficial 5 of 10
FunctionStatusWhere it lives
check_target_module_exists Ran mediabrain-sjtu/medicalgpt-zh/ming/model/utils.py
code served (permissive licence) · get_code("ce4b943c1f4622e6")
convert_to_json Ran mediabrain-sjtu/medicalgpt-zh/ming/eval/eval_gpt4.py
code served (permissive licence) · get_code("f315af9021bb4de9")
get_chunk Ran mediabrain-sjtu/medicalgpt-zh/ming/eval/model_diverse_gen.py
code served (permissive licence) · get_code("42a46570620cd9fa")
get_model_name_from_path Ran mediabrain-sjtu/medicalgpt-zh/ming/utils.py
code served (permissive licence) · get_code("fa6d2859d3461c9e")
normalize Ran mediabrain-sjtu/ming/ming/eval/eval_em.py
code served (permissive licence) · get_code("0133c7875f5508e2")
normalize_dfrac Ran mediabrain-sjtu/ming/ming/eval/eval_em.py
code served (permissive licence) · get_code("ceb9b19eaecb1183")
normalize_frac Ran mediabrain-sjtu/ming/ming/eval/eval_em.py
code served (permissive licence) · get_code("75cb6a84ff49f59b")
split_list Ran mediabrain-sjtu/medicalgpt-zh/ming/eval/model_diverse_gen.py
code served (permissive licence) · get_code("076c252c52cbb161")
create_data_loader Not yet run mediabrain-sjtu/medicalgpt-zh/ming/eval/model_diverse_gen.py
code served (permissive licence) · get_code("8948016310b7f832")
create_mixoflora_module Not yet run mediabrain-sjtu/medicalgpt-zh/ming/model/utils.py
code served (permissive licence) · get_code("ff08a7b61f0ff773")
get_default_conv_template Not yet run mediabrain-sjtu/medicalgpt-zh/ming/conversations.py
code served (permissive licence) · get_code("6f3e31642958b2ee")
get_mixoflora_model Not yet run mediabrain-sjtu/medicalgpt-zh/ming/model/utils.py
code served (permissive licence) · get_code("80d3d94e5c8311ac")
load_pretrained_model Not yet run mediabrain-sjtu/medicalgpt-zh/ming/model/builder.py
code served (permissive licence) · get_code("e9663a782ec03800")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Large language models like ChatGPT have shown substantial progress in natural language understanding and generation, proving valuable across various disciplines, including the medical field. Despite advancements, challenges persist due to the complexity and diversity inherent in medical tasks which often require multi-task learning capabilities. Previous approaches, although beneficial, fall short in real-world applications because they necessitate task-specific annotations at inference time, limiting broader generalization. This paper introduces MING-MOE, a novel Mixture-of-Expert~(MOE)-based medical large language model designed to manage diverse and complex medical tasks without requiring task-specific annotations, thus enhancing its usability across extensive datasets. MING-MOE employs a Mixture of Low-Rank Adaptation (MoLoRA) technique, allowing for efficient parameter usage by maintaining base model parameters static while adapting through a minimal set of trainable parameters. We demonstrate that MING-MOE achieves state-of-the-art (SOTA) performance on over 20 medical tasks, illustrating a significant improvement over existing models. This approach not only extends the capabilities of medical language models but also improves inference efficiency.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2404.09027")
get_code_for_paper("2404.09027")
have("2404.09027")

Connect an agent — have() is free.