Zhen-Hua Ling, Liu, Cong Liu, Jun-Yu Ma, Jia-Chen Gu, Zhigang Chen, Beiduo Chen, Wu Guo
We lifted 14 functions out of this paper's own repositories and ran 12 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| mckysse/msd | canonical | 12 of 14 |
| Function | Status | Where it lives |
|---|---|---|
| swish | Ran | mckysse/msd/transformers/modeling_openai.py code served (permissive licence) · get_code("0f786c407fb1ee4c") |
| angle_defn | Ran | mckysse/msd/transformers/modeling_ctrl.py code served (permissive licence) · get_code("671a619bed5669a3") |
| gelu | Ran | mckysse/msd/transformers/modeling_gpt2.py code served (permissive licence) · get_code("8d23fbe2b99b840b") |
| gelu | Ran | mckysse/msd/transformers/modeling_bert.py code served (permissive licence) · get_code("211753ba29188d4e") |
| gelu | Ran | mckysse/msd/transformers/modeling_tf_bert.py code served (permissive licence) · get_code("9bc31d06256e1691") |
| gelu | Ran | mckysse/msd/transformers/modeling_distilbert.py code served (permissive licence) · get_code("69c6d14de8190cfb") |
| gelu_new | Ran | mckysse/msd/transformers/modeling_bert.py code served (permissive licence) · get_code("77601724cad03f95") |
| gelu_new | Ran | mckysse/msd/transformers/modeling_tf_bert.py code served (permissive licence) · get_code("83257a2b015273fa") |
| load_tf_weights_in_bert | Ran | mckysse/msd/transformers/modeling_bert.py code served (permissive licence) · get_code("0c145728a35789fa") |
| positional_encoding | Ran | mckysse/msd/transformers/modeling_ctrl.py code served (permissive licence) · get_code("59ac77747a12b1cb") |
| scaled_dot_product_attention | Ran | mckysse/msd/transformers/modeling_ctrl.py code served (permissive licence) · get_code("57ebc1447d921c46") |
| swish | Ran | mckysse/msd/transformers/modeling_tf_bert.py code served (permissive licence) · get_code("86d13b9f67274f19") |
| load_tf_weights_in_gpt2 | Not yet run | mckysse/msd/transformers/modeling_gpt2.py code served (permissive licence) · get_code("654e91efc679fe84") |
| load_tf_weights_in_openai_gpt | Not yet run | mckysse/msd/transformers/modeling_openai.py code served (permissive licence) · get_code("42ee168c3ae12e60") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Zero-shot cross-lingual named entity recognition (NER) aims at transferring knowledge from annotated and rich-resource data in source languages to unlabeled and lean-resource data in target languages. Existing mainstream methods based on the teacher-student distillation framework ignore the rich and complementary information lying in the intermediate layers of pre-trained language models, and domaininvariant information is easily lost during transfer. In this study, a mixture of short-channel distillers (MSD) method is proposed to fully interact the rich hierarchical information in the teacher model and to transfer knowledge to the student model sufficiently and efficiently. Concretely, a multi-channel distillation framework is designed for sufficient information transfer by aggregating multiple distillers as a mixture. Besides, an unsupervised method adopting parallel domain adaptation is proposed to shorten the channels between the teacher and student models to preserve domaininvariant features. Experiments on four datasets across nine languages demonstrate that the proposed method achieves new state-of-the-art performance on zero-shot cross-lingual NER and shows great generalization and compatibility across languages and fields.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2212.03506")
get_code_for_paper("2212.03506")
have("2212.03506")
Connect an agent — have() is free.