We lifted 13 functions out of this paper's own repositories and ran 12 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| neuspeech/mad-meg2text | canonical | 12 of 13 |
| Function | Status | Where it lives |
|---|---|---|
| change_sampling_rate | Ran | neuspeech/mad-meg2text/data_augmentation_utils/eeg_aug.py code served (permissive licence) · get_code("434a023482ca36fa") |
| compute_accuracy | Ran | neuspeech/mad-meg2text/utils/model_utils.py code served (permissive licence) · get_code("262947201dc066e9") |
| cosine_similarity | Ran | neuspeech/mad-meg2text/utils/loss.py code served (permissive licence) · get_code("38ede58bd9091f21") |
| cut_signal | Ran | neuspeech/mad-meg2text/data_augmentation_utils/eeg_aug.py code served (permissive licence) · get_code("9b5228012d8f05df") |
| inter_class_relation | Ran | neuspeech/mad-meg2text/utils/loss.py code served (permissive licence) · get_code("b45038694ef6bb75") |
| match_modules | Ran | neuspeech/mad-meg2text/utils/load_model.py code served (permissive licence) · get_code("bf84bf80d676a07a") |
| match_modules_string | Ran | neuspeech/mad-meg2text/utils/load_model.py code served (permissive licence) · get_code("1c6ebf4c6f313f2c") |
| multi_segment_speed_change | Ran | neuspeech/mad-meg2text/data_augmentation_utils/eeg_aug.py code served (permissive licence) · get_code("5f0ebfcfa62171f4") |
| pad_multiple | Ran | neuspeech/mad-meg2text/utils/brain_module.py code served (permissive licence) · get_code("4ecb4c91cf8f4446") |
| pearson_correlation | Ran | neuspeech/mad-meg2text/utils/loss.py code served (permissive licence) · get_code("d00eea76cc523c64") |
| preprocess_text | Ran | neuspeech/mad-meg2text/evaluation.py code served (permissive licence) · get_code("2908e3c7f6faafb0") |
| projection_module | Ran | neuspeech/mad-meg2text/utils/model_utils.py code served (permissive licence) · get_code("db94765af5a0e4dd") |
| concatenate_images | Not yet run | neuspeech/mad-meg2text/wandb_callback.py code served (permissive licence) · get_code("ac8331082a4cff98") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Deciphering language from brain activity is a crucial task in brain-computer interface (BCI) research. Non-invasive cerebral signaling techniques including electroencephalography (EEG) and magnetoencephalography (MEG) are becoming increasingly popular due to their safety and practicality, avoiding invasive electrode implantation. However, current works under-investigated three points: 1) a predominant focus on EEG with limited exploration of MEG, which provides superior signal quality; 2) poor performance on unseen text, indicating the need for models that can better generalize to diverse linguistic contexts; 3) insufficient integration of information from other modalities, which could potentially constrain our capacity to comprehensively understand the intricate dynamics of brain activity. This study presents a novel approach for translating MEG signals into text using a speech-decoding framework with multiple alignments. Our method is the first to introduce an end-to-end multi-alignment framework for totally unseen text generation directly from MEG signals. We achieve an impressive BLEU-1 score on the \textit{GWilliams} dataset, significantly outperforming the baseline from 5.49 to 6.86 on the BLEU-1 metric. This improvement demonstrates the advancement of our model towards real-world applications and underscores its potential in advancing BCI research. Code is available at $\href{https://github.com/NeuSpeech/MAD-MEG2text}{https://github.com/NeuSpeech/MAD-MEG2text}$.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2406.01512")
get_code_for_paper("2406.01512")
have("2406.01512")
Connect an agent — have() is free.