We lifted 2 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| iip-sogang/olkavs-avspeech | canonical | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| load_label | Ran | iip-sogang/olkavs-avspeech/preprocess.py pointer only (licence: NONE) · get_code("5970aa426a56c1f4") |
| single_infer | Ran | iip-sogang/olkavs-avspeech/inference.py pointer only (licence: NONE) · get_code("141e5c52bc003139") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Inspired by humans comprehending speech in a multi-modal manner, various audio-visual datasets have been constructed. However, most existing datasets focus on English, induce dependencies with various prediction models during dataset preparation, and have only a small number of multi-view videos. To mitigate the limitations, we recently developed the Open Large-scale Korean Audio-Visual Speech (OLKAVS) dataset, which is the largest among publicly available audio-visual speech datasets. The dataset contains 1,150 hours of transcribed audio from 1,107 Korean speakers in a studio setup with nine different viewpoints and various noise situations. We also provide the pre-trained baseline models for two tasks, audio-visual speech recognition and lip reading. We conducted experiments based on the models to verify the effectiveness of multi-modal and multi-view training over uni-modal and frontal-view-only training. We expect the OLKAVS dataset to facilitate multi-modal research in broader areas such as Korean speech recognition, speaker recognition, pronunciation level classification, and mouth motion analysis.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2301.06375")
get_code_for_paper("2301.06375")
have("2301.06375")
Connect an agent — have() is free.