We lifted 16 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ehwa009/Co-Speech_Gesture_Generation | reimplementation | 2 of 2 |
| youngwoo-yoon/youtube-gesture-dataset | pwc_unofficial | 1 of 12 |
| pieterwolfert/co-speech-humanoids | reimplementation | 1 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| encodeinputstring | Ran | pieterwolfert/co-speech-humanoids/inference.py pointer only (licence: NONE) · get_code("c4246a111279f4bf") |
| get_pad_mask | Ran | ehwa009/Co-Speech_Gesture_Generation/transformer/models.py code served (permissive licence) · get_code("da23fd901ddaf194") |
| get_subsequent_mask | Ran | ehwa009/Co-Speech_Gesture_Generation/transformer/models.py code served (permissive licence) · get_code("b84290f81844cc1c") |
| unicode_to_ascii | Ran | youngwoo-yoon/youtube-gesture-dataset/script/make_ted_dataset.py code served (permissive licence) · get_code("3b0218c9534b6ee7") |
| draw_skeleton_on_image | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/data_utils.py code served (permissive licence) · get_code("76294a3b3ad8a25b") |
| get_closest_skeleton | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/data_utils.py code served (permissive licence) · get_code("b8fce4e5f8180782") |
| get_vid_from_filename | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/run_openpose.py code served (permissive licence) · get_code("ac8986943eac334d") |
| is_list_empty | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/data_utils.py code served (permissive licence) · get_code("eedbb532217dcb7f") |
| normalize_skeleton | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/motion_preprocessor.py code served (permissive licence) · get_code("47e5434a74faaa2f") |
| normalize_string | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/make_ted_dataset.py code served (permissive licence) · get_code("61474285bf6c43cc") |
| read_sceneinfo | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/run_clip_filtering.py code served (permissive licence) · get_code("b7fe79023c1fb93b") |
| read_skeleton_json | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/run_openpose.py code served (permissive licence) · get_code("5b4fc0454f5233d4") |
| read_subtitle | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/make_ted_dataset.py code served (permissive licence) · get_code("b4d548c56111c912") |
| run_filtering | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/run_clip_filtering.py code served (permissive licence) · get_code("00c7dd2d57e0080f") |
| speech2gesture | Not yet run | pieterwolfert/co-speech-humanoids/inference.py pointer only (licence: NONE) · get_code("8587006901a615dd") |
| video_filter | Not yet run | youngwoo-yoon/youtube-gesture-dataset/script/download_video.py code served (permissive licence) · get_code("79171f8ed610e990") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Co-speech gestures enhance interaction experiences between humans as well as between humans and robots. Existing robots use rule-based speech-gesture association, but this requires human labor and prior knowledge of experts to be implemented. We present a learning-based co-speech gesture generation that is learned from 52 h of TED talks. The proposed end-to-end neural network model consists of an encoder for speech text understanding and a decoder to generate a sequence of gestures. The model successfully produces various gestures including iconic, metaphoric, deictic, and beat gestures. In a subjective evaluation, participants reported that the gestures were human-like and matched the speech content. We also demonstrate a co-speech gesture with a NAO robot working in real time.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1810.12541")
get_code_for_paper("1810.12541")
have("1810.12541")
Connect an agent — have() is free.