SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2202.14019 · ECCV · 2022

Domain Knowledge-Informed Self-Supervised Representations for Workout Form Assessment

Helge Rhodin, Paritosh Parmar, Amol Gharat

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Maintaining proper form while exercising is important for preventing injuries and maximizing muscle mass gains. Detecting errors in workout form naturally requires estimating human's body pose. However, off-the-shelf pose estimators struggle to perform well on the videos recorded in gym scenarios due to factors such as camera angles, occlusion from gym equipment, illumination, and clothing. To aggravate the problem, the errors to be detected in the workouts are very subtle. To that end, we propose to learn exercise-oriented image and video representations from unlabeled samples such that a small dataset annotated by experts suffices for supervised error detection. In particular, our domain knowledge-informed self-supervised approaches (pose contrastive learning and motion disentangling) exploit the harmonic motion of the exercise actions, and capitalize on the large variances in camera angles, clothes, and illumination to learn powerful representations. To facilitate our self-supervised pretraining, and supervised finetuning, we curated a new exercise dataset, Fitness-AQA (https://github.com/ ParitoshParmar/Fitness-AQA), comprising of three exercises: BackSquat, BarbellRow, and OverheadPress. It has been annotated by expert trainers for multiple crucial and typically occurring exercise errors. Experimental results show that our self-supervised representations outperform off-the-shelf 2D-and 3D-pose estimators and several other baselines. We also show that our approaches can be applied to other domains/tasks such as pose estimation and dive quality assessment. 1. Novel self-supervised approaches that leverage domain knowledge. We initiate the work in the direction of domain knowledge-informed selfsupervised representation learning by developing two contrastive learningbased approaches that capitalize on the harmonic motion of workout actions and the large variance in unlabeled gym videos to learn robust fitness domain-oriented representations (Sec. 3). Our domain knowledge-informed self-supervised representations outperform 2D-and 3D-pose estimators [2,22,27], and various general self-supervised approaches [1,5,17,18] on the task of workout form assessment on existing and our newly introduced datasets. This indicates that future work on representation learning would benefit from us-

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2202.14019")
get_code_for_paper("2202.14019")
have("2202.14019")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2202.14019.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2202.14019.svg)](https://syntology.ai/paper/2202.14019)

Connect an agent — have() is free.