SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2212.08570 · 2022

Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 9 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

FunctionStatusWhere it lives
conv1x1 Ran alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/BNNBaseline/lib/ResNetSource.py
code served (permissive licence) · get_code("2a80220dabcb742a")
conv3x3 Ran alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/BNNBaseline/lib/ResNetSource.py
code served (permissive licence) · get_code("600ff2c45e0de056")
evaluate_metrics Ran alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/BNNBaseline/lib/evaluate.py
code served (permissive licence) · get_code("07b5a3d2cc0c0aa2")
get_file_label_list_for_split Ran alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/BNNBaseline/lib/extract_feat.py
code served (permissive licence) · get_code("de1370a2d5e42ab7")
isclose Ran alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/data-paper/unit_tests.py
code served (permissive licence) · get_code("4c75e29bcf8520c2")
load_dataset_modality Ran alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/SvmBaseline/combine_features.py
code served (permissive licence) · get_code("77a09064753e9740")
model_predict Ran alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/BNNBaseline/lib/evaluate.py
code served (permissive licence) · get_code("c2ec4e1aa5e7224f")
get_file Not yet run alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/analysis_splits/ciab_data.py
code served (permissive licence) · get_code("72d53618367ce071")
resnet18 Not yet run alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/BNNBaseline/lib/ResNetSource.py
code served (permissive licence) · get_code("b35676336da8c1ea")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Recent work has reported that AI classifiers trained on audio recordings can accurately predict severe acute respiratory syndrome coronavirus 2 (SARSCoV2) infection status. Here, we undertake a large scale study of audio-based deep learning classifiers, as part of the UK governments pandemic response. We collect and analyse a dataset of audio recordings from 67,842 individuals with linked metadata, including reverse transcription polymerase chain reaction (PCR) test outcomes, of whom 23,514 tested positive for SARS CoV 2. Subjects were recruited via the UK governments National Health Service Test-and-Trace programme and the REal-time Assessment of Community Transmission (REACT) randomised surveillance survey. In an unadjusted analysis of our dataset AI classifiers predict SARS-CoV-2 infection status with high accuracy (Receiver Operating Characteristic Area Under the Curve (ROCAUC) 0.846 [0.838, 0.854]) consistent with the findings of previous studies. However, after matching on measured confounders, such as age, gender, and self reported symptoms, our classifiers performance is much weaker (ROC-AUC 0.619 [0.594, 0.644]). Upon quantifying the utility of audio based classifiers in practical settings, we find them to be outperformed by simple predictive scores based on user reported symptoms.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2212.08570")
get_code_for_paper("2212.08570")
have("2212.08570")

Connect an agent — have() is free.