SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2306.01196 · ICML · 2023

An Effective Meaningful Way to Evaluate Survival Models

Neeraj Kumar, Russell Greiner, Rajesh Ranganath, Weijie Sun, Shi-Ang Qi, Mahtab Farrokh, Li-Hao Kuan, Ricardo Henao

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

One straightforward metric to evaluate a survival prediction model is based on the Mean Absolute Error (MAE) -the average of the absolute difference between the time predicted by the model and the true event time, over all subjects. Unfortunately, this is challenging because, in practice, the test set includes (right) censored individuals, meaning we do not know when a censored individual actually experienced the event. In this paper, we explore various metrics to estimate MAE for survival datasets that include (many) censored individuals. Moreover, we introduce a novel and effective approach for generating realistic semi-synthetic survival datasets to facilitate the evaluation of metrics. Our findings, based on the analysis of the semi-synthetic datasets, reveal that our proposed metric (MAE using pseudoobservations) is able to rank models accurately based on their performance, and often closely matches the true MAE -in particular, is better than several alternative methods.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2306.01196")
get_code_for_paper("2306.01196")
have("2306.01196")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2306.01196.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2306.01196.svg)](https://syntology.ai/paper/2306.01196)

Connect an agent — have() is free.