SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2104.06982 · ICML · 2021

To Trust or Not to Trust a Regressor: Estimating and Explaining Trustworthiness of Regression Predictions

Kim De Bie, Ana Lucic, Hinda Haned

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In hybrid human-AI systems, users often need to decide whether or not to trust an algorithmic prediction while the true error in the prediction is unknown. To accommodate such settings, we introduce RETRO-VIZ, a method for (i) estimating and (ii) explaining trustworthiness of regression predictions. It consists of RETRO, a quantitative estimate of the trustworthiness of a prediction, and VIZ, a visual explanation that helps users identify the reasons for the (lack of) trustworthiness of a prediction. We find that RETRO-scores negatively correlate with prediction error. In a user study with 41 participants, we confirm that RETRO-VIZ helps users identify whether and why a prediction is trustworthy or not. Cyclepower (4) -0.947 -0.370 -0.690 -0.327 -0.372 -0.282 † -0.389 -0.906 -0.741 -0.593 (0.258) Airfoil (5) -0.880 -0.739 -0.697 -0.495 -0.567 -0.999 -0.199 -0.909 -0.393 -0.653 (0.263) Store sales** (5) -0.062 -0.782 -0.663 -0.614 -0.650 -1.000 -0.534 -0.780 -0.834 -0.658 (0.262) Fish toxicity (6) -0.695 -0.589 -0.563 -0.460 -0.419 -1.000 -0.414 -0.791 -0.456 -0.599 (0.198) Abalone (7) -0.634 -0.427 -0.399 -0.063 -0.109 -0.999 -0.609 -0.552 -0.212 -0.445 (0.295) Autompg (7) -0.566 -0.892 -0.954 -0.017 -0.210 -0.992 -0.542 -0.914 -0.586 -0.630 (0.345) Cal. housing* (8) -0.995 -0.607 -0.964 -0.263 -0.261 -1.000 -0.475 -0.271 -0.433 -0.585 (0.322) Energy eff. (8) -0.980 -0.415 -0.948 -0.472 -0.409 -0.734 -0.225 -0.940 -0.458 -0.620 (0.284) Diabetes* (10) -0.056 -0.237 -0.246 -0.341 -0.305 -0.751 -0.255 -0.460 -0.147 -0.311 (0.200) Wine quality (11) -0.163 -0.289 -0.512 -0.483 -0.234 -0.999 -0.448 -0.311 -0.212 -0.406 (0.255) Boston* (13) -0.348 -0.863 -0.793 -0.864 -0.657 -0.699 -0.433 -0.687 -0.412 -0.640 (0.197) Supercond. (81) -0.946 -0.275 -0.361 -0.546 -0.511 -1.000 -0.463 -0.719 -0.745 -0.618 (0.251) Communities (100) -0.880 -0.323 -0.186 -0.718 -0.408 -0.617 -0.604 -0.620 -0.535 -0.543 (0.210)

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2104.06982")
get_code_for_paper("2104.06982")
have("2104.06982")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2104.06982.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2104.06982.svg)](https://syntology.ai/paper/2104.06982)

Connect an agent — have() is free.