SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2010.09350 · 2020

The efficacy of Neural Planning Metrics: A meta-analysis of PKL on nuScenes

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
nv-tlabs/planning-centric-metrics canonical 3 of 3
FunctionStatusWhere it lives
get_corners Ran nv-tlabs/planning-centric-metrics/planning_centric_metrics/planning_kl.py
pointer only (licence: NOASSERTION) · get_code("04dfccfe539850e2")
get_rot Ran nv-tlabs/planning-centric-metrics/planning_centric_metrics/planning_kl.py
pointer only (licence: NOASSERTION) · get_code("619154382cb66a93")
objects2frame Ran nv-tlabs/planning-centric-metrics/planning_centric_metrics/planning_kl.py
pointer only (licence: NOASSERTION) · get_code("8da01dca7ce6f9b1")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

A high-performing object detection system plays a crucial role in autonomous driving (AD). The performance, typically evaluated in terms of mean Average Precision, does not take into account orientation and distance of the actors in the scene, which are important for the safe AD. It also ignores environmental context. Recently, Philion et al. proposed a neural planning metric (PKL), based on the KL divergence of a planner's trajectory and the groundtruth route, to accommodate these requirements. In this paper, we use this neural planning metric to score all submissions of the nuScenes detection challenge and analyze the results. We find that while somewhat correlated with mAP, the PKL metric shows different behavior to increased traffic density, ego velocity, road curvature and intersections. Finally, we propose ideas to extend the neural planning metric.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2010.09350")
get_code_for_paper("2010.09350")
have("2010.09350")

Connect an agent — have() is free.