SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2306.07087 · ICLR · 2023

MaskedFusion360: Reconstruct LiDAR Data by Querying Camera Features

Royden Wagner, Marvin Klemp, Carlos Fernandez Lopez

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In self-driving applications, LiDAR data provides accurate information about distances in 3D but lacks the semantic richness of camera data. Therefore, state-ofthe-art methods for perception in urban scenes fuse data from both sensor types. In this work, we introduce a novel self-supervised method to fuse LiDAR and camera data for self-driving applications. We build upon masked autoencoders (MAEs) and train deep learning models to reconstruct masked LiDAR data from fused Li-DAR and camera features. In contrast to related methods that use birds-eye-view representations, we fuse features from dense spherical LiDAR projections and features from fish-eye camera crops with a similar field of view. Therefore, we reduce the learned spatial transformations to moderate perspective transformations and do not require additional modules to generate dense LiDAR representations. Code is available at: https://github.com/KIT-MRT/masked-fusion-360 Camera encoder MAE decoder MAE LiDAR encoder

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2306.07087")
get_code_for_paper("2306.07087")
have("2306.07087")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2306.07087.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2306.07087.svg)](https://syntology.ai/paper/2306.07087)

Connect an agent — have() is free.