Multimodal Industrial Anomaly Detection via Hybrid Fusion

Yue Wang, Jiangning Zhang, Chengjie Wang, Ran Yi, Yabiao Wang, Jinlong Peng

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 9 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• nomewang/m3dm — marked official in Papers with Code; named in the paper
• nomewang/M3DM

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

2D-based Industrial Anomaly Detection has been widely discussed, however, multimodal industrial anomaly detection based on 3D point clouds and RGB images still has many untouched fields. Existing multimodal industrial anomaly detection methods directly concatenate the multimodal features, which leads to a strong disturbance between features and harms the detection performance. In this paper, we propose Multi-3D-Memory (M3DM), a novel multimodal anomaly detection method with hybrid fusion scheme: firstly, we design an unsupervised feature fusion with patch-wise contrastive learning to encourage the interaction of different modal features; secondly, we use a decision layer fusion with multiple memory banks to avoid loss of information and additional novelty classifiers to make the final decision. We further propose a point feature alignment operation to better align the point cloud and RGB features. Extensive experiments show that our multimodal industrial anomaly detection model outperforms the state-of-the-art (SOTA) methods on both detection and segmentation precision on MVTec-3D AD dataset. Code at github.com/nomewang/M3DM.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2303.00601")
get_code_for_paper("2303.00601")
have("2303.00601")

Connect an agent — have() is free.