All projects
Physical AIPerception

AI for Industry Challenge

Perception for robotic cable insertion in Intrinsic's industrial manipulation challenge.

Simulated wrist camera view during insertion

Overview

Intrinsic's AI for Industry Challenge targets hard, high-impact manufacturing tasks. Our track was inserting SFP and SC cable plugs into NIC-card and patch-panel ports. I led perception for the team, estimating plug and port poses from the arm's cameras so the policy could align and insert. The pipeline combines 3D semantic segmentation for socket localization with fine-grained pin detection and orientation estimation.

A key lesson was data diversity. Our first datasets used a nearly static arm, so every image looked the same. I rebuilt collection around full scripted insertion episodes with randomized component placement, recording labelled samples across the whole approach, alignment and descent.

Highlights

  • 01Multi-camera pipeline: 3D semantic segmentation to localize the socket, then pin detection and orientation estimation.
  • 02ROS 2 dataset-collection node recording labelled samples during full insertion episodes.
  • 03Multi-session pipeline with randomized component positions for view diversity.
  • 04Compact perception network trained for port/plug localization.
  • 05Demonstration recording and conversion to LeRobot format for policy learning.

Command menu

Jump to a section or project