A Web-Scale Data Engine for Video-to-Action Robot Learning through Egocentric Views
EgoInfinity is a data engine that automatically curates and processes any-view video clips into high-quality 4D hand-object-interaction (HOI) sequences, and transforms them into any other views, including egocentric. EgoInfinity also provides a framework to retarget processed human motions to motions on any robots.
Click any clip in the Browse section below to load it into the 3D viewer and inspect panels.
Click any clip in the Browse section below to load its 3D scene here. Embedded viser session with the same renderer, GUI controls, and layers as exo_pipeline.py --load-cache. One viser process at a time; selecting an episode swaps it.
Click any clip in the Browse section below to populate these panels. Annotated stream (hand skeleton), raw clip, MoGe-2 depth, MEMFOF flow, plus per-frame contact / grasp / motion / trust signals from the 6-DoF tracker.
Search, filter, and sort all favorites. Switch between thumbnail and table view. Click any clip to load it into the inspect and viser panels above.
| Clip | Actor | Source | Objects | Frames | Duration | Grasp |
|---|
Statistics from Action100M annotations.
The vocabulary, sunburst rings, and n-gram frequencies above are from Action100M's human-written action labels and detailed descriptions. Each EgoInfinity clip carries its source action_brief + action_detailed + summary verbatim in scene.json.action100m_metadata.
Please cite Action100M alongside EgoInfinity if you use these statistics:
@article{chen2026action100m,
title = {Action100M: A Large-scale Video Action Dataset},
author = {Chen, Delong and Kasarla, Tejaswi and Bang, Yejin
and Shukor, Mustafa and Chung, Willy and Yu, Jade
and Bolourchi, Allen and Moutakanni, Théo
and Fung, Pascale},
journal = {arXiv preprint arXiv:2601.10592},
year = {2026}
}
Public-facing milestones for the EgoInfinity dataset and pipeline. Status: shipped · active · queued.
Run the pipeline across the full Action100M corpus, growing the curated subset from 106 clips toward the 100M-clip headline number.
We'll email you when there's an update. Unsubscribe anytime.
We usually respond within a few business days.
Built by the Rice RobotPI Lab together with collaborators at the Robotics and AI Institute.