HubEgo-100K
HubEgo-100K is an open dataset of egocentric video of skilled manual work for robot learning. Release v1.0 has 10,124 hours of 1080p video with synchronized 100 Hz IMU data from one head-mounted phone camera, recorded by 12,227 contributors in 15,548 environments. It is free under CC BY 4.0 on Hugging Face, and the program releases data in stages toward 100,000 hours.

Key facts (v1.0)
| Metric | Value | Source |
|---|---|---|
| Hours of video (v1.0) | 10,124 hours | arXiv paper |
| Total video frames | 1,093,194,072 frames | HF dataset card |
| Episodes | 72,754 episodes | arXiv paper |
| Median episode length | 4.9 min | HF dataset card |
| Maximum episode length | 30 min | HF dataset card |
| Unique contributors | 12,227 contributors | arXiv paper |
| Unique environments | 15,548 environments | arXiv paper |
| Setting types | 11 setting types | arXiv paper |
| Hours in workplace settings | 83% | arXiv paper |
| Work categories | 30 work categories | arXiv paper |
| Task types | 56 task types | arXiv paper |
| Action families | 35 action families | arXiv paper |
| Unique task descriptions | 62,000+ | HF dataset card |
| Sampled frames with at least one hand | 94.7% | arXiv paper |
| Sampled frames with both hands | 76.3% | arXiv paper |
| On-task share (model estimate) | 96.9% | HF dataset card |
| Hours annotated as hard | 34% | HF dataset card |
| Hours annotated as medium | 47% | HF dataset card |
| Video specification | 1920x1080, H.264, 30 fps | HF dataset card |
| IMU sample rate | 100 Hz | arXiv paper |
| Capture hardware | one head-mounted phone camera | arXiv paper |
| Format | LeRobot v3 + MCAP | HF dataset card |
| License | CC BY 4.0 | arXiv paper |
| Access process | short form, automatic approval | arXiv paper |
| Program target (staged releases) | toward 100,000 hours | arXiv paper |
What the recordings show
Every episode in HubEgo-100K v1.0 is filmed by a Hub contributor performing their own unscripted manual work in their own workplace or home, using their own tools. Tasks span bricklaying, electrical wiring, plumbing, industrial sewing, lathe machining, watch repair and phone board microsoldering.
Across the release, 83% of hours take place in workplace settings and 18 work categories each hold more than 200 hours, with the largest category accounting for 13.1% of hours. No single action family exceeds 10% of hours.
| Setting | What it covers |
|---|---|
| Trades done in homes | A trade or craft done in a home: a home sewing table or repair bench, or a tradesperson working in someone's home (plumbing, electrical, appliance and bike repair, jewelry, printing) |
| Workshop & repair bench | A dedicated workshop or repair shop |
| Construction & renovation site | Building sites and homes under renovation |
| Salon & personal care | Hair, nail, beauty and pet grooming, in a salon or at home |
| Commercial kitchen | Restaurant, bakery and food-production kitchens |
| Shop floor | Shops and supermarkets |
| Warehouse & logistics | Stock rooms, warehouses, packing |
| Service & front of house | Offices, clinics, hotels, reception |
| Farm & garden centre | Fields, greenhouses, nurseries |
| Factory & industrial | Production lines, factory floors |
| Household | Home DIY and gardening, and service tasks done at home (laundry, room cleaning) |



Annotations
Every episode has a row in meta/annotations.parquet, keyed by episode_index:
| Field | Content |
|---|---|
description |
Short task description, for example "Machine-sew and overlock sleeveless top seams" |
task_instruction |
One or two sentences describing the main goal, actions and objects in order |
action_family |
Main manual action across 35 families (cut, sew, assemble, fasten, clean) |
skill_group |
Skill area the task exercises across 18 groups (such as Electronics & Diagnostics) |
difficulty |
Easy, medium or hard (34% hard, 47% medium by hours) |
environment |
Setting type where the episode was filmed (11 setting types) |
on_task_share |
Estimated share of episode time spent actively doing the task |
meta/subtasks.parquet and meta/subtask_occurrences.parquet expose the step-level instructions in the LeRobot v3 subtask table format.
Data format and loading
All head-mounted monocular recordings live in the mono/ directory as a LeRobot v3 dataset, paired with native MCAP sensor archives and per-episode parquet metadata:
- Video (
mono/videos/observation.images.main/): one MP4 per episode (1920x1080, H.264, 30 fps). 84% of episodes are the phone's original stream; the rest were recorded at 60 fps and converted to 30 fps. - Frame data (
mono/data/): LeRobot v3 parquet with one row per video frame.observation.stateholds frame-aligned accelerometer (m/s2, gravity included) and gyroscope (rad/s) values. - Native sensors (
mono/mcap/): one MCAP file per episode storing native accelerometer, gyroscope and device motion streams (plus magnetometer, heading and barometer where available), indexed by archive, member and byte offset incatalog.parquet. - Catalog (
mono/catalog.parquet): task type, work category, pseudonymous contributor and recording ids, duration, device model, lens, field of view, hand visibility estimates and synchronization signals.
Read the catalog and annotation tables directly without downloading video:
import pandas as pd
cat = pd.read_parquet("hf://datasets/HubLabs/HubEgo-100K/mono/catalog.parquet")
ann = pd.read_parquet("hf://datasets/HubLabs/HubEgo-100K/mono/meta/annotations.parquet")
Load a small subset of episodes locally with LeRobot v3:
import glob
import pandas as pd
from huggingface_hub import hf_hub_download, snapshot_download
from lerobot.datasets.lerobot_dataset import LeRobotDataset
REPO, G = "HubLabs/HubEgo-100K", "mono"
EPISODES = [503, 203, 454]
local = snapshot_download(
REPO,
repo_type="dataset",
local_dir="HubEgo-100K",
allow_patterns=[f"{G}/meta/**", f"{G}/catalog.parquet"],
)
root = f"{local}/{G}"
meta = pd.concat(map(pd.read_parquet, glob.glob(f"{root}/meta/episodes/*/*.parquet"))).set_index("episode_index")
cat = pd.read_parquet(f"{root}/catalog.parquet").set_index("episode_index")
for e in EPISODES:
c, f = meta.loc[e, "data/chunk_index"], meta.loc[e, "data/file_index"]
hf_hub_download(REPO, f"{G}/data/chunk-{c:03d}/file-{f:03d}.parquet", repo_type="dataset", local_dir=local)
hf_hub_download(REPO, f"{G}/{cat.loc[e].video_path}", repo_type="dataset", local_dir=local)
ds = LeRobotDataset(REPO, root=root, episodes=EPISODES)
Access and license
HubEgo-100K is released under CC BY 4.0. It is free to use, share and adapt for any purpose, including commercial use, with credit:
HubEgo-100K, Hub Labs (Hub Data, Inc.), https://huggingface.co/datasets/HubLabs/HubEgo-100K, CC BY 4.0
Access on Hugging Face uses short form, automatic approval. The catalog and annotation tables can be inspected without downloading video, and individual episodes can be fetched by index.
How the data was collected and curated
- Contributors recorded a named, unscripted task with one head-mounted phone camera during their own work, in their own workplace or home.
- A dense vision-language pass segmented every recording in time and retained stretches where the wearer's hands perform the task (96.9% on-task time).
- Human review decisions, where available, overrode automated segmentation; frame timing, repeated frames and frame-rate drift were logged per episode in
catalog.parquet. - Privacy filtering ran on every episode before release.
- Hours were rebalanced across the 56 task types so no single task dominates v1.0.
Privacy
Every contributor agreed that their recordings may be used to train and evaluate AI and robotics systems. Before release, Hub removed:
- Stretches where a child, another person's face or private text (documents, screens, cards, license plates) was detected, cut with a 5-second margin, as well as recordings where faces fill a large share of the footage.
- Episodes in which the video annotation detected a minor.
- Care tasks performed on another person.
- Audio, location, device identifiers and contributor identity (identifiers in the release are pseudonymous strings; phone make and model remain as capture metadata).
Limitations
- Machine-generated annotations: Descriptions, instructions, action families, difficulty labels, hand visibility and on-task shares were generated by a vision-language model and have not been audited by human reviewers.
- Monocular single-camera setup: v1.0 is recorded from one head-mounted phone camera with synchronized IMU. It does not include stereo depth, multi-camera views or ground-truth 3D hand poses.
- Re-encoded subset: 84% of episodes are the phone's original stream; the rest were recorded at 60 fps and converted to 30 fps. Per-episode encoding and timing signals are stored in
catalog.parquet. - Audio removed: Audio tracks are stripped from every episode as part of privacy filtering.
- Staged release and benchmarks: v1.0 contains 10,124 hours, the first stage of a program releasing data toward 100,000 hours. Model benchmarks are planned for later versions of the paper.
Cite HubEgo-100K
Amirian, J., Kiani, A., Schaeffer, L., Sprecher, T., Taha, M., Yadati, K. (2026). HubEgo-100K: In-the-Wild Egocentric Video of Skilled Work for Robot Learning. arXiv:2611.21938.
@misc{amirian2026hubego100kinthewildegocentricvideo,
title = {{HubEgo-100K}: In-the-Wild Egocentric Video of Skilled Work for Robot Learning},
author = {Javad Amirian and Armin Kiani and Loïc Schaeffer and Tim Sprecher and Mohamed Taha and Karthik Yadati},
year = {2026},
eprint = {2611.21938},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
doi = {10.48550/arXiv.2611.21938},
url = {https://arxiv.org/abs/2611.21938}
}
@dataset{hubego100k2026,
author = {{Hub Labs (Hub Data, Inc.)}},
title = {{HubEgo-100K}},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/datasets/HubLabs/HubEgo-100K}
}Changelog
- v1.0: First release: 10,124 hours across 72,754 episodes, 12,227 contributors and 15,548 environments.