Egocentric human data · SenseXperience第一视角人体数据 · SenseXperience
Egocentric data for embodied AI面向具身智能的第一视角数据
SenseXperience captures real-world egocentric and wrist-view human data—video, audio, and IMU in sync—then EmbodiFlow turns raw capture into training-ready datasets.SenseXperience 采集真实世界第一视角与腕部视角人体数据（视频、音频与 IMU 同步），再由 EmbodiFlow 转为可训练数据集。
- Ego RGBEgo RGB2048×1536 · 30 Hz
- Wrist RGB腕部 RGB1920×1080 · 30 Hz
- IMU240 Hz
- Export导出LeRobot · HDF5 · MCAP
Why human data为什么是人体数据
Coverage for long-tail real-world tasks覆盖真实世界长尾任务
As VLA models, Physical AI, and world models advance, embodiment-agnostic human data is a critical training foundation—covering diverse tasks, complex manipulation strategies, and unstructured environments.随着 VLA 模型、物理 AI 与世界模型发展，与本体无关的人体数据是关键训练基础，覆盖多样任务、复杂操作策略与非结构化环境。
Task diversity任务多样性
Human demonstrations span more task types than a fixed robot setup typically covers.人类示范可覆盖的任务类型，通常多于固定机器人平台。
Manipulation detail操作细节
Record grasp, adjustment, and bimanual coordination as they occur in real work.记录真实作业中的抓取、调整与双手协同。
Unstructured settings非结构化场景
Collect in homes, workplaces, and other everyday environments at scale.在家庭、工作场所等日常环境中规模化采集。
Ego + wrist第一视角 + 腕部
Global context and local interaction in one recording一次录制同时获得全局语境与局部交互
An egocentric camera alone often misses contact. Pairing it with wrist cameras records both task context and fine-grained manipulation.仅靠第一视角相机常会错过接触瞬间。与腕部相机组合，可同时记录任务语境与细粒度操作。
Egocentric camera第一视角相机
Where and why在哪里、为什么
Global semantics and task context for where and why a task occurs.提供全局语义与任务语境，说明任务发生的位置与目的。
Wrist camera腕部相机
How it is executed如何执行
Localized interaction detail for how the operation is mechanically performed.提供局部交互细节，说明操作在机械层面如何执行。
Ego-only limitation仅第一视角的局限
Occlusion遮挡
Head-mounted cameras often lose the exact hand–object contact point during close-up work.近距离操作时，头戴相机常看不到手与物体的精确接触点。
Ego-only limitation仅第一视角的局限
Fine-grained detail细粒度信息
Local shape, pose, texture, and material cues are hard to capture from a distance.局部形状、姿态、纹理与材质细节远距离难以捕捉。
Ego-only limitation仅第一视角的局限
Background noise背景噪声
Task-irrelevant pixels can lead models to rely on incidental background cues.与任务无关的画面可能导致模型依赖偶然背景线索。
Capture hardware采集硬件
Modular SenseXperience baselineSenseXperience 模块化基线
SenseXperience is IO’s integrated hardware and software system for real-world human egocentric and UMI-style data. Modules deploy independently or as one system.SenseXperience 是艾欧自研的真实世界人体第一视角与 UMI 风格数据采集软硬件系统。各模块可独立部署，也可组成统一系统。
Ego Unit
Egocentric camera第一视角相机
- Stereo RGB, 65 mm baseline双目 RGB，基线 65 mm
- 2048×1536 @ 30 Hz, global shutter2048×1536 @ 30 Hz，全局快门
- 170° FoV · IMU at 240 Hz170° 视场角 · IMU 240 Hz
- 157 ± 5 g
Wrist Unit
Wrist-mounted camera腕部相机
- RGB 1920×1080 @ 30 Hz, 170° FoVRGB 1920×1080 @ 30 Hz，170° 视场角
- Palm mount: hand articulation掌部佩戴：呈现手掌姿态
- Wrist mount: reduces occlusion on large objects腕部佩戴：抓握大物体时减少遮挡
- 62 ± 5 g
Compute Unit
Main control主控
- 1 TB SSD on-device storage1 TB SSD 本地存储
- 2.5G Ethernet, 5× USB-C2.5G 网口，5× Type-C
- Baseline kit controller基线套件主控
- 214 ± 5 g
From capture to training data从采集到训练数据
Native EmbodiFlow integration与 EmbodiFlow 原生集成
Capture采集
SenseXperience wearable modules record synchronized video, IMU, audio, and interaction signals in real environments.SenseXperience 可穿戴模块在真实环境中同步记录视频、IMU、音频与交互信号。
Process后处理
IO-Agent-Lab algorithms perform timestamp alignment, pose estimation, and gesture recognition.IO-Agent-Lab 算法完成时间对齐、位姿估计与手势识别。
Review & QA审核与质检
EmbodiFlow automated and manual QA filters incomplete or low-quality samples before training.EmbodiFlow 自动与人工质检在训练前过滤不完整或低质量样本。
Export导出
Export LeRobot, HDF5, and MCAP datasets compatible with ROS and common robot-learning workflows.导出 LeRobot、HDF5、MCAP 格式，兼容 ROS 与主流机器人学习工作流。
FAQ
Frequently asked questions常见问题
What modalities can be captured?能采集哪些模态？
The baseline kit captures egocentric video, wrist-view observations, gripper interactions, audio, and IMU, with optional extension to motion capture, tactile sensing, and other modalities.基线方案覆盖第一视角视频、腕部视角、夹爪交互、音频与 IMU，并可按场景扩展动捕、触觉等模态。
How does captured data become training-ready?数据如何变为可训练格式？
Native EmbodiFlow integration supports timestamp alignment, pose estimation, gesture recognition, review, and export to LeRobot, HDF5, and MCAP.与 EmbodiFlow 原生集成，支持时间对齐、位姿估计、手势识别、审核，并导出 LeRobot、HDF5、MCAP。
Can the system be customized?系统可以定制吗？
Yes. The modular architecture supports customization of cameras, grippers, sensors, and processing pipelines.可以。模块化架构支持相机、夹爪、传感器与处理流程定制。
Where is the documentation?文档在哪里？
SenseXperience documentation covers safety, hardware connection, setup, the software workbench, data formats, FAQ, and camera calibration.SenseXperience 文档涵盖安全须知、硬件连接、使用准备、软件工作台、数据格式、常见问题与相机标定。
Configure a capture system for your tasks.按任务配置采集系统。
Describe your tasks and data requirements—IO-AI will help design the SenseXperience configuration.说明任务与数据需求，艾欧智能协助设计 SenseXperience 配置方案。