Sourcing multi-view manipulation

Quick facts

DROID

76,000 trajectories with 3 camera views (2 exterior + 1 wrist) at 180×320 / 15 FPS / 350 hours / Franka Panda (2024)

Open X-Embodiment (subset)

1M+ trajectories pooled across 22 embodiments; some contributing datasets ship with multi-camera streams in RLDS format (2023).

Why multi-view matters

Wrist cameras solve close-up contact; exterior cameras give scene context; pairing them lets policies generalize beyond what either view supports alone.

Spec checklist

Camera intrinsics + extrinsics, sync offset (microsecond precision), per-view resolution and codec, object identity across views, time-aligned action stream.

Comparison

Dataset requirements

Buyers should specify egocentric, wrist, and third-person video captured together, accepted scenes in tabletop, bin, shelf, and workbench manipulation, consent rules, and a pilot package large enough to include accepted clips, rejected clips, metadata, consent/provenance artifacts, and edge cases for buyer review. The exact metadata package should include: camera calibration, sync offset, object set, task phase, and consent artifact ^[1]. The Datasheets framework spells out which dataset-documentation questions matter before any commercial training program begins.

"The machine learning community currently has no standardized process for documenting datasets, which can lead to severe consequences in high-stakes domains."

Best-fit buyers

The strongest fit is teams needing scene context and first-person interaction in one dataset ^[3]. It can also work as a smaller eval set before a larger net-new capture program when the buyer defines the held-out tasks, failure cases, and acceptance rubric in writing.

Multi-view manipulation sample package

A credible multi-view manipulation supplier should provide a sample package that includes raw files, a manifest, capture context, and these critical metadata fields: camera calibration, sync offset, object set, task phase, and consent artifact ^[4]. The buyer should be able to inspect accepted and rejected examples to confirm whether egocentric, wrist, and third-person video captured together actually appears in tabletop, bin, shelf, and workbench manipulation, not just trust a verbal description of the inventory.

Multi-view manipulation licensing check

The licensing review for multi-view manipulation dataset should confirm whether the data is off-the-shelf or net-new, whether it can be used for commercial model training, whether contributors or sites consented, and whether the supplier can reproduce the same rights package for the full delivery ^[5]. A well-scoped buyer reviews multiple supplier samples before approving scale; without those checks, an apparently useful dataset can become a legal or procurement blocker.

Related pages

Use these to move from category-level context into specific task, dataset, format, and comparison detail.

External references and source context

1. Datasheets for Datasets

   Supports the dataset-requirements framework dimensions: dataset motivation, composition, collection process, recommended uses, and license review.

   arXiv ↩
2. Datasheets for Datasets

   Datasheets for Datasets defines the consent, provenance, and intended-use questions buyers must ask before commercial training; quoted verbatim in the dataset-requirements section.

   arXiv ↩
3. Open X-Embodiment: Robotic Learning Datasets and RT-X Models

   Open X-Embodiment establishes the cross-embodiment robotics pretraining baseline — a useful reference for buyer fit when mapping a deployment-specific dataset onto a generalist policy.

   arXiv ↩
4. Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI

   Data Cards capture dataset origins, development, intent, and ethical considerations buyers can attach to each delivered batch for procurement audit.

   arXiv ↩
5. encord

   Commercial vendors deliver licensed dataset collection programs with explicit contributor consent, rights, and per-batch documentation buyers can audit before scale.

   encord.com ↩

FAQ