> ## Documentation Index > Fetch the complete documentation index at: https://docs.holos.io/llms.txt > Use this file to discover all available pages before exploring further. # Export as HDF5 > One HDF5 file per capture, laid out to published conventions so it opens in h5py, xarray, MATLAB and robomimic without custom code. **Download** hands back the raw `.capture` package: a folder of CSVs, JSON and video. **Export as HDF5** writes the same data as one `.h5` file with typed datasets, a shared time axis and the media embedded, for notebooks, robotics pipelines and anyone who would rather not parse CSVs. ## Exporting Right-click a completed capture in the Captures table or tile grid and choose **Export as HDF5**. A worker writes the file, so the toast follows the job (waiting for a worker, writing, uploading) and the download starts when it is done. A capture whose export is still current hands back the same file without a new job; editing the transcript makes the last export stale, and the next ask writes a fresh one. Picture captures have no package and do not offer the export. ## The conventions The group tree is built from three published conventions, each one checkable with a tool you already have, rather than a layout of our own. > **HDF Group specifications (file level)** > > * **Dimension Scales.** Every axis of every gridded dataset carries a scale: `time` for the sample axis, `pose_component` (`x y z qx qy qz qw`), `joint`, `tracked_object`, `state_component`. This is the mechanism netCDF-4 is built on, so xarray, netCDF4 and MATLAB read the axes without custom code. > * **Image and Palette Specification 1.2** on image datasets (`CLASS="IMAGE"`, `IMAGE_VERSION="1.2"`, `IMAGE_SUBCLASS="IMAGE_TRUECOLOR"`, `INTERLACE_MODE="INTERLACE_PIXEL"`), so HDFView renders them. > * **Opaque datatypes with a MIME tag** for embedded container bytes: the camera `.mov` files are `H5T_OPAQUE` tagged `video/quicktime`, the microphone `.wav` is tagged `audio/wav`. Readers know what they are holding without a sidecar. > * Chunked storage with gzip, shuffle and Fletcher-32 checksums on anything that is not tiny; one self-contained file with no external links; strings as variable-length UTF-8. > **CF-1.11 attributes (metadata level)** > > * Root: `Conventions="CF-1.11, HolosCapture-1.0"`, `title`, `institution`, `source` (device and app), `history` (exporter version and time), `references` (this page), `capture_id`, `organization_id`, `modality`, `capture_start`. > * Every numeric dataset: `units` (`m`, `s`, `ns`, `1`), `long_name`, and `_FillValue` (NaN) where a stream can be absent. > * `time`: float64 with `units="seconds since "` and `calendar="proleptic_gregorian"`, so xarray decodes it to datetimes. `timestamp_ns` beside it is int64 nanoseconds since the Unix epoch, the ROS 2 convention. > * The coordinate frame is stated once on the demo group and never left implicit: `frame="arkit_world"`, `handedness="right"`, `up_axis="Y"`, `quaternion_order="xyzw"`, `length_unit="m"`. > **robomimic dataset layout (domain level)** > > robomimic's HDF5 format is the de facto shape for learning-from-demonstration data (robosuite, MimicGen, LIBERO and ManiSkill exporters all read or write it). One capture is one demo: > > * `/data` with attrs `total` (sample count) and `env_args` (JSON: `env_name="holos_capture"`, `type=99` meaning no simulator behind this, and the capture's metadata under `env_kwargs`). > * `/data/demo_0` with attrs `num_samples` and the frame conventions above, holding `obs/*`, `actions`, `states`, `time` and `timestamp_ns`. > * `/mask/all` listing `demo_0`, so `filter_by_attribute="all"` works. > > Everything Holos-specific lives outside `/data`, under `/holos` and `/media`, so a stock robomimic loader never sees it. ## What is inside ```text theme={null} /pose_component, /joint, /tracked_object shared coordinate scales (string arrays) /data attrs: total, env_args /data/demo_0 attrs: num_samples, frame, handedness, up_axis, quaternion_order, length_unit, master_clock, resampling /data/demo_0/time float64 [T] seconds since capture start (dimension scale) /data/demo_0/timestamp_ns int64 [T] epoch nanoseconds /data/demo_0/obs/timestamp float64 [T] same values as time, as an obs key /data/demo_0/obs/device_pose float64 [T, 7] headset pose, ARKit world frame /data/demo_0/obs/hand_left_joints_world float64 [T, J, 7] J = 26 ARKit HandSkeleton joints (names on the joint scale) /data/demo_0/obs/hand_right_joints_world float64 [T, J, 7] /data/demo_0/obs/hand_left_joints_local float64 [T, J, 7] relative to the wrist anchor /data/demo_0/obs/hand_right_joints_local float64 [T, J, 7] /data/demo_0/obs/object_pose float64 [T, K, 7] tracked object anchors (ids and names on the tracked_object scale) /data/demo_0/obs/hand_{left,right}_landmarks_3d float64 [T, 21, 3] iPhone: Vision landmarks in metres (names on the landmark scale) /data/demo_0/obs/hand_{left,right}_landmarks_confidence float64 [T, 21] iPhone: per-landmark confidence, 0 = not seen /data/demo_0/obs/imu_accel, imu_gyro float64 [T, 3] iPhone or derived: user acceleration (m s-2), rotation rate (rad s-1), device frame /data/demo_0/actions float64 [T, 7] dominant-hand wrist delta (see below) /data/demo_0/states float64 [T, S] device pose + wrist poses, layout on the state_component scale /holos/raw/device_pose/{time, timestamp_ns, pose} every sample, on its own clock /holos/raw/hand_pose_world/{left,right}/{time, timestamp_ns, joints} /holos/raw/hand_pose_local/{left,right}/{time, timestamp_ns, joints} /holos/raw/object_pose//{time, timestamp_ns, pose} /holos/raw/imu/{time, timestamp_ns, accel, gyro} iPhone / derived: m s-2 and rad s-1, device frame; attr source /holos/raw/hand_landmarks_3d/{left,right}/{time, timestamp_ns, points, confidence, hand_confidence} iPhone: Vision's 21 landmarks in metres /holos/raw/hand_landmarks_2d/{left,right}/{time, timestamp_ns, points, confidence, hand_confidence} iPhone: normalized image u, v /holos/files/ opaque bytes, MIME-tagged any package file nothing above types /holos/metadata/session JSON string metadata.json verbatim /holos/metadata/stereo_calibration JSON string /holos/metadata/thumbnail uint8 [H, W, 3] Image Spec 1.2 /holos/transcript/segments compound (start_s, end_s, text, is_final) /holos/transcript/tokens compound (segment, start_s, end_s, text) /holos/transcript/full_text string /holos/events/procedure_steps compound (index, step_id, title, t_start, t_end, start_sample, end_sample) /holos/events/procedure_steps_json JSON string events/procedure_steps.json verbatim /holos/vlm/output compound (t_mono, t_wall, prompt, output) /media/ opaque bytes, video/quicktime; attrs codec, width, height, fps, duration_s. camera_left, camera_right and camera_sbs on Vision Pro; camera_back on iPhone /media/audio opaque bytes, audio/wav; attrs sample_rate_hz, channels ``` Only what the package holds is written. Sources the capture lacks are listed in the job's notes as skipped. A file the exporter has no typed home for is embedded verbatim under `/holos/files/` with its MIME type and listed in the notes as an extra, so a new track is never dropped silently. ### Vision Pro and iPhone packages A Vision Pro capture carries ARKit's 26-joint hand skeleton (`hand_*_joints_*`) and stereo video. An iPhone capture carries Vision's 21 hand landmarks instead (`hand_*_landmarks_3d` in metres and `_2d` in image space, each landmark with a confidence, 0 where Vision did not see it), a Core Motion IMU track, one back-camera video, and a `tracking_state` column on the device pose. Both export into the same tree; the demo group's `master_clock` says which hand track set the clock, and `source` on the root names the device. ### One clock for the demo, every clock under raw The device records streams at different rates: the head pose at about 60 Hz, each hand at about 90 Hz on Vision Pro and about 15 Hz on iPhone, the IMU at 100 Hz, object anchors when they move. robomimic assumes one `T`, so `/data/demo_0` samples everything on the dominant hand's clock (the right hand when it was tracked, else the left, ARKit's skeleton before Vision's landmarks, else the device) by zero-order hold: each sample carries the latest value of every other stream, and NaN where a stream has nothing yet or its newest value is older than 0.25 s. The demo group's `master_clock` and `resampling` attrs say exactly this. Nothing is thrown away: `/holos/raw` keeps every stream at its own rate with its own `time` scale. ### What `actions` means for a human demonstration robomimic loaders require the key, and a person has no commanded action. `actions[t]` is the dominant hand's wrist pose delta: the position difference to the next sample and the relative rotation `q[t+1] * conj(q[t])` (xyzw), with an identity delta on the last row. On an iPhone capture, where Vision's wrist landmark carries no rotation, it is the `[T, 3]` position delta alone. The dataset's `derivation` and `source_hand` attrs state it, so nobody has to guess. ### Procedure steps as sample ranges When the capture was recorded against a procedure, `/holos/events/procedure_steps` carries each step visit with `t_start` and `t_end` in capture seconds and `start_sample` and `end_sample` as indices into `/data/demo_0` (end exclusive), so a step is a slice: `obs[start:end]`. ## Reading it ```python h5py theme={null} import h5py, json with h5py.File("capture.h5") as f: print(f.attrs["Conventions"], f.attrs["title"]) demo = f["data/demo_0"] t = demo["time"][...] # seconds since capture start right = demo["obs/hand_right_joints_world"][...] # [T, 26, 7] joints = [j.decode() for j in f["joint"][...]] wrist = right[:, joints.index("forearmWrist"), :3] # [T, 3] metres, ARKit world env = json.loads(f["data"].attrs["env_args"]) steps = f["holos/events/procedure_steps"][...] for s in steps: print(s["title"].decode(), t[s["start_sample"]], "to", t[s["end_sample"] - 1]) ``` ```python xarray theme={null} import xarray as xr obs = xr.open_dataset("capture.h5", group="data/demo_0/obs", engine="h5netcdf") obs["hand_right_joints_world"].sel(joint="forearmWrist", pose_component=["x", "y", "z"]).plot.line(x="time") raw = xr.open_dataset("capture.h5", group="holos/raw/device_pose", engine="h5netcdf") print(raw["pose"].sizes) # {'time': 5486, 'pose_component': 7} ``` ```python robomimic theme={null} from robomimic.utils.dataset import SequenceDataset import robomimic.utils.file_utils as FileUtils meta = FileUtils.get_env_metadata_from_dataset("capture.h5") # env_name "holos_capture" ds = SequenceDataset( hdf5_path="capture.h5", obs_keys=["device_pose", "hand_right_joints_world"], dataset_keys=["actions"], seq_length=10, filter_by_attribute="all", ) sample = ds[0] ``` ```python media theme={null} import h5py, numpy as np with h5py.File("capture.h5") as f: ds = f["media/camera_left"] print(ds.id.get_type().get_tag()) # b'video/quicktime' raw = np.empty(ds.shape, dtype="V1") ds.id.read(h5py.h5s.ALL, h5py.h5s.ALL, raw, mtype=ds.id.get_type()) open("camera_left.mov", "wb").write(raw.tobytes()) ``` ## Decoded frames The export embeds the camera files as bytes rather than decoding them: a ten-minute stereo capture decoded to uint8 arrays is tens of gigabytes. Asking the API with `decodeFrames: true` adds `/holos/raw/video//frames` (`uint8 [F, H, W, 3]`, Image Spec 1.2, one frame per chunk, with its own `time` scale) and `/data/demo_0/obs/_frame_index` (`int64 [T]`, the frame nearest each demo sample), so a loader can gather frames per sample without the file repeating them. ## The convention version The root attrs carry `holos_convention_version="HolosCapture-1.0"`. On top of robomimic's layout, 1.0 adds: * the shared coordinate scales at the root (`pose_component`, `joint`, `tracked_object`, `axis_xyz`, `landmark`, `image_uv`) and the per-demo `state_component` and `action_component` scales * `timestamp_ns`, `time` as a CF dimension scale, and the frame attrs on the demo group * `/holos` (metadata, raw streams, transcript, events, VLM) and `/media` * `actions` as the wrist delta, declared in `derivation` A change to any of this is a new version here, not a surprise in a file. ## API | Endpoint | What it does | | - | - | | `POST /captures/{captureId}/export-hdf5` | Queue an export (202), return the open one (200), or return a current finished export with a presigned download (200 `ready`). Body: `force` to re-export regardless, `decodeFrames` to add frame arrays. | | `GET /captures/{captureId}/export-hdf5` | `status` (`none`, `queued`, `in_progress`, `ready`, `failed`), `progress`, `artifact` (`filename`, `size`, `downloadUrl`, `resolvedAt`, `current`) and `error`. | The file lives under the capture's `exports/` prefix and is removed with the capture. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.