# Holos API Source: https://docs.holos.io/api/index Programmatic access to procedures, captures, objects, and organizations. The Holos API is a REST API that gives you programmatic access to all platform resources. It's the same API used by the Holos web dashboard and mobile apps. ## Base URL ``` https://api.holos.io ``` ## Authentication All requests require a bearer token obtained through the magic link flow. Pass the token in the `Authorization` header: ```http theme={null} Authorization: Bearer ``` See [Authentication](/concepts/authentication) for the full token flow. ## Resources | Resource | Description | | - | - | | [Procedures](/domains/procedures) | Canonical step sequences, corroborated by captures | | [Guidebooks](/domains/guidebooks) | Everything known about one physical thing, in one place | | [Captures](/domains/captures) | Recorded spatial sessions | | [Objects](/domains/objects) | 3D models used in captures and courses | | [Courses](/domains/courses) | Immersive training delivered in Holos Learn | | [Lessons](/domains/lessons) | The ordered units a course is taught in | | [Packs](/domains/packs) | The platform's open-license object catalogue, grouped by subject | | [Organizations](/domains/organizations) | Tenant accounts and member management | | [Users](/domains/users) | User identity and member profiles | ## Endpoint reference The full endpoint-by-endpoint reference and interactive playground are currently **internal-only** — they aren't published here yet. Holos engineers can preview them locally; see [the holos-docs README](https://github.com/holosinc/holos-docs#internal-api-reference). The OpenAPI 3.1 spec is generated from and maintained in the `holos-api` repository. The pages below cover the concepts you need to integrate: authentication, webhooks, error shapes, and the per-resource guides. ## Concepts > **[Authentication](/concepts/authentication)** > > Session tokens and the bearer flow > **[Webhooks](/concepts/webhooks)** > > Event notifications via Stytch and Svix > **[Errors](/concepts/errors)** > > Error format and status codes # Holos Capture Source: https://docs.holos.io/capture/index Record multi-modal spatial captures from real-world environments. ![Holos Capture](https://mintcdn.com/holosinc/3qjjKOLHhziCZQk_/images/capture/capture-hero.png?fit=max\&auto=format\&n=3qjjKOLHhziCZQk_\&q=85\&s=5537609602936ca0156551405bc3c1e3) Holos Capture is a spatial recording app for capturing rich multi-modal data from real-world environments and procedures. A single capture session records synchronized streams across multiple modalities (stereo video, hand skeleton tracking, device pose, named object tracking, on-device transcription, and GPS location), all packaged together for upload to the Holos platform. Currently available on Apple Vision Pro. Captures are the raw material for Holos procedures and training. Once uploaded, they can be reviewed in the web dashboard and linked to procedures as evidence for their steps. ## What gets recorded | Modality | Description | | - | - | | Stereo video | 1080p left/right camera feed | | Device pose | 6-DOF headset position and orientation | | Hand tracking | Full skeleton for both hands, world and local frame | | Object tracking | Named reference objects tracked in real time | | Transcription | On-device speech-to-text with timestamps | | Location | GPS coordinates when authorized | ## Next steps > **[Set up your device](/capture/setup)** > > Install Holos Capture via TestFlight and sign in. > **[Record a session](/capture/recording)** > > Start your first capture and understand the modality controls. > **[Tracked objects](/capture/tracked-objects)** > > Set up reference objects so the app knows what to track. > **[Upload a session](/capture/uploading)** > > Review, annotate, and upload completed sessions. # Recording a session Source: https://docs.holos.io/capture/recording Start a capture session and understand the modality controls. ## Starting a session From the Holos Capture home screen: 1. Tap **New Capture** to open the session setup sheet 2. Enter a **Name** for the session (it defaults to today's date) 3. Under **Objects**, select any trackable objects you want followed during the session 4. Tap **Start** The app enters an immersive space and begins recording all active modalities simultaneously. ## Modalities Each modality captures a distinct data stream. All streams are timestamped and synchronized to the same clock. | Modality | Notes | | - | - | | Stereo video | Always active. Records left and right camera at 1080p. | | Device pose | Always active. Records headset position and orientation at full frame rate. | | Hand tracking | Active when hands are in frame. Records full joint skeleton for both hands. | | Object tracking | Active only for objects selected when the session starts. See [Tracked objects](/capture/tracked-objects). | | Transcription | On-device speech recognition. Produces timestamped transcript segments. | | Location | Recorded once at session start. | ## During a session * A recording indicator is visible in the corner of your view while capturing * Tracked objects appear highlighted when the system has a lock on them * Transcription happens in real time; speak clearly if you want good captions ## Stopping a session Tap the **Stop** button to end the recording. The app will finalize all streams and package them into a session file. This may take a few seconds for longer recordings. Once finalized, you'll be taken to the session review screen. See [Uploading](/capture/uploading). ## Tips for good captures See the full [Tips for good captures](/capture/tips) guide for detailed guidance on environment setup, movement, narration, object tracking, and common mistakes to avoid. # Camera settings Source: https://docs.holos.io/capture/settings Configure recording quality and file size for capture sessions. Camera settings are accessible from the main screen via the gear icon, under the **Camera** tab. ## Recording quality Controls the resolution of the stereo video streams. | Option | Resolution | Notes | | - | - | - | | Full HD | 1920 × 1080 | Default. Recommended for most captures. | | Standard | 864 × 704 | Smaller file sizes. | Lower resolution only affects the video streams. Pose, hand tracking, object tracking, transcription, and location are all recorded at the same fidelity regardless of this setting. ## Bitrate Controls the video encoding bitrate. Adjustable from **4 to 20 Mbps** in steps of 4, defaulting to **12 Mbps**. Higher bitrate produces better image quality at the cost of larger file sizes. 12 Mbps is a good balance for most field conditions. Drop to 8 Mbps if storage space or upload bandwidth is a concern. ## Record SBS separately When enabled (default), the app encodes an additional side-by-side stereo-rectified video file alongside the separate left and right camera files. This SBS file is useful for viewing the capture in stereo but adds approximately 50% to the total session size. Disable this if you only need the individual left and right streams and want to reduce upload size. # Device setup Source: https://docs.holos.io/capture/setup Install and set up Holos Capture on Apple Vision Pro. > [!NOTE] > > These instructions are for Holos Capture on Apple Vision Pro. ## Requirements * Apple Vision Pro running visionOS 2.0 or later * A Holos account ## Distribution Holos Capture is distributed through the **Apple Developer Enterprise Program**, not the App Store. This is required because the app uses special enterprise-only entitlements to access the Apple Vision Pro's stereo camera, a capability Apple restricts to enterprise-signed builds. Depending on how your organization has deployment configured, you'll install the app either directly via an enterprise distribution link or through TestFlight. ### Via enterprise distribution 1. Open the installation link provided by your Holos admin in Safari on your Vision Pro 2. Follow the prompt to install the app 3. If you see an "Untrusted Enterprise Developer" warning, go to **Settings → General → VPN & Device Management**, find the Holos certificate, and tap **Trust** ### Via TestFlight 1. Open TestFlight on your Apple Vision Pro 2. Accept the invitation from your Holos admin 3. Tap **Install** next to Holos Capture 4. Once installed, open the app from your home screen ## Sign in Holos uses passwordless authentication. 1. Open Holos Capture 2. Enter your work email address 3. Check your email for a magic link and tap it to complete sign-in 4. Select your organization if you belong to more than one ## Permissions On first launch, Holos Capture will request access to several system capabilities. All of the following are required; the app will not allow you to start a recording session until each has been granted. | Permission | What it's for | | - | - | | **Camera** | Stereo video recording from the Vision Pro cameras | | **World Sensing** | Spatial tracking and device pose | | **Hand Tracking** | Full hand skeleton tracking during sessions | | **Accessory Tracking** | Reference object recognition and tracking | | **Microphone** | Audio recording and on-device speech transcription | | **Location** | GPS coordinates attached to the session at start time | Grant all six when prompted. If you deny any of them, go to **Settings → Privacy & Security** on your Vision Pro to enable them before trying again. # Tips for good captures Source: https://docs.holos.io/capture/tips How to record spatial captures that produce accurate, useful training data. A good capture is deliberate, well-narrated, and unhurried. The data it produces (pose, hands, object positions, transcript) is only as useful as the care taken to record it. This guide covers what to do before, during, and after a session to maximize capture quality. ## Before you start ### Set up your environment * **Light the subject well.** Natural or overhead lighting works best. Avoid recording with a bright window directly behind the subject; this creates silhouetting that degrades video quality and hurts object recognition. * **Clear the workspace.** Remove objects and clutter that aren't part of the procedure. A clean field of view helps the tracker focus on the right objects and makes the footage easier to review. * **Know the procedure in advance.** Walk through what you're going to do before starting the recording. Captures of someone working from memory mid-session produce hesitant, disjointed footage that's hard to learn from. Think of it like a rehearsed demonstration, not a first attempt. ### Check your tracked objects Confirm every object you want tracked is selected in the New Capture sheet before tapping Start. Any object not selected there won't be tracked during the session, and the selection can't be changed once recording begins. Position the objects where they'll be used before starting. Getting them into frame early gives the tracker more time to acquire a lock. *** ## During the session ### Move slowly and deliberately This is the single most important thing you can do for a good capture. Pose tracking works best when head movements are smooth and intentional. Fast pans, sudden turns, and shaky movements introduce noise into the pose data and can cause brief tracking losses. The same applies to hand tracking; quick, jerky motions are harder to reconstruct than smooth, deliberate ones. **A good rule of thumb:** move at roughly half the speed you would in a normal work context. It will feel slow. That's correct. When transitioning between steps (turning to pick something up, moving to a different part of the workspace), turn your whole body rather than just your head, and do it smoothly. ### Narrate everything out loud On-device transcription produces a timestamped record of everything you say during the session. This transcript becomes part of the capture data and is used to generate captions and training outputs. A well-narrated capture is dramatically more useful than a silent one. **Narrate as you go, not after.** Describe what you're about to do just before doing it, then describe what you did. For example: > "I'm picking up the torque wrench; this one is set to 25 foot-pounds. I'm positioning it on the left-side bolt here, and applying steady pressure clockwise until I feel the click." Speak clearly and at a normal pace. Avoid filler words. If you make a mistake, narrate that too ("I grabbed the wrong tool there, let me swap to the correct one") so reviewers understand what they're seeing. Don't worry about sounding scripted. A clear, informative narration is far more valuable than a natural-sounding but vague one. ### Pause between steps After completing each discrete step, pause for 1–2 seconds before moving to the next. This gives the tracker time to settle, produces clean step boundaries in the data, and makes the footage easier to review and annotate later. Think in terms of chapters: complete a step, hold briefly, move on. ### Keep the subject in frame If you're demonstrating a procedure on an object or surface, keep it in your field of view as much as possible. The stereo cameras capture what you're looking at; if the subject drifts to the edge of your view, the useful data goes with it. For bench work, position yourself so the work surface is roughly centered in your view at a comfortable working distance. Avoid leaning in very close (below \~30cm) or stepping back far; both extremes degrade tracking quality. ### Let object tracking acquire a lock When you first encounter a tracked object, look directly at it for 1–2 seconds before touching or moving it. This gives the tracker time to acquire a lock. You'll see the highlight appear when it does. If a tracked object loses its lock mid-session (the highlight disappears), look directly at it again before continuing with that part of the procedure. *** ## Common mistakes to avoid | Mistake | Why it matters | Fix | | - | - | - | | Moving too fast | Introduces noise into pose and hand data | Slow down intentionally; half speed is a good target | | Working in silence | Transcript is empty or sparse, outputs are lower quality | Narrate continuously, including transitions and decisions | | Poor lighting | Degrades video and object recognition | Ensure good overhead or front lighting before starting | | Rushing through steps | No natural pause points in the data | Pause 1–2 seconds between each discrete step | | Not checking tracked objects first | Objects not tracked even though they're present | Confirm objects loaded in the pre-session sheet | | Looking away from the subject | Subject drops out of frame at key moments | Keep work surface centered in your view throughout | | Recording a first attempt | Hesitation and mistakes produce noisy data | Rehearse the procedure before recording | *** ## Multiple takes Don't rely on a single capture. Recording 2–3 takes of the same procedure gives reviewers options and makes it easier to catch steps that were unclear or poorly framed in one recording. Takes don't need to be perfect. A take that clearly shows a specific sub-step well is valuable even if the rest isn't usable. *** ## Session length There's no hard limit on session length, but shorter, focused captures are generally more useful than long continuous ones. If a procedure has natural break points (phases, sub-tasks, or tool changes), consider recording each phase as a separate session rather than one long take. Shorter sessions are also faster to review and upload. # Inspector panel Source: https://docs.holos.io/capture/tour-inspector Session metadata and visibility controls during replay. The inspector panel slides in from the side during replay and shows metadata and display controls for the current session. ## Information Shows session details: date, duration, device, and location. If GPS was recorded, a **Show map** button displays a map pinned to the capture location. Tap **Hide map** to collapse it. ## Visibility Controls what's shown in the 3D replay scene: * **Skeleton**: toggles the body skeleton overlay. On by default. * **Tracked objects**: each object that was selected for the session has its own toggle. Objects that were detected during the session can be shown or hidden individually. Objects that were selected but not detected appear dimmed and can't be toggled. ## Video Expandable inline players for each camera stream recorded in the session: * **Capture Left**: left camera stream * **Capture Right**: right camera stream * **Capture Spatial**: side-by-side stereo-rectified video, if [Record SBS separately](/capture/settings#record-sbs-separately) was enabled Tap a stream name to expand its player. All streams stay in sync with the main playback timeline. # Library Source: https://docs.holos.io/capture/tour-library Browse and manage your recorded sessions from the Holos Capture home screen. The **Library** is the home screen of Holos Capture. It shows all sessions recorded on your device, both uploaded and pending. ## Session list Each session appears as a tile showing: * A thumbnail from the recording * The session title (or a timestamp if untitled) * Upload status (pending or uploaded) * Duration Tap any session to open its review screen, where you can play it back, review the transcript, and upload it if you haven't already. ## Starting a new capture Tap the **New Capture** button to open the pre-recording sheet. See [Recording a session](/capture/recording). ## Settings and account Tap the gear icon in the top corner to access [camera settings](/capture/settings) and your account. Organization admins can also manage org details from the account tab. # Replay Source: https://docs.holos.io/capture/tour-replay Play back a recorded session and review its data. Tapping a session in the Library opens the replay screen. This is a full spatial playback of the recorded session: video, skeleton, and tracked object positions are replayed in sync. ## Playback controls The control bar at the bottom of the view contains: * **Play / Pause**: start and stop playback * **Scrubber**: drag to seek to any point in the session * **Captions**: toggle the transcription overlay on and off. Disabled if the session has no transcript. * **Mute**: silence the audio track * **Repeat**: loop the session back to the start when it ends ## Inspector panel Tap the inspector icon to open the side panel. It shows session metadata and visibility controls for the 3D elements in the scene. See [Inspector panel](/capture/tour-inspector). ## Uploading If the session hasn't been uploaded yet, an **Upload** button is available from the replay screen. See [Uploading](/capture/uploading). # Tracked objects Source: https://docs.holos.io/capture/tracked-objects Set up 3D reference objects so the app knows what to track during a session. Object tracking lets Holos Capture recognize and follow specific real-world objects during a session, recording their position over time alongside the other modalities. ## How it works When you start a session, the **New Capture** sheet lists every trackable object in your organization's library. Select the ones that will be in the scene, and the app prepares each one and tracks its position in real time throughout the recording. ## Setting up tracked objects Tracked objects are managed in the Holos web dashboard, not in the Capture app itself. 1. Go to [Objects](https://app.holos.io/objects) in the Holos dashboard 2. Open the object, or upload a new one 3. Click the **scan** icon and choose **Enable object tracking**. See [Object tracking](/dashboard/objects/tracking) for the settings 4. The Holos team processes the request; the object's **Trackable** field flips to **Yes** when it's ready Once an object is trackable, it appears in the **Objects** list the next time you open the New Capture sheet. ## Object library The Holos platform includes a library of CC0 objects and supports org-private uploads. Admins and managers can upload GLB, OBJ, FBX, or USDZ models via [Objects](https://app.holos.io/objects) in the dashboard. \[!NOTE] Uploading objects requires admin or manager access. Uploaded models are automatically converted to the formats needed for web preview and AR tracking. ## During a session When a tracked object is recognized: * A highlight appears around it in your view * Its position is recorded each frame for the duration of the session If an object goes out of frame or is briefly occluded, the app will re-acquire it when it reappears. Tracking gaps are recorded as-is. # Uploading a session Source: https://docs.holos.io/capture/uploading Review and upload completed capture sessions to the Holos platform. After stopping a recording, Holos Capture packages all streams into a session file and opens the review screen. ## Session review The review screen shows a summary of what was recorded: * Duration and timestamp * Which modalities were active * A preview of the transcript (if transcription was enabled) * Thumbnail from the recording Review the session before uploading. If the recording was unusable (wrong take, equipment issue), you can discard it from this screen. ## Uploading Tap **Upload** to send the session to the Holos platform. Upload progress is shown on screen. Once the upload completes, the session appears in [Captures](https://app.holos.io/captures) in the Holos dashboard, where it can be: * Reviewed in the 3D capture viewer * Linked to a [procedure](/dashboard/procedures) as evidence for its steps * [Exported as HDF5](/dashboard/captures/hdf5-export) for downstream tools ## Upload requirements * An active internet connection (Wi-Fi recommended for large sessions) * You must be signed in to your Holos account Sessions are stored locally until uploaded. If you leave the review screen without uploading, the session remains in the **Library** tab and can be uploaded later. ## Library The Library tab shows all recorded sessions on your device, both uploaded and pending. Tap any session to open its review screen and upload if you haven't already. # Authentication Source: https://docs.holos.io/concepts/authentication How to obtain and use session tokens with the Holos API. The Holos API uses token-based authentication. Every request must include a session token in the `Authorization` header. Tokens are obtained through a passwordless magic link flow. ## Obtaining a token Authentication is a three-step flow: **1. Request a magic link** ```http theme={null} POST https://api.holos.io/users/send-magic-link Content-Type: application/json { "email_address": "you@example.com" } ``` **2. Authenticate the token from the link** The user receives an email containing a magic link. Extract the token from the link and exchange it: ```http theme={null} POST https://api.holos.io/users/authenticate-magic-link Content-Type: application/json { "token": "" } ``` This returns an `intermediate_session_token` along with a list of organizations the user belongs to. **3. Exchange for an org-scoped session** ```http theme={null} POST https://api.holos.io/users/exchange-session Content-Type: application/json { "intermediate_session_token": "", "organization_id": "" } ``` The response contains a `session.jwt`. This is the token you'll use for all subsequent requests. ## Making authenticated requests Pass the session JWT as a bearer token: ```http theme={null} GET https://api.holos.io/procedures Authorization: Bearer ``` Requests without a valid token receive a `401 Unauthorized` response. ## Token expiry Session tokens expire after a configurable period. When a token expires, re-authenticate using the magic link flow above. The Holos dashboard and apps handle refresh automatically. ## Organization context Most endpoints are org-scoped, but the organization is not named in the URL. It comes from the session token, so `GET /procedures` returns the procedures of whichever organization your session is currently bound to. ``` /procedures the current org's procedures /objects the current org's objects /captures the current org's captures ``` A user may belong to several organizations, and a session is bound to one of them at a time. To act on a different one, exchange the session for a session in that org: ```http theme={null} POST https://api.holos.io/users/exchange-session Authorization: Bearer ``` The endpoints that do name an organization in the path are the ones that administer it rather than work inside it, such as `GET /organizations/{id}/members` and `POST /organizations/{id}/objects/upload`. Authorization is enforced per-org from the member's role in the bound organization. ## Roles | Role | Permissions | | - | - | | `owner` | Full access including org settings | | `admin` | Full access excluding ownership transfer | | `manager` | Can create and edit procedures, captures, and objects | | `member` | Can view and record captures | # Errors Source: https://docs.holos.io/concepts/errors Error format and HTTP status codes returned by the Holos API. The Holos API returns standard HTTP status codes and a consistent JSON error body. ## Error format ```json theme={null} { "error": "Not found", "message": "Procedure not found", "statusCode": 404 } ``` | Field | Description | | - | - | | `error` | Short error label | | `message` | Human-readable description | | `statusCode` | HTTP status code (mirrors the response status) | ## Status codes | Code | Meaning | | - | - | | `200` | Success | | `201` | Resource created | | `400` | Bad request: invalid or missing parameters | | `401` | Unauthorized: missing or invalid session token | | `403` | Forbidden: authenticated but insufficient role | | `404` | Not found: resource doesn't exist or is outside your org | | `409` | Conflict: duplicate resource (e.g. a duplicate identifier within an org) | | `422` | Unprocessable: validation failed (Joi schema rejection) | | `500` | Internal server error | ## Authorization errors `401` and `403` are distinct: * `401` means no valid session token was provided * `403` means the token is valid but the user's role doesn't permit the action Both indicate an auth failure but require different handling: `401` should trigger a re-auth flow, while `403` is a permission issue that re-auth won't fix. # MCP (Model Context Protocol) Source: https://docs.holos.io/concepts/mcp Query and manage Holos procedures directly from Claude, ChatGPT, Gemini, and other AI assistants. > [!NOTE] > > Holos MCP support is in development. This page describes what we're building. Reach out if you'd like to be an early tester. The [Model Context Protocol](https://modelcontextprotocol.io) (MCP) is an open standard that lets AI assistants connect directly to external tools and data. A Holos MCP server means your organization's procedure library becomes queryable and writable from Claude, ChatGPT, Gemini, or any other MCP-compatible assistant, without leaving the tools your team already uses. ## What you'll be able to do > **Query procedures** > > Ask your AI assistant which procedures exist, what they cover, and which are waiting on review, all in plain language. > **Draft procedures** > > Draft a new procedure with a title and steps directly from a chat prompt. > **Trigger captures** > > Kick off a capture workflow without switching to the Holos dashboard. > **Retrieve outputs** > > Pull generated SOPs, course outlines, and training content into your AI session for review or further editing. ## Authentication The Holos MCP server uses an OAuth 2.1-compliant authorization flow built on the same identity infrastructure that powers your existing Holos sessions. When you connect an AI assistant for the first time: 1. **Authorization request.** Your AI assistant initiates an OAuth 2.1 authorization request. 2. **One-time approval.** You complete a single browser prompt using the same passwordless flow you use to log in to Holos. Permissions are presented in plain language so you know exactly what access you're granting. 3. **Scoped access token.** The assistant receives a short-lived token with only the permissions you approved. An assistant can never be granted more access than you yourself have. ## Security Access is controlled at both the user and organization level: * **RBAC scopes.** Permissions are structured in logical sets so agents get only the access they need, nothing more. * **Org-level controls.** Admins get a centralized view of all connected assistants, authorized scopes, and active sessions across the organization. * **Allowlists.** Organizations can restrict which AI assistants members are permitted to connect. * **One-click revocation.** Any connected assistant can be disconnected instantly from your account settings or by an org admin. * **Audit trail.** A real-time log of all connected apps and the scopes they've been granted. ## Interested in early access? We're evaluating MCP support now. If you have a specific integration in mind or want to be among the first to try it, [let us know](mailto:hello@holos.io). # Webhooks Source: https://docs.holos.io/concepts/webhooks Event notifications from Stytch and Svix. Holos uses webhooks to notify your infrastructure of platform events. Two webhook sources are active: * **Stytch**: identity and organization lifecycle events (member created, org updated, SSO configured) * **Svix**: platform-level events dispatched by the Holos API ## Stytch webhooks Stytch sends webhook payloads to the Holos API when identity events occur. These are used internally to keep the Holos member and organization records in sync with Stytch's B2B data. If you need to react to member or org changes in your own systems, Stytch also supports outbound webhooks directly; configure them from the Stytch dashboard. ## Svix webhooks Holos uses Svix to dispatch platform events to external subscribers. Events are available for: * Capture lifecycle (created, uploaded) * Object processing status changes Webhook endpoints are configured per-organization. Contact your Holos admin to set up a webhook endpoint for your org. ## Payload format All webhook payloads are JSON. Each event has a `type` field and a `data` object: ```json theme={null} { "type": "capture.uploaded", "data": { "captureId": "cap_abc123", "organizationId": "org_xyz", "uploadedAt": "2026-05-12T20:00:00Z" } } ``` ## Verification Svix signs all webhook payloads with a secret specific to your endpoint. Verify the signature before processing the event. See the [Svix verification docs](https://docs.svix.com/receiving/verifying-payloads/how) for language-specific examples. # Captures Source: https://docs.holos.io/dashboard/captures Review and manage captures in the Holos dashboard. The **Captures** section gives managers visibility into everything recorded by the team. Captures can be browsed, filtered, and reviewed in a full 3D spatial viewer. ## Browsing captures Captures can be viewed in three modes, toggled in the top right: * **Table**: sortable columns for title, status, modality, captured by, location, and date * **Tiles**: a thumbnail grid showing spatial footage with status and member avatar * **Map**: a geographic view of all capture locations. Clusters group nearby sessions; click a cluster to see the list of captures at that location ## Filtering Use the **All statuses** dropdown to filter by processing state: | Status | Meaning | | - | - | | Pending | Recorded on device, not yet uploaded | | Uploaded | Transferred to the platform, awaiting processing | | Processing | Modality data being extracted | | Completed | All data available and ready for review | | Failed | Processing error; [contact support](mailto:hello@holos.io) | ## Capture detail Click any capture to open the detail view. The left side is a full 3D spatial viewer; the right side is a properties panel. ### Properties panel | Field | Description | | - | - | | Title | Editable display name | | ID | Unique capture identifier | | Performer | The member who recorded the session | | Created | Date and time of recording | | Device | Device used (e.g. Apple Vision Pro) | | OS | OS version at time of recording | | Duration | Session length | | Location | GPS address with **Show map** link | ### 3D viewer The viewer plays back the full spatial session. Use the timeline scrubber and playback controls at the bottom to seek through the session. **View toggles** control what's visible in the scene: * **Skeleton**: the body skeleton overlay * **Gaze**: gaze direction indicator * Individual [tracked objects](/dashboard/objects) (listed by name), with toggles for each **Camera** switches the viewer perspective: * **First Person**: from the performer's point of view * **Third Person**: a third-party observer view of the scene ### Video The **Video** section in the properties panel shows the raw camera streams recorded during the session: * **Capture Left**: left stereo camera * **Capture Right**: right stereo camera These stay in sync with the 3D viewer timeline. ### Transcript The **Transcript** section shows the timestamped speech-to-text generated during the session. Individual segments can be edited directly; click a segment to correct the text. Edits are stored as overrides and applied on read; the original transcript is always preserved underneath. ## Downloading and exporting Right-click a capture in the table or tile grid for **Download** and **Export as HDF5**. Download hands back the raw `.capture` package as a zip. Export as HDF5 writes the same data as one `.h5` file with typed datasets on a shared time axis, laid out to published conventions (robomimic's dataset layout, CF attributes, HDF5 dimension scales) so it opens in h5py, xarray, MATLAB and robomimic without custom code. The export runs as a job; the download starts when the file is ready. See [Export as HDF5](/dashboard/captures/hdf5-export) for the layout. # Export as HDF5 Source: https://docs.holos.io/dashboard/captures/hdf5-export One HDF5 file per capture, laid out to published conventions so it opens in h5py, xarray, MATLAB and robomimic without custom code. **Download** hands back the raw `.capture` package: a folder of CSVs, JSON and video. **Export as HDF5** writes the same data as one `.h5` file with typed datasets, a shared time axis and the media embedded, for notebooks, robotics pipelines and anyone who would rather not parse CSVs. ## Exporting Right-click a completed capture in the Captures table or tile grid and choose **Export as HDF5**. A worker writes the file, so the toast follows the job (waiting for a worker, writing, uploading) and the download starts when it is done. A capture whose export is still current hands back the same file without a new job; editing the transcript makes the last export stale, and the next ask writes a fresh one. Picture captures have no package and do not offer the export. ## The conventions The group tree is built from three published conventions, each one checkable with a tool you already have, rather than a layout of our own. > **HDF Group specifications (file level)** > > * **Dimension Scales.** Every axis of every gridded dataset carries a scale: `time` for the sample axis, `pose_component` (`x y z qx qy qz qw`), `joint`, `tracked_object`, `state_component`. This is the mechanism netCDF-4 is built on, so xarray, netCDF4 and MATLAB read the axes without custom code. > * **Image and Palette Specification 1.2** on image datasets (`CLASS="IMAGE"`, `IMAGE_VERSION="1.2"`, `IMAGE_SUBCLASS="IMAGE_TRUECOLOR"`, `INTERLACE_MODE="INTERLACE_PIXEL"`), so HDFView renders them. > * **Opaque datatypes with a MIME tag** for embedded container bytes: the camera `.mov` files are `H5T_OPAQUE` tagged `video/quicktime`, the microphone `.wav` is tagged `audio/wav`. Readers know what they are holding without a sidecar. > * Chunked storage with gzip, shuffle and Fletcher-32 checksums on anything that is not tiny; one self-contained file with no external links; strings as variable-length UTF-8. > **CF-1.11 attributes (metadata level)** > > * Root: `Conventions="CF-1.11, HolosCapture-1.0"`, `title`, `institution`, `source` (device and app), `history` (exporter version and time), `references` (this page), `capture_id`, `organization_id`, `modality`, `capture_start`. > * Every numeric dataset: `units` (`m`, `s`, `ns`, `1`), `long_name`, and `_FillValue` (NaN) where a stream can be absent. > * `time`: float64 with `units="seconds since "` and `calendar="proleptic_gregorian"`, so xarray decodes it to datetimes. `timestamp_ns` beside it is int64 nanoseconds since the Unix epoch, the ROS 2 convention. > * The coordinate frame is stated once on the demo group and never left implicit: `frame="arkit_world"`, `handedness="right"`, `up_axis="Y"`, `quaternion_order="xyzw"`, `length_unit="m"`. > **robomimic dataset layout (domain level)** > > robomimic's HDF5 format is the de facto shape for learning-from-demonstration data (robosuite, MimicGen, LIBERO and ManiSkill exporters all read or write it). One capture is one demo: > > * `/data` with attrs `total` (sample count) and `env_args` (JSON: `env_name="holos_capture"`, `type=99` meaning no simulator behind this, and the capture's metadata under `env_kwargs`). > * `/data/demo_0` with attrs `num_samples` and the frame conventions above, holding `obs/*`, `actions`, `states`, `time` and `timestamp_ns`. > * `/mask/all` listing `demo_0`, so `filter_by_attribute="all"` works. > > Everything Holos-specific lives outside `/data`, under `/holos` and `/media`, so a stock robomimic loader never sees it. ## What is inside ```text theme={null} /pose_component, /joint, /tracked_object shared coordinate scales (string arrays) /data attrs: total, env_args /data/demo_0 attrs: num_samples, frame, handedness, up_axis, quaternion_order, length_unit, master_clock, resampling /data/demo_0/time float64 [T] seconds since capture start (dimension scale) /data/demo_0/timestamp_ns int64 [T] epoch nanoseconds /data/demo_0/obs/timestamp float64 [T] same values as time, as an obs key /data/demo_0/obs/device_pose float64 [T, 7] headset pose, ARKit world frame /data/demo_0/obs/hand_left_joints_world float64 [T, J, 7] J = 26 ARKit HandSkeleton joints (names on the joint scale) /data/demo_0/obs/hand_right_joints_world float64 [T, J, 7] /data/demo_0/obs/hand_left_joints_local float64 [T, J, 7] relative to the wrist anchor /data/demo_0/obs/hand_right_joints_local float64 [T, J, 7] /data/demo_0/obs/object_pose float64 [T, K, 7] tracked object anchors (ids and names on the tracked_object scale) /data/demo_0/obs/hand_{left,right}_landmarks_3d float64 [T, 21, 3] iPhone: Vision landmarks in metres (names on the landmark scale) /data/demo_0/obs/hand_{left,right}_landmarks_confidence float64 [T, 21] iPhone: per-landmark confidence, 0 = not seen /data/demo_0/obs/imu_accel, imu_gyro float64 [T, 3] iPhone or derived: user acceleration (m s-2), rotation rate (rad s-1), device frame /data/demo_0/actions float64 [T, 7] dominant-hand wrist delta (see below) /data/demo_0/states float64 [T, S] device pose + wrist poses, layout on the state_component scale /holos/raw/device_pose/{time, timestamp_ns, pose} every sample, on its own clock /holos/raw/hand_pose_world/{left,right}/{time, timestamp_ns, joints} /holos/raw/hand_pose_local/{left,right}/{time, timestamp_ns, joints} /holos/raw/object_pose//{time, timestamp_ns, pose} /holos/raw/imu/{time, timestamp_ns, accel, gyro} iPhone / derived: m s-2 and rad s-1, device frame; attr source /holos/raw/hand_landmarks_3d/{left,right}/{time, timestamp_ns, points, confidence, hand_confidence} iPhone: Vision's 21 landmarks in metres /holos/raw/hand_landmarks_2d/{left,right}/{time, timestamp_ns, points, confidence, hand_confidence} iPhone: normalized image u, v /holos/files/ opaque bytes, MIME-tagged any package file nothing above types /holos/metadata/session JSON string metadata.json verbatim /holos/metadata/stereo_calibration JSON string /holos/metadata/thumbnail uint8 [H, W, 3] Image Spec 1.2 /holos/transcript/segments compound (start_s, end_s, text, is_final) /holos/transcript/tokens compound (segment, start_s, end_s, text) /holos/transcript/full_text string /holos/events/procedure_steps compound (index, step_id, title, t_start, t_end, start_sample, end_sample) /holos/events/procedure_steps_json JSON string events/procedure_steps.json verbatim /holos/vlm/output compound (t_mono, t_wall, prompt, output) /media/ opaque bytes, video/quicktime; attrs codec, width, height, fps, duration_s. camera_left, camera_right and camera_sbs on Vision Pro; camera_back on iPhone /media/audio opaque bytes, audio/wav; attrs sample_rate_hz, channels ``` Only what the package holds is written. Sources the capture lacks are listed in the job's notes as skipped. A file the exporter has no typed home for is embedded verbatim under `/holos/files/` with its MIME type and listed in the notes as an extra, so a new track is never dropped silently. ### Vision Pro and iPhone packages A Vision Pro capture carries ARKit's 26-joint hand skeleton (`hand_*_joints_*`) and stereo video. An iPhone capture carries Vision's 21 hand landmarks instead (`hand_*_landmarks_3d` in metres and `_2d` in image space, each landmark with a confidence, 0 where Vision did not see it), a Core Motion IMU track, one back-camera video, and a `tracking_state` column on the device pose. Both export into the same tree; the demo group's `master_clock` says which hand track set the clock, and `source` on the root names the device. ### One clock for the demo, every clock under raw The device records streams at different rates: the head pose at about 60 Hz, each hand at about 90 Hz on Vision Pro and about 15 Hz on iPhone, the IMU at 100 Hz, object anchors when they move. robomimic assumes one `T`, so `/data/demo_0` samples everything on the dominant hand's clock (the right hand when it was tracked, else the left, ARKit's skeleton before Vision's landmarks, else the device) by zero-order hold: each sample carries the latest value of every other stream, and NaN where a stream has nothing yet or its newest value is older than 0.25 s. The demo group's `master_clock` and `resampling` attrs say exactly this. Nothing is thrown away: `/holos/raw` keeps every stream at its own rate with its own `time` scale. ### What `actions` means for a human demonstration robomimic loaders require the key, and a person has no commanded action. `actions[t]` is the dominant hand's wrist pose delta: the position difference to the next sample and the relative rotation `q[t+1] * conj(q[t])` (xyzw), with an identity delta on the last row. On an iPhone capture, where Vision's wrist landmark carries no rotation, it is the `[T, 3]` position delta alone. The dataset's `derivation` and `source_hand` attrs state it, so nobody has to guess. ### Procedure steps as sample ranges When the capture was recorded against a procedure, `/holos/events/procedure_steps` carries each step visit with `t_start` and `t_end` in capture seconds and `start_sample` and `end_sample` as indices into `/data/demo_0` (end exclusive), so a step is a slice: `obs[start:end]`. ## Reading it ```python h5py theme={null} import h5py, json with h5py.File("capture.h5") as f: print(f.attrs["Conventions"], f.attrs["title"]) demo = f["data/demo_0"] t = demo["time"][...] # seconds since capture start right = demo["obs/hand_right_joints_world"][...] # [T, 26, 7] joints = [j.decode() for j in f["joint"][...]] wrist = right[:, joints.index("forearmWrist"), :3] # [T, 3] metres, ARKit world env = json.loads(f["data"].attrs["env_args"]) steps = f["holos/events/procedure_steps"][...] for s in steps: print(s["title"].decode(), t[s["start_sample"]], "to", t[s["end_sample"] - 1]) ``` ```python xarray theme={null} import xarray as xr obs = xr.open_dataset("capture.h5", group="data/demo_0/obs", engine="h5netcdf") obs["hand_right_joints_world"].sel(joint="forearmWrist", pose_component=["x", "y", "z"]).plot.line(x="time") raw = xr.open_dataset("capture.h5", group="holos/raw/device_pose", engine="h5netcdf") print(raw["pose"].sizes) # {'time': 5486, 'pose_component': 7} ``` ```python robomimic theme={null} from robomimic.utils.dataset import SequenceDataset import robomimic.utils.file_utils as FileUtils meta = FileUtils.get_env_metadata_from_dataset("capture.h5") # env_name "holos_capture" ds = SequenceDataset( hdf5_path="capture.h5", obs_keys=["device_pose", "hand_right_joints_world"], dataset_keys=["actions"], seq_length=10, filter_by_attribute="all", ) sample = ds[0] ``` ```python media theme={null} import h5py, numpy as np with h5py.File("capture.h5") as f: ds = f["media/camera_left"] print(ds.id.get_type().get_tag()) # b'video/quicktime' raw = np.empty(ds.shape, dtype="V1") ds.id.read(h5py.h5s.ALL, h5py.h5s.ALL, raw, mtype=ds.id.get_type()) open("camera_left.mov", "wb").write(raw.tobytes()) ``` ## Decoded frames The export embeds the camera files as bytes rather than decoding them: a ten-minute stereo capture decoded to uint8 arrays is tens of gigabytes. Asking the API with `decodeFrames: true` adds `/holos/raw/video//frames` (`uint8 [F, H, W, 3]`, Image Spec 1.2, one frame per chunk, with its own `time` scale) and `/data/demo_0/obs/_frame_index` (`int64 [T]`, the frame nearest each demo sample), so a loader can gather frames per sample without the file repeating them. ## The convention version The root attrs carry `holos_convention_version="HolosCapture-1.0"`. On top of robomimic's layout, 1.0 adds: * the shared coordinate scales at the root (`pose_component`, `joint`, `tracked_object`, `axis_xyz`, `landmark`, `image_uv`) and the per-demo `state_component` and `action_component` scales * `timestamp_ns`, `time` as a CF dimension scale, and the frame attrs on the demo group * `/holos` (metadata, raw streams, transcript, events, VLM) and `/media` * `actions` as the wrist delta, declared in `derivation` A change to any of this is a new version here, not a surprise in a file. ## API | Endpoint | What it does | | - | - | | `POST /captures/{captureId}/export-hdf5` | Queue an export (202), return the open one (200), or return a current finished export with a presigned download (200 `ready`). Body: `force` to re-export regardless, `decodeFrames` to add frame arrays. | | `GET /captures/{captureId}/export-hdf5` | `status` (`none`, `queued`, `in_progress`, `ready`, `failed`), `progress`, `artifact` (`filename`, `size`, `downloadUrl`, `resolvedAt`, `current`) and `error`. | The file lives under the capture's `exports/` prefix and is removed with the capture. # Guidebooks Source: https://docs.holos.io/dashboard/guidebooks One guidebook per physical object, linking the procedures, documents, and 3D model that apply to it, with an assistant you can ask by text or voice. A **Guidebook** collects everything your organization knows about one physical thing: a drawing, an assembly, a piece of equipment. Create one guidebook per object, then attach the [procedures](/dashboard/procedures) that apply to it, the technical documentation for it, and its CAD or digital-twin [model](/dashboard/objects). Guidebooks are deliberately thin: identity plus pointers, never content. A guidebook has no instructions of its own beyond a plain-text description; instructions live on procedures, and a shared procedure (lockout/tagout, torque verification, ESD handling) can appear in many guidebooks without being copied. Every guidebook gets a short ID (`GBK-3`) used in lists, breadcrumbs, and URLs. ## Creating a guidebook Go to [**Guidebooks**](https://app.holos.io/guidebooks) and click **New guidebook**. Name it for the thing it answers for. The dialog also asks for: * **Linked object** (optional): the 3D model of the thing, from your [objects library](/dashboard/objects). Picking one auto-fills the title from the object's name. Each object can belong to only one guidebook, and a guidebook holds one object at a time. * **Drawing number**, **Revision**, **Customer item ID**: your organization's internal identifiers from the drawing title block. Scans and QR codes resolve against these later, and they keep regenerated 3D objects matched to the right drawing revision. All the identity fields can be edited afterwards on the detail page, in the **External identity** card. ## The guidebooks list ![The guidebooks list in the Holos dashboard, showing guidebook cards with object thumbnails, short IDs, and procedure counts](/images/dashboard/guidebooks-list-light.png) The list shows each guidebook with its linked object's thumbnail, short ID, title, and procedure count, as **Cards** or a **Table** (the table adds drawing identifiers, reference counts, and creation dates). Search matches titles, short IDs, and all three drawing identifiers, so a pasted drawing number or item ID lands on the right guidebook. ## The guidebook page ### QR code ![A guidebook's QR code card, with the Holos mark in the center and a caption reading Right-click to copy or print](/images/dashboard/guidebook-qr-light.png) Every guidebook page opens with its QR code. A guidebook stands for a real piece of equipment, so the code is meant to end up on that equipment: print it, stick it on the enclosure, and a scan opens the guidebook. Right-click the code for **Copy link**, **Copy image**, **Download PNG**, and **Download SVG**. The code encodes the guidebook's URL, not a bare ID, so renaming the guidebook does not strand a label already in the field. It stays dark-on-white in both themes because a printed label and the on-screen code have to agree. ### Linked object The object chip under the title shows the model's thumbnail, its trackable status, and its size. Right-click it to **Open object**, **Link a different object** (a swap), or **Unlink from guidebook**. If the object hasn't been processed for tracking yet, the chip reads **Awaiting processing**. ### Procedures The **Procedures** card lists the procedures linked to this guidebook. Each row shows the procedure's short ID, title, and its published state: `v2 · 14 steps` for a published procedure, or **Not published** with a **Draft only** badge for one that has never been published. Use the **+** to link an existing procedure. The procedure stays where it is; a guidebook only points at it. Each row's menu offers: * **Pin** / **Unpin**: keep it at the top * **Hide** / **Show**: suppress it from the guidebook without removing the link * **Remove from guidebook**: drop the link (the procedure itself is untouched) ### References The **References** card holds the documents for the asset: the drawing, BOM, manual, or anything else (PDF, DOCX, XLSX, and other common formats). Drag files in or browse. Rows download on click; right-click for **Open document**, **Extract procedure** (PDFs), or **Remove**. 3D models don't go here; they live in the linked-object chip. ## Extracting a procedure from a document Right-click a PDF in the References card and choose **Extract procedure**. The platform reads the document and drafts a procedure from it: its steps, safety callouts, outline nesting, and even the document's own revision table (which appears on the procedure as read-only **External history**). Extraction takes a couple of minutes and survives a page refresh; when it finishes, a toast offers **Open draft**. The draft is automatically linked into the guidebook and nothing is published; it goes through the normal [review and publish path](/dashboard/procedures#publishing-and-versions). ## Asking the guidebook Organizations in the assistant's private preview see an **Ask** button in the guidebook page's header. It opens a conversation grounded in that guidebook's procedures and documents: which procedures exist, what a step says, where a wire goes, who published a revision. Questions can be typed or spoken, and every answer cites what it read. See [Assistant](/dashboard/guidebooks/assistant) and [Voice mode](/dashboard/guidebooks/voice). [Contact us](https://holos.io/contact) to get it turned on for your organization. ## Archive and delete The **⋯** menu on the guidebook page offers: * **Archive**: the guidebook disappears from the list but keeps its links, documents, and history. Its linked object is released, so a successor guidebook can claim it. There is no unarchive yet. * **Delete permanently**: removes the guidebook and its uploaded documents. Procedures and 3D objects it points at are not deleted. # Assistant Source: https://docs.holos.io/dashboard/guidebooks/assistant A conversation grounded in one guidebook's procedures and documents, with cited answers. The guidebook assistant is in private preview. Organizations with it turned on see an **Ask** button in the header of every guidebook page. It opens the **Ask this guidebook** sheet: a conversation grounded in that guidebook's procedures and documents, and nothing else. Type a question in the composer, or pick one of the starter questions. The starters are written for each guidebook from its published procedures and converted documents ("Where does wire 14 go and how long is it?") and are rewritten whenever a procedure is published or a document is converted. [Contact us](https://holos.io/contact) to get it turned on for your organization. ## What it can answer The assistant reads before it answers, and it answers only from what the guidebook holds: * **Procedures**: which ones exist, what their steps, callouts, and verifications say, and which steps carry a caution or warning. It checks procedures first, because a procedure is the reviewed answer for the equipment, and goes to a document only when no procedure covers the question. Pinned procedures are preferred when a question could fit several. * **Reference documents**: what a drawing, BOM, or manual says, once the PDF has been converted. Wiring and parts questions (where a wire goes, what a terminal connects to, a part number) are answered from the document's own tables, row by row. * **History**: who created, edited, or published a procedure and when, from the procedure's activity log, with people shown by name and photo. If the guidebook's content does not answer the question, the assistant says so rather than guessing. A document whose text is not available yet (still converting, or the conversion failed) is reported as such; right-click it in the References card to convert it. ## Reading an answer * Steps quoted from a procedure render as the procedure page's own step rows, with the same numbering style, callout icons, and colors. A step's part chip opens the [3D guide](/dashboard/procedures#3d-guide) on that part. * Procedures are linked by short ID and title. A link to a particular step opens the procedure page on that step, in a new tab, so the conversation stays put. * Documents appear as chips that open the source viewer. Every fact taken from a document carries a citation mark; click it to see the page with the passage outlined. * A list of values flattened into a step's description (a torque list, settings, parts with quantities) is shown as a table after the step card. * The reads behind an answer fold under one line ("Read 1 procedure, ran a search and opened 1 document"). Expand it to see each read and what it found. The conversation survives closing and reopening the sheet. A page refresh starts over. To ask by voice instead of typing, see [Voice mode](/dashboard/guidebooks/voice). # Voice mode Source: https://docs.holos.io/dashboard/guidebooks/voice Ask the guidebook assistant aloud and hear the answer, with the full written answer still in the sheet. Voice is part of the [guidebook assistant](/dashboard/guidebooks/assistant), which is in private preview. Click **Talk** in the sheet's composer to start a voice conversation; the browser asks for microphone access the first time. Speak your question and the assistant answers aloud. The full written answer, with its step rows, links, and citations, appears in the sheet exactly as it would for a typed question, so you can read the detail after hearing the summary. You can interrupt the assistant mid-sentence, and it will not go quiet while it looks something up: a lookup that runs longer than a few seconds is narrated ("Still checking the SD-1080 r9 document for wire 14"). While a voice conversation is live: * Captions under the conversation show what you and the assistant last said, with a status line that reads **Listening**, **Hearing you**, **Looking it up** (with the read it is on), or **Speaking**. * A microphone meter beside **End** shows the input level, so it is plain the mic is on. * In the transcript, spoken questions carry a microphone mark, and the short summary the assistant said aloud sits above each written answer with a speaker mark. Click **End** to stop talking. Closing the sheet also ends the voice conversation. The **Talk** button appears only where voice is available for your organization; typed questions work everywhere the assistant does. # Holos Dashboard Source: https://docs.holos.io/dashboard/index Manage procedures, guidebooks, captures, and objects at app.holos.io. The [Holos dashboard](https://app.holos.io) is the management layer of the platform. It sits between the two apps: after experts record with [Holos Capture](/capture/index) and before learners train with [Holos Learn](/learn/index). Procedures are authored here, captures are reviewed, and 3D objects are curated. > **[Procedures](/dashboard/procedures)** > > Author the canonical step sequences your team performs by, corroborated by captures. > **[Guidebooks](/dashboard/guidebooks)** > > One per physical object, linking its procedures, documents, and 3D model. > **[Captures](/dashboard/captures)** > > Review captures, explore them on a map, and export them for downstream tools. > **[Objects](/dashboard/objects)** > > Browse and upload 3D models for AR tracking and course content. > **[Organization](/dashboard/organization)** > > Manage members, roles, and org-wide settings. # Objects Source: https://docs.holos.io/dashboard/objects Browse, upload, and manage 3D models for tracking and course content. The **Objects** library holds the 3D models your organization has uploaded: CAD assemblies, scans, and models of the equipment you train on. Objects marked as trackable can be used as reference targets during [Holos Capture](/capture/index) sessions. The platform's own catalogue of open-license models is not listed here. It is grouped into [object packs](/dashboard/objects/packs) that an admin enables for the organization, and those objects appear when you build a stage rather than in this library. ## Browsing and searching Objects can be viewed as tiles or a table. Search by name, provider, or tag. Paginated at 56 per page; use **Flagged** and **Archived** links in the bottom bar to switch views. Each tile shows the object name and provider. ## Object detail Click any object to open its detail page. The top of the page shows: * **Name** and **created by** * **Description**: editable free-text field * **License**: dropdown (e.g. None / All Rights Reserved, CC0) * **Tags**: free-form labels for search and filtering The stats panel in the top right shows: | Field | Description | | - | - | | Triangles | Polygon count of the mesh | | Size | File size | | Trackable | Whether an AR reference descriptor exists for this object; see [Object tracking](/dashboard/objects/tracking) | The full 3D viewer below lets you inspect the object from any angle, browse its part hierarchy, and play its animations. See [Inspecting a model](/dashboard/objects/inspecting-a-model). ### Actions The icon row in the top right of the detail page provides: * **Info**: object metadata * **Flag**: report an issue with the object * **Status**: approval state * **Scan**: enable [object tracking](/dashboard/objects/tracking) * **Download**: download the object file; click to select format (GLB, USDZ, etc.) * **Delete**: remove the object from your org ## Reporting issues Click the flag icon to open the **Report issue** dialog. Select the issue category and click **Report**: | Issue | Description | | - | - | | Backfaced textures | Geometry normals are inverted | | Large textures | Texture resolution is excessive | | Off origin | Object pivot point is misplaced | | Bad thumbnail | Auto-generated thumbnail doesn't represent the object | | Bad orientation | Object is rotated incorrectly relative to its expected use | | Other | Any other issue | ## Uploading objects > [!NOTE] > > Uploading objects requires admin or manager access. Admins and managers can upload new org-private objects via the **Upload** button. | Format | Self-contained? | Notes | | - | - | - | | GLB | Yes | Primary format, no conversion needed | | USDZ | Yes | Retained for AR; GLB generated for web preview | | STEP / STP | Yes | CAD geometry, tessellated to a mesh on upload | | PLY | Yes | Scan output from photogrammetry or LiDAR. Color is stored on the vertices, so there are no texture files to bring | | OBJ | **No** | Always needs its `.mtl` and texture files | | FBX | Usually | Needs its texture folder unless textures were embedded at export | | ZIP | Yes | A model plus the files it references, zipped together. Subfolders are preserved | | .referenceobject | Yes | Produced by Holos Capture; the USDZ inside it is extracted automatically | See [Preparing a model](/dashboard/preparing-a-model) for what to include per format, what happens when files are missing, and what does not survive conversion. ## Approving uploads Object approvals are handled in the [Object Approvals](https://app.holos.io/objects/approvals) queue. See [Object Approvals](/dashboard/organization#object-approvals) for the full review workflow. # Inspecting a model Source: https://docs.holos.io/dashboard/objects/inspecting-a-model Inspect a model's part hierarchy, toggle CAD edges, and play its animations. Every [object detail page](/dashboard/objects) includes a full 3D viewer. It renders the object on the same theme-aware stage as the capture and course viewers, with studio lighting so metals and other PBR materials reflect properly, and a ground grid for scale. The same viewer, with the same panels, appears in the [approvals queue](/dashboard/organization#object-approvals). **Navigation**: left-drag to orbit, right-drag to pan, scroll or pinch to zoom. Click a part to select it; click empty space to deselect. ## Inspecting the part hierarchy When a model has real structure (more than one mesh), a **Parts** panel appears in the top-right corner showing the model's assembly tree: groups and parts, nested the way the CAD file was authored. Conversion noise is filtered out, and each part row shows its triangle count. Single-mesh models get no panel. This is the place to validate a STEP upload: a CAD assembly's sub-components should survive conversion as named parts rather than a single blob, and the tree is where you check that. * **Click a row** to select the part. It's outlined in the viewport, and a translucent ghost shows through occluding geometry, so a part behind a panel still reveals where it sits, the way CAD tools show selections through walls. Clicking a part in the viewport selects it in the tree too, expanding and scrolling to its row. * **Selection details**: the panel footer shows the selected part's name, triangle count, and bounding box, measured in the model's own units, before any display normalization. Useful for checking that a part is really the size the drawing says. * **Hover a row** for two actions: the **eye** hides or shows the part and everything under it, and the **isolate** icon shows only that part's subtree, hiding the rest of the model. **Show all** in the header restores everything. ## Edges The **Edges** toggle in the Parts panel header draws the sharp-angle outlines that make same-colored CAD faces readable. Edges default on for objects uploaded as STEP files and off otherwise; flipping the toggle is saved as the object's preference, so every visit, on the web and in the headset, opens the object the same way. Edges on parts that are tiny on screen are suppressed automatically and return as you zoom in, so a dense assembly doesn't turn into line mush at a distance. ## Viewing animations When a model carries animations, an **Animations** panel appears in the top-left corner listing each animation with its duration. The first one plays automatically on load, looping, so you see motion without hunting; click a row to switch animations, and use the header button to pause and resume in place. Exporters often split one authored animation into a clip per animated target (bones, shape keys, individual meshes). The panel groups those back into a single row and plays them together, so a character animates with its skeleton and its face at once. # Object packs Source: https://docs.holos.io/dashboard/objects/packs The platform's open-license model catalogue, grouped by subject and enabled per organization. The platform maintains a catalogue of roughly 2,750 open-license 3D models, largely from Poly by Google. As one flat list it is unusable for picking something to place in a stage, so it is grouped into 14 **object packs** by subject: Animals & wildlife, Food & drink, Architecture & buildings, Nature & plants, Vehicles & transport, Electronics & tech, Furniture & home, Sports, toys & games, Office & stationery, Fashion & accessories, Space & science, Music & audio, and two opt-in packs described below. Every pack is **off by default**. An organization enables the ones it wants, and only those objects reach the stage editor. This is deliberate: a whale sitting beside your industrial control panel is confusing rather than useful. ## Enabling packs > [!NOTE] > > Enabling packs requires admin or manager access. Go to **Organization → General → Object packs** and switch on the packs you want. **Enable all** turns on every general pack in one click. Changes apply to the whole organization. Enabling a pack does not add its objects to the [Objects library](/dashboard/objects), which lists only your organization's own uploads; they become available when building a stage. ## Opt-in packs Two packs are held back and marked **Opt-in only**: | Pack | Contents | | - | - | | Weapons & military | Firearms, bladed weapons, ordnance and military equipment | | Alcohol, tobacco & gambling | Alcoholic drinks, smoking, and gambling | They are excluded from **Enable all** and must be switched on individually. There is nothing sexual or graphic in either; they are separated because some platforms restrict this content, and because a defence training scenario that wants military props should not have to enable beer and poker chips to get them. # Object tracking Source: https://docs.holos.io/dashboard/objects/tracking Make a 3D model trackable so Holos Capture can recognize and follow the real thing. **Object tracking** lets Holos Capture recognize a specific real-world object and record its position throughout a session. Not all objects are trackable by default — trackable objects display a scan icon on their tile in the library, and the **Trackable** stat on the detail page shows **Yes** or **No**. ## Why it's necessary A 3D model on its own is just geometry — the app has no way to spot the matching physical object in the room. To bridge that gap, the platform builds a **reference object**: a compact descriptor trained from the model that the Capture app matches against the live camera feed to find and follow the real thing. Building a reference object is a compute-heavy job that runs on Holos infrastructure, so it isn't instant. You enable it once per object; afterwards the object is permanently trackable and ready to use in any capture. > [!NOTE] > > You only need to enable tracking for objects you want the app to follow during a session. Models used purely as course content or for reference don't need it. ## Enabling object tracking 1. Open the object from the library 2. Click the **scan** icon in the action row (top right of the detail page) 3. Choose your tracking settings (see below) 4. Click **Enable object tracking** The **Trackable** stat reads **No** with tracking *queued* or *in progress* until the job completes, and a small dot on the scan icon indicates an active request. You can reopen the dialog any time to update the settings while the job is still queued. > [!NOTE] > > Reference objects are built by the Holos team's processing pipeline. Once the job finishes, the **Trackable** field flips to **Yes** and the object becomes available in Holos Capture. ## Tracking settings The settings you choose describe how the object is handled in the real world, which helps the pipeline build an accurate reference object. **View mode** — how the object is approached during a session: > ![](https://mintcdn.com/holosinc/M4HR1UgvFmLr_sUN/images/icons/view-front.svg?fit=max\&auto=format\&n=M4HR1UgvFmLr_sUN\&q=85\&s=ba93011e5e467e02ef7d889ee36b5998) > > **Front** > Viewed only from the front. Wall-mounted panels, controls, screens. > ![](https://mintcdn.com/holosinc/M4HR1UgvFmLr_sUN/images/icons/view-upright.svg?fit=max\&auto=format\&n=M4HR1UgvFmLr_sUN\&q=85\&s=ed5ebc8ee8c800e1eb5bbe2f0c18783b) > > **Upright** > Stays upright, approached from any side. Benchtop equipment, freestanding machines. > ![](https://mintcdn.com/holosinc/M4HR1UgvFmLr_sUN/images/icons/view-all-angles.svg?fit=max\&auto=format\&n=M4HR1UgvFmLr_sUN\&q=85\&s=65066591d1ab24ed539b8f15487b7a1b) > > **All angles** > Picked up and rotated freely. Handheld parts and tools. > [!NOTE] > > **All angles** takes longer to train and carries a higher false-positive risk. Only choose it for objects that are genuinely handled from every direction. **Training mode** — how much compute the pipeline spends building the reference object: | Mode | Typical runtime | Best for | | - | - | - | | **Standard** | \~8 hours | Most objects | | **Extended** | \~18 hours | Small, handheld objects that need extra detail | **Objects to avoid** — optionally pick look-alike objects from your organization's library so the tracker doesn't confuse them with this one. Most useful when you have several visually similar models. Only objects that have a USDZ file can be selected, and only your own objects are listed: an object to avoid is something physically present in the room, which a catalogue model never is. Once an object is trackable, it can be selected in Holos Capture's New Capture sheet. See [Tracked objects](/capture/tracked-objects) for how in-session tracking works. # Organization Source: https://docs.holos.io/dashboard/organization Manage members, roles, authentication, and object approvals. Organization settings are managed under **Settings** in the dashboard sidebar. Owners and admins have full access; managers have read-only access to most settings. > [!NOTE] > > All pages in this section require admin or owner access, except where noted. ## Members The [**Member Management**](https://app.holos.io/admin/members) page shows all members in your org with their email, role, and status. To invite someone: 1. Click **+ Invite** in the top right 2. Enter their email address and select a role 3. They'll receive a magic link to join your org ### Roles | Role | Permissions | | - | - | | `owner` | Full access including org settings and ownership transfer | | `admin` | Full access excluding ownership transfer | | `manager` | Can create and edit [procedures](/dashboard/procedures), [captures](/dashboard/captures), and [objects](/dashboard/objects) | | `member` | Can view and record captures | ## Authentication & Access The [**Authentication & Access**](https://app.holos.io/admin/access) page controls how members sign in and how new members are provisioned. **Authentication settings** * Toggle which primary and secondary auth methods are allowed * Require MFA for all users in your org **User onboarding** * **Invites**: allow any email or restrict to specific domains * **JIT Provisioning**: automatically add members who sign in via allowed email domains or SSO connections * **Access control list**: define the email domains and SSO connections that are permitted **Automatic role assignments** * Automatically grant roles based on email domain, SCIM group membership, or SSO connection ## SSO > [!NOTE] > > SSO is available on enterprise plans. The [**SSO**](https://app.holos.io/admin/sso) page manages SAML-based single sign-on connections. Click **+ New connection** to configure your identity provider. Once active, members can sign in through your IdP and are provisioned automatically based on your JIT and role assignment settings. ## SCIM > [!NOTE] > > SCIM is available on enterprise plans. The [**SCIM**](https://app.holos.io/admin/scim) page lets you connect a directory sync provider to automate member provisioning and deprovisioning. Use this to keep your Holos membership in sync with your IdP. ## Object packs The platform's catalogue of open-license 3D models is grouped into 14 **object packs** by subject. Every pack is off until someone turns it on, and only enabled packs reach the stage editor. > [!NOTE] > > Enabling packs requires admin or manager access. Under **General → Object packs**, switch on the packs your organization wants. **Enable all** turns on every general pack at once; the two opt-in packs, **Weapons & military** and **Alcohol, tobacco & gambling**, are excluded from it and must be enabled individually. Enabling a pack does not add its objects to the [Objects](/dashboard/objects) library, which lists only your organization's own uploads. Pack objects become available when building a stage. See [Object packs](/dashboard/objects/packs) for what each pack contains. ## Object Approvals The [**Object Approvals**](https://app.holos.io/objects/approvals) queue is where admins review 3D objects before they become available to the org. Each object is inspected in a full 3D viewer and can be approved or rejected. When flagging an issue, select from the standard categories: | Issue | Description | | - | - | | Backfaced textures | Geometry normals are inverted | | Large textures | Texture resolution is excessive for the use case | | Off origin | Object pivot point is misplaced | | Bad thumbnail | Auto-generated thumbnail doesn't represent the object well | | Bad orientation | Object is rotated incorrectly relative to its expected use | | Other | Any other issue; add a note when rejecting | Use **A** to approve, **R** to reject, and arrow keys to navigate between objects in the queue. ### Applying fixes Some of those issues can be corrected in place. Rotate the object to the orientation it should have, then use **Apply Fixes** to write that correction into the model itself, along with re-centering it and repairing inverted normals or oversized textures where those were flagged. The correction is permanent rather than a display setting, so everything derived from the model is rebuilt to match: its thumbnail, the USDZ headsets use, and the lighter versions large models carry. Fixing orientation once is enough, and nothing downstream is left showing the old one. # Preparing a model Source: https://docs.holos.io/dashboard/preparing-a-model How to upload a model so its materials, textures, and colors survive. Most models reach us as more than one file, and uploading only the mesh produces a grey object with nothing to say why. This page covers what to include, what happens when something is missing, how scans differ, and the few things that do not survive conversion. For the format table and the upload button itself, see [Objects](/dashboard/objects#uploading-objects). ## Models that come as more than one file An OBJ never stores its own materials. The colors and textures live in a companion `.mtl` file, which in turn points at image files. An FBX can embed its textures, but only if whoever exported it chose to. Upload a model without those companions and it converts successfully into a grey, untextured object. The geometry is correct and nothing errors, so it is easy to miss. To avoid it, hand over the whole model rather than the one file. The upload dialog accepts any of these: * **Drag the model's folder in.** Usually the simplest, and it keeps the folder structure intact. * **Select several files at once** in the file picker. * **Pick a `.zip`** you have already made. Everything is packaged together and unpacked on our side. Subfolders are preserved, because a `.mtl` locates its textures relative to its own position, so a `textures/` folder has to stay where the model expects it. > [!NOTE] > > If your exporter offers **Embed Media** for FBX, using it avoids the problem entirely: the textures travel inside the `.fbx` file and there is nothing separate to remember. OBJ has no equivalent. ## Scans A scan from photogrammetry or LiDAR is the one case where the advice above does not apply. Upload the `.ply` on its own. Scanners store color on the vertices themselves rather than in a material, so there is no `.mtl`, no texture folder, and nothing that can go missing. A scan that looks right in your scanning software will look right here. The trade is that color detail and mesh detail become the same thing. A scan carries millions of triangles because that is what it takes to hold the color, not because the shape needs them. Upload the scan at full density and let the platform reduce it: we derive lighter versions for you, and for a dense scan we bake its color into a texture image so the detail survives the reduction. Markings, labels, and painted text stay readable on a mesh far smaller than the original. Reducing a scan yourself before uploading is usually the wrong move. Decimating a vertex-colored mesh throws away the paint along with the triangles, and there is no way to recover it afterwards. Scans also arrive in whatever orientation the scanner recorded, which is often on their side. That is corrected once during review, in [Object Approvals](/dashboard/organization#object-approvals). ## When files are missing If a model references files that did not arrive, the object says so rather than quietly appearing grey. The warning names the files, and appears in the objects list, on the object's detail page, and in the [approvals queue](/dashboard/objects#approving-uploads). You can fix it without re-uploading: open the object in the approvals queue and drop the missing files onto the warning. The model reconverts in place and keeps its name, tags, packs, and approval state. Drag in a whole folder if it is easier; anything not needed is left alone. A missing `.mtl` is usually reported in two stages. Until that file arrives, nothing can know which images it references, so supplying it often reveals a second, shorter list of missing textures. That is expected rather than a fault. ## What does not survive conversion Everything is converted to GLB for the web and to USDZ for headsets, and a few material features have no equivalent in those formats: * **Vertex color on a scan** becomes a texture image. This is a gain rather than a loss, and it is why a reduced scan keeps its markings, but it does mean the converted model carries a material where the original had none. * **Specular and Phong materials**, common in older FBX exports, arrive as base color plus whatever texture maps carried over. Models authored with PBR materials convert faithfully. * **Procedural or generated textures** are not evaluated. Bake them to image files before exporting. # Procedures Source: https://docs.holos.io/dashboard/procedures Author, corroborate, and publish the canonical step sequences your team performs by. A **Procedure** is a canonical step sequence: the how-to document your team performs by. Each new capture either corroborates its steps or raises a proposal for you to review, so the written procedure and the way the work is actually done converge over time. Every procedure gets a short ID (`PRC-107`) that appears in lists, breadcrumbs, and URLs. Renaming a procedure never breaks its links; the ID is what resolves. Procedures stand on their own. The same procedure can be linked into any number of [guidebooks](/dashboard/guidebooks). The guidebook only points at it; the procedure owns all of its content. ## Creating a procedure There are two ways to create one: 1. **From the [Procedures](https://app.holos.io/procedures) list**: click **New procedure**, name it, and it opens ready for authoring. You can link it to guidebooks and add steps after. 2. **Extracted from a document**: right-click a PDF in a guidebook's References card and choose **Extract procedure**. A draft is generated from the document, steps, callouts, and outline structure included. See [Guidebooks](/dashboard/guidebooks#extracting-a-procedure-from-a-document). A procedure can be written before anything is recorded; authoring doesn't wait on a capture to seed it. ## The procedures list The list at [**Procedures**](https://app.holos.io/procedures) shows every procedure in the organization: short ID, title, and a `{n} steps · {n} captures` summary. Search matches titles and short IDs, so a pasted `PRC-107` lands on the row. The **All** / **Needs review** toggle filters to procedures with open proposals; the Needs review badge counts pending proposals across the org. ## Description The **Description** is the how-to prose: context, safety framing, prerequisites, and any diagrams or media. It's a full rich-text editor, and images pasted or dropped in belong to the procedure itself. ## Steps Steps are the heart of a procedure. Each step has a **title**, an optional **description**, and a kind: | Kind | Renders as | | - | - | | **Step** | A numbered step | | **Info** | A blue callout banner | | **Caution** | A yellow callout banner | | **Warning** | An orange callout banner | | **Danger** | A red callout banner | Callouts sit in the sequence but don't consume a step number, so safety notices can be placed exactly where they matter without renumbering the work. Steps can be nested up to five levels deep with **Indent** / **Outdent**, reordered by dragging, and managed from a right-click context menu (**Add step above**, **Add step below**, **Edit step**, **Delete step**). Deleting a step offers an **Undo**. Step text supports limited inline markdown: `**bold**`, `*italic*`, `__underline__`, and links. How nested steps are numbered is an organization-wide setting (**Settings → Procedure numbering**): **Outline** (`1. → a) → i)`, the default), **Decimal** (`1 → 1.1 → 1.1.1`), or **Plain** (flat `1. 2. 3.`). ### Sources When a capture corroborates a step, the step gains a citation chip (`{capture} · 3:42`). Hovering previews the video frame at that timecode; clicking opens the capture viewer at that moment. Hand-written steps simply have no chip yet. Steps added or edited since the last publish carry a small `NEW` or `EDITED` marker so you can see what has drifted from the published standard. Publishing clears the markers. ## 3D guide Link a 3D object to a procedure and the page splits in two: the model on the left, the steps on the right. Selecting a step shows you where it happens on the model. ### Linking an object Use the object button in the procedure header to link an object. If the procedure's guidebook already tracks one, it is offered first. The same menu opens the object, links a different one in its place, or unlinks it. ### Component tags A step points at the model through its **component tags**. On a CAD model, a tag such as `CB1` or `TB1:7` matches the part with that name, and selecting the step outlines the part. Tags are edited in the step editor, with suggestions drawn from the model's part names and from the step's own text. A tag that matches nothing in the model is listed as not in this model. That is a normal state: the model may trail the document, or the object may have been swapped. ### Selecting a step Selecting a step highlights its parts and points, and the camera eases to the angle they are best seen from. For a point that is head-on to its surface; for parts it is the side nothing else on the model blocks. The camera never goes below the floor. * Click the step again to deselect it and return to the default view. * Click empty space around the model to deselect the step and leave the camera where it is. * The selected step is kept in the page's address, so a refresh or a shared link opens on that step. ### Points A 3D scan has no named parts to tag: the whole model is one mesh. **Points** fill that gap. A point is a named spot you place on the model's surface, and its name works exactly like a component tag: add it to a step and selecting the step shows the point. A step can mix part tags and points freely. To place a point: 1. Click **Add point** in the top-right corner of the 3D guide (or **+** beside **Points** once the object has some). You can also click **place a new point** in a step's tag field while editing the step. 2. Click the model where the point goes. 3. Name it and click **Save**. If a step was selected, or you started from a step's editor, the point is linked to that step and added to its tags. Otherwise it is placed unlinked, and you can link it to a step later. A point cannot share a name with a part in the model. ### The Points list **Points** in the top-right corner lists every point on the object, alphabetically, with how many steps use each one (or **unused**). Each row can: * **Link** the point to the selected step, or unlink it * **Rename** it: the tag is renamed on every step that uses it * **Delete** it: if steps use the point, you are asked first, and the point is removed from those steps too Clicking a pin on the model selects it and opens the list at its row. With no step selected, selecting a point moves the camera to it. Drag a pin along the surface to move it. ### Reading the model Pins keep the same size on screen however far you zoom. * A pin on the selected step is orange and shows its name. When a step uses several points, they are numbered in the order the step lists them. * Other pins are small white dots; hover one to see its name. * A pin on a surface facing away from you is dimmed. * A pin hidden behind the model is not shown, unless it is on the selected step, in which case it stays faintly visible so you know it is round the back. ### When the model changes Points stay on their feature when the object's model is later rotated, rescaled or re-centered: the dashboard records each of those changes and moves the points with the model, in the draft and in published versions alike. If the model is re-converted from its original upload, points placed before that may no longer sit on their feature, and the guide says so. Check them and drag any that moved. ### Versions Publishing freezes the points along with the steps, so a published version always shows its points where they were when it was published. Moving or renaming a point marks the draft as having unpublished changes. ## Captures and proposals Use **Link capture** to attach a recorded session. Its extracted steps are compared against the sequence: where they agree, the capture is added as a source on those steps; where they differ, each difference comes back as a **proposal** (add a step, reword a step, or remove a step) with the capture's rationale and a citation. You **Accept** or **Dismiss** each proposal individually; a dismissed proposal won't be raised by that capture again. **Check for changes** re-runs this alignment against every linked capture at any time. ## Publishing and versions The page you edit is always the **draft**. Clicking **Publish** snapshots the current sequence as an immutable revision (`v1`, `v2`, …), optionally with a version note, and further edits open on a new draft. The version chip in the header shows the latest published revision, or **Draft** if nothing has been published yet; a dot beside it means the draft has unpublished changes. Clicking the chip opens **Revision history**, where each revision lists who published it, when, its note, and its step and change counts. Opening a revision shows the sequence exactly as published, along with **Changes in this version**, the edit history that led to that publish. Snapshots are immutable; there is no unpublish, only publishing the next version. Procedures extracted from a document also show **External history**: the source document's own revision table, read-only, as written. ## Activity The **Activity** feed records everything as readable sentences: steps moved or reworded, captures linked, checks run, and where an extracted draft came from. Publishes appear as dividers (`{Name} published v3`), splitting the feed into per-version chapters. ## Deleting **Delete** on the procedure page permanently deletes the procedure and removes it from any guidebooks. Linked captures and their extracted steps are not affected, and published versions stay available for audit. You must type the procedure's short ID to confirm. # Captures Source: https://docs.holos.io/domains/captures Recorded spatial sessions uploaded from Holos Capture. A **Capture** is a recorded session from the Holos Capture app. It contains synchronized streams of spatial data (video, pose, hand tracking, object tracking, and transcription), packaged and uploaded to the platform for review, approval, and output generation. ## Key fields | Field | Description | | - | - | | `captureId` | Unique within the org | | `title` | Display name (editable after upload) | | `member_id` | The member who recorded it | | `tracked_objects` | Objects observed during the session, with resolved `_id` references | | `transcript` | Timestamped speech-to-text segments | | `location` | GPS coordinates at time of recording | | `device` | Device model and OS version | | `start_wall` | Wall-clock timestamp of when recording started | | `tags` | Free-form labels | | `thumbnailUrl` | Presigned URL for the session thumbnail | ## Lifecycle Captures are uploaded from the Holos Capture app. Once processed, they appear in the dashboard with all modality data available. Captures can be linked to [procedures](/domains/procedures), where each one either corroborates the procedure's steps or raises a proposal for review. ## Transcript overrides Individual transcript segments can be edited after upload. Edits are stored as override records and applied on read; the original transcript is preserved. ## Endpoints * `GET /captures`: list captures * `GET /captures/{captureId}`: get a capture * `GET /captures/{captureId}/transcript`: its timecoded transcript * `GET /captures/{captureId}/steps`: its extracted procedure steps * `GET /captures/{captureId}/download-url`: a signed URL for the recording * `POST /captures/{captureId}/export-hdf5` and `GET /captures/{captureId}/export-hdf5`: the capture as one HDF5 file, written by a worker job; see [Export as HDF5](/dashboard/captures/hdf5-export) # Courses Source: https://docs.holos.io/domains/courses Immersive training delivered in Holos Learn, built from ordered lessons. A **Course** is a unit of training delivered in [Holos Learn](/learn/index) on Apple Vision Pro. It holds ordered [lessons](/domains/lessons), the media they present, and the list of members permitted to open it. Alongside an organization's own courses, the platform publishes a shared catalogue. `GET /courses/platform` returns those, and they appear to every organization without being copied into it. ## Key fields | Field | Description | | - | - | | `title` / `description` | The course itself | | `coverPhoto` | Tile image in the course library | | `organizationId` | Owning organization. Absent on a platform course | | `createdBy` | The member who created it | ## Access A course is not open to an organization by default. Access is granted per member: * `POST /courses/{courseId}/grants` grants a member access, and `DELETE /courses/{courseId}/grants/{memberId}` revokes it. * A member without access can ask for it with `POST /courses/{courseId}/access-requests`, which an admin then grants or dismisses. * `GET /courses/{courseId}/access` reports where a given member stands. ## Media Lessons present images, video, and 3D models, uploaded to the course rather than to a lesson so the same asset can be used more than once. Upload is a two-step exchange: request a URL with `POST /courses/{courseId}/media/upload-url`, then confirm with `POST /courses/{courseId}/media/{mediaId}/complete` once the bytes have landed. An existing 3D object can be used directly, without uploading anything, via `POST /courses/{courseId}/media/feature-from-object`. ## Endpoints Full endpoint reference is available in the OpenAPI spec. Key operations: * `GET /courses`: list the organization's courses * `GET /courses/platform`: the shared platform catalogue * `POST /courses`: create a course * `PATCH /courses/{courseId}`: update title, description, or cover * `DELETE /courses/{courseId}`: archive a course, recoverable with `POST /courses/{courseId}/restore` * `POST /courses/{courseId}/grants`: grant a member access * `POST /courses/{courseId}/media/upload-url`: begin a media upload # Guidebooks Source: https://docs.holos.io/domains/guidebooks Everything an organization knows about one physical thing, in one place. A **Guidebook** collects everything an organization knows about a single physical thing: a drawing, an assembly, a piece of equipment. It gathers the [procedures](/domains/procedures) that apply to it, the technical documentation for it, and its CAD or digital-twin [model](/domains/objects). A guidebook does not own procedures. It links them, so the same procedure can appear in any number of guidebooks and still have one place where its content lives. ## Key fields | Field | Description | | - | - | | `shortId` | Stable public identifier. Renaming never breaks a link | | `title` / `description` | The guidebook itself | | `drawing_number` | The customer's drawing number, where the subject is a drawing | | `drawing_revision` | Revision of that drawing | | `customer_item_id` | The customer's own identifier for the item | | `procedures` | Links to procedures, each with its own placement (see below) | | `trackedObjects` | Objects the Capture app should track for this guidebook | | `referenceMaterials` | Uploaded documentation such as PDFs and drawings | | `archivedAt` | Set when archived; archived guidebooks are hidden rather than deleted | ## Linked procedures Each link carries how it got there and how it is presented: | Field | Description | | - | - | | `provenance` | `manual` (someone added it), `derived` (it came from the subject), or `extracted` (it was generated from a reference document) | | `pinned` | Surfaced above the rest | | `hidden` | Kept on the guidebook but out of the default view | | `order` | Explicit position, when the default ordering is not wanted | ## Extracting a procedure from a document A reference document can become a draft procedure. `POST /guidebooks/{guidebookId}/reference-material/{fileId}/extract-procedure` reads the document and produces a procedure with steps, callouts, and outline structure, linked back with `provenance: extracted`. Converting a document first, with `/convert`, gives the structured form the extraction reads. ## Endpoints Full endpoint reference is available in the OpenAPI spec. Key operations: * `GET /guidebooks`: list guidebooks * `POST /guidebooks`: create one * `GET /guidebooks/{guidebookId}`: get a guidebook with its links * `PATCH /guidebooks/{guidebookId}`: update its fields * `POST /guidebooks/{guidebookId}/procedures`: link a procedure * `PATCH /guidebooks/{guidebookId}/procedures/{procedureId}`: pin, hide, or reorder a link * `POST /guidebooks/{guidebookId}/tracked-objects`: link an object for tracking * `POST /guidebooks/{guidebookId}/reference-material/upload-url`: upload documentation # Lessons Source: https://docs.holos.io/domains/lessons The ordered units a course is taught in. A **Lesson** is one unit within a [course](/domains/courses). Lessons are ordered, and that order is what a learner moves through in Holos Learn. ## Key fields | Field | Description | | - | - | | `title` / `description` | The lesson itself | | `courseId` | The course it belongs to | | `lessonIndex` | Position within that course | | `createdBy` | The member who created it | ## Ordering `lessonIndex` is not edited directly. `PATCH /lessons/reorder` takes the sequence and rewrites the indices together, so the order stays contiguous no matter how a lesson is moved. ## Endpoints Full endpoint reference is available in the OpenAPI spec. Key operations: * `GET /lessons`: list lessons, scoped to a course * `POST /lessons`: create a lesson * `GET /lessons/{id}`: get a lesson * `PUT /lessons/{id}`: update it * `PATCH /lessons/reorder`: reorder a course's lessons * `DELETE /lessons/{id}`: delete a lesson # Objects Source: https://docs.holos.io/domains/objects 3D models used for AR tracking in captures and as content in courses. A **HolosObject** is a 3D model that can be tracked during a capture session, placed in a Holos Learn course, or both. The platform maintains a catalogue of CC0 (open-license) objects alongside org-private objects that admins upload themselves. ## Object types | Type | Description | | - | - | | Platform (CC0) | Open-license objects owned by the platform. Reached through the [object packs](/dashboard/objects/packs) an organization enables, not through its object library | | Org-private | Objects uploaded by a specific org, visible only to that org. These are what the object library lists | An organization's object library shows only its own uploads. Platform objects are grouped into packs that an admin switches on, and appear when building a stage. ## Supported formats Org admins can upload models in any of these formats: | Format | Notes | | - | - | | GLB | Primary format, no conversion needed | | USDZ | Converted to GLB post-upload; USDZ retained for AR | | OBJ | Converted to GLB on upload | | FBX | Converted to GLB on upload | | PLY | Scan output. Converted to GLB on upload, carrying vertex color across | | STEP / STP | CAD geometry, tessellated to a mesh on upload | | ZIP | A bundle: the root model plus the sidecar files it references | | .referenceobject | Produced by Holos Capture; the USDZ inside it is extracted automatically | All uploads automatically produce a GLB for web preview and a USDZ for AR tracking where not already provided. ## Making an object trackable For an object to be tracked during a capture session, it needs a `.referenceobject` file, an Apple ML-trained descriptor of its geometry generated from the USDZ mesh. The platform handles this as a processing request: an admin or manager enables tracking on the object, and the Holos team processes it and uploads the resulting `.referenceobject` file. See [Object tracking](/dashboard/objects/tracking). ## Key fields | Field | Description | | - | - | | `name` | Display name | | `tags` | Searchable labels | | `source.glb` | Signed download URL for the GLB mesh | | `source.usdz` | Signed download URL for the USDZ mesh | | `source.representations` | Lighter versions of the same model, each with its own `level`, triangle count, and size. See below | | `referenceObjectUrl` | Signed download URL for the `.referenceobject` AR descriptor | | `thumbnail` | Signed URL for the thumbnail image | | `triangleCount` | Triangles in the full mesh | | `approvalStatus` | `pending` · `approved` · `rejected` | | `conversionStatus` | `pending` · `processing` · `complete` · `failed` | ## Representations A large model carries lighter versions of itself alongside the full one, so a client can fetch what its own triangle budget allows rather than always taking the largest file. `source.glb` and `source.usdz` remain the full-fidelity version, and `source.representations` holds the rest. | Field | Description | | - | - | | `level` | Density. `0` is the full model; higher numbers are coarser | | `kind` | `mesh` today. Levels are the same kind of thing at different densities, not different kinds of model | | `glb` / `usdz` | Signed download URLs for that level. Either may be `null` when that format was not produced for it | | `triangleCount` | Triangles at that level | | `fileSizeBytes` / `usdzSizeBytes` | Size of each format at that level. The same triangle count is a very different byte count in each | A client picks the coarsest level under its own budget. Levels are produced only where they earn their place, so a model light enough already carries none. ## Endpoints * `GET /objects`: list objects (platform + org-private) * `GET /objects/{objectId}`: get an object * `PUT /objects/{objectId}`: update name, tags, and other editable fields * `PUT /objects/{objectId}/approval`: set approval status * `POST /organizations/{id}/objects/upload`: initiate an org object upload # Organizations Source: https://docs.holos.io/domains/organizations Tenant accounts and member management. An **Organization** is the top-level tenant in Holos. All resources (procedures, captures, objects, courses) are scoped to an org. Members belong to an org with a specific role that determines what they can do. ## Roles | Role | Capabilities | | - | - | | `owner` | Full access including billing and ownership transfer | | `admin` | Full access to all resources; can manage members | | `manager` | Can create and edit procedures, captures, and objects | | `member` | Can view resources and record captures | Role assignment is managed through the Holos dashboard or via Stytch's B2B member management. ## SSO and SCIM Organizations can configure SSO (SAML or OIDC) and SCIM provisioning via Stytch. When SCIM is active, member provisioning and deprovisioning is handled automatically by your identity provider. ## Key fields | Field | Description | | - | - | | `orgId` | Unique org identifier (slug) | | `name` | Display name | | `slug` | URL-safe identifier used in API paths | | `members` | List of member records with roles | ## Endpoints * `GET /organizations`: the organizations the caller belongs to * `GET /organizations/{id}`: get organization details * `PATCH /organizations/{id}`: update name, branding, and settings * `GET /organizations/{id}/members`: list members # Packs Source: https://docs.holos.io/domains/packs The platform's open-license object catalogue, grouped into subjects an organization can switch on. A **Pack** is a named group of platform [objects](/domains/objects). The platform's catalogue of open-license models is too large to browse as one list, so it is grouped into packs by subject, and an organization subscribes to the ones it wants. Packs are the only route to platform objects. An organization's object library lists its own uploads; platform objects appear when building a stage, and only from packs that organization has enabled. ## Key fields | Field | Description | | - | - | | `name` / `description` | The pack itself | | `visibility` | `public` (offered to every organization) or `private` | | `restricted` | Opt-in rather than offered by default, for subjects that need a deliberate choice | | `defaultEnabled` | Whether a new organization starts subscribed | | `managedBy` | `taxonomy` (membership follows the catalogue's own classification) or `curated` (membership is chosen by hand) | | `publishedBy` | The organization that published it, for packs that are not the platform's | | `icon` / `color` / `coverPhoto` | How the pack presents in the dashboard | | `tags` | Free-form labels | ## Subscription Subscription is per organization and is what makes a pack's objects available: * `POST /packs/{packId}/subscription` subscribes the caller's organization. * `DELETE /packs/{packId}/subscription` unsubscribes it, which hides the pack's objects without affecting anything already placed. ## Endpoints Full endpoint reference is available in the OpenAPI spec. Key operations: * `GET /packs`: list packs, with the caller's subscription state * `POST /packs`: create a pack * `PATCH /packs/{packId}`: update its fields * `GET /packs/{packId}/objects`: the objects in a pack * `POST /packs/{packId}/objects`: add objects to a curated pack * `POST /packs/{packId}/subscription`: subscribe the organization # Procedures Source: https://docs.holos.io/domains/procedures Canonical step sequences, corroborated by captures and published as revisions. A **Procedure** is a canonical step sequence: the how-to document a team performs by. It owns its steps, and each capture recorded against it either corroborates them or raises a proposal for review. A guidebook that links a procedure only points at it. ## Key fields | Field | Description | | - | - | | `shortId` | Stable public identifier such as `PRC-107`. Renaming never breaks a link, because the short ID is what resolves | | `title` / `description` | The procedure itself | | `capture_ids` | Captures recorded against it, the evidence its steps are corroborated by | | `objectBindings` | 3D objects bound to the procedure, so a step can point at real geometry. Each carries its placed `landmarks` | | `referenceMaterials` | Uploaded source files such as PDFs and drawings | | `sourceDocument` | Set when the procedure was extracted from a document rather than written, with its `kind`: `procedure`, `form`, `drawing`, or `reference` | ## Steps Steps are a sub-resource rather than an array on the procedure. Beyond a title and description, each carries a `depth` of `0` to `4` for sub-steps, an optional `callout` (`info` · `caution` · `warning` · `danger`), a `verification` describing how the person performing it confirms it is done, and `sources` and `citations` recording where it came from. Deleting a step is recoverable. ## Points on a bound object A step points at a bound object through its `componentTags`, which match the names of the model's parts. A 3D scan has no named parts, so a binding can also carry **landmarks**: named points placed on the model's surface. A landmark's `name` is a tag like any other, so a step references it by adding the name to its `componentTags`. | Field | Description | | - | - | | `name` | The tag steps reference. Unique per binding, ignoring case and separators | | `position` | `{ x, y, z }` in the model's own space and units, before any viewer centering, orientation or scaling | | `normal` | The surface direction at the point, or `null` | | `note` | Optional text shown with the point | Placing a landmark can link it to a step in the same request. Renaming one renames the tag on every step that uses it. Deleting one through the API leaves those steps' tags in place, where they no longer resolve; the dashboard's delete removes them. Each landmark records the object's `bakedTransform` when it was placed or moved, as `frame`. The detail and revision reads serve every landmark in the object's current frame, so a point placed before a later rotation, scale or recentre of the model still lands on its feature; write responses and list reads serve the stored position with its `frame`. The detail read also returns `objectGeometryChangedAt`, the last time the model was rebuilt in a way the recorded transform can't describe, so a client can flag landmarks placed before it. Publishing freezes the landmarks into the revision. ## Proposals A proposal is a change suggested by a capture rather than by a person, which is how a procedure stays honest about how the work is really done. Each is `pending`, `accepted`, or `rejected`, and one of three kinds: | Kind | Meaning | | - | - | | `insert` | Something happened that is not written down | | `reword` | The written step does not match what was done | | `unsupported` | A written step nothing corroborates | Accepting applies the change to the steps. Rejecting records the decision without changing them. ## Revisions Publishing snapshots the whole step sequence as a numbered, read-only revision, so what someone performed by on a given date stays recoverable after the procedure moves on. ## Endpoints Full endpoint reference is available in the OpenAPI spec. Key operations: * `GET /procedures`: list procedures, filterable by review state * `POST /procedures`: create a procedure * `GET /procedures/{procedureId}`: get a procedure with its steps * `PATCH /procedures/{procedureId}`: update title, description, or filing * `POST /procedures/{procedureId}/steps`: add a step * `POST /procedures/{procedureId}/steps/reorder`: reorder the sequence * `GET /procedures/{procedureId}/proposals`: open proposals awaiting review * `POST /procedures/{procedureId}/object-bindings`: bind a 3D object * `POST /procedures/{procedureId}/object-bindings/{objectId}/landmarks`: place a point, optionally linking it to a step * `PATCH /procedures/{procedureId}/object-bindings/{objectId}/landmarks/{landmarkId}`: rename or move a point * `DELETE /procedures/{procedureId}/object-bindings/{objectId}/landmarks/{landmarkId}`: remove a point * `POST /procedures/{procedureId}/publish`: snapshot the current steps as a revision # Users & Members Source: https://docs.holos.io/domains/users User identity and org membership in Holos. Holos distinguishes between **Users** (identity) and **Members** (org membership). ## Users A User represents an individual's identity: their email address and display name. A user can belong to multiple organizations. ## Members A Member record links a User to an Organization with a specific role. Most API endpoints work with Member IDs rather than User IDs, since the meaningful context is always within an org. ## Key fields ### User | Field | Description | | - | - | | `externalId` | External auth identifier, used as the primary identity reference across the platform | | `firstName` / `lastName` | Display name | | `email` | Primary email address | ### Member | Field | Description | | - | - | | `memberId` | External auth identifier for this org membership | | `authLevel` | Role: `owner` · `admin` · `manager` · `member` | | `organization_id` | The org this membership belongs to | ## Denormalized name fields Some resources (such as Captures) store `memberFirstName` and `memberLastName` directly on the record. This is intentional; it ensures display names remain stable even if a user updates their profile after the fact. ## Endpoints * `GET /users/me`: the caller's own profile * `GET /users/me/capabilities`: what the caller is permitted to do * `POST /users/exchange-session`: bind a session to an organization * `GET /organizations/{id}/members`: list the members of an org # Holos Foundry Source: https://docs.holos.io/foundry/index Dedicated, secure compute that processes your organization's captures and objects. > [!NOTE] > > Holos Foundry is available to **select customers only**. If your organization needs dedicated processing capacity or wants captures and scans processed on your own hardware, [contact us](mailto:hello@holos.io) to request access. **Holos Foundry** is the compute layer that processes your organization's captures and objects: a fleet of Macs (**runners**) that pull queued jobs, do the work locally, and upload the results. Foundry started as the way to build **trackable objects** — the reference descriptor that lets [Holos Capture](/capture/index) recognize a real-world object and follow it through a session — because that job is Apple-toolchain-only and takes hours on a Mac. It now runs the capture pipeline too: preparing a video's poster and scrubber previews, transcribing its audio, and extracting procedure steps from the transcript. See [Services and data residency](/foundry/services) for the full list and where each one runs. For most organizations this happens automatically on Holos-operated infrastructure and needs no setup. Foundry becomes visible to your team when you run **dedicated runners**: either capacity we manage on your behalf, or Macs your own organization operates. ## Why a dedicated Foundry > **Keep sensitive material in-house** > > With bring-your-own-Mac runners, captures and 3D models are processed on hardware your organization controls. Most services keep everything on the machine; [the residency table](/foundry/services#where-your-data-is-processed) names the exceptions rather than leaving you to assume. > **Dedicated capacity** > > Reserved runners mean your jobs aren't waiting behind anyone else's queue. Add Macs to process more captures and train more objects in parallel. > **Secure by design** > > Runners only ever pull work over outbound HTTPS: no inbound ports, no SSH. They authenticate with short-lived, scoped, revocable machine tokens. > **Managed and monitored** > > Enroll, monitor, and revoke runners from the dashboard. Builds auto-update, so your fleet stays current without manual maintenance. ## How it fits together Every service follows the same shape. Using object training as the example: 1. Someone on your team [enables object tracking](/dashboard/objects/tracking#enabling-object-tracking) for an object, choosing a view mode and training mode. This creates a **training job**. 2. The job lands in the Foundry **queue** as *Queued*. 3. An available **runner** that advertises this service claims the job and downloads what it needs. 4. The runner streams progress back while it works, then uploads the finished reference object. 5. The object's **Trackable** field flips to **Yes** and it becomes available in Holos Capture. Capture jobs work the same way but are enqueued for you: uploading a video capture queues its video prep, and each finished stage queues the next, so a capture arrives with a poster, scrubber previews, a transcript, and extracted steps without anyone requesting them. A runner can hold one job per service at a time, so a Mac granted several services works on several jobs at once rather than serializing everything behind an eight-hour training run. ## The Foundry dashboard Organizations with Foundry access see a **Foundry** section in the [dashboard](https://app.holos.io): > **[Job queue](/foundry/jobs)** > > Watch jobs move through Queued → In Progress → Done, with live progress and the service each one belongs to. > **[Runners](/foundry/runners)** > > Enroll and name the Macs that do the work, choose which services they take, and rotate or revoke their credentials. > [!NOTE] > > The Foundry section is visible to organization **admins and managers**. The Runners page additionally requires that your organization be enabled for its own runners — if it isn't, the page says so and [we can turn it on](mailto:hello@holos.io). ## Requesting access Foundry is offered as part of an engagement with the Holos team, tailored to your capacity and data-residency needs. > **[Contact us about Foundry](mailto:hello@holos.io)** > > Tell us about your object catalog and any data-residency requirements, and we'll help you set up dedicated runners. # Job queue Source: https://docs.holos.io/foundry/jobs Track jobs of every service as they move through the Foundry queue. The **Job queue** (**Foundry → Jobs** in the [dashboard](https://app.holos.io)) is the live view of every job for your organization, across every [service](/foundry/services). Each row is one unit of work — training an object, preparing a video, transcribing it, extracting its steps — and the queue shows where it is in its lifecycle. > [!NOTE] > > Foundry is available to select customers only, and the Job queue is visible to organization admins and managers. [Contact us](mailto:hello@holos.io) to request access. ## Job types Every row carries the service it belongs to. Object training and procedure extraction are requested by a person; the rest are enqueued automatically as an upload or a capture moves through the pipeline. | Type | Created when | Produces | | - | - | - | | **Object training** | Someone [enables tracking](/dashboard/objects/tracking) on an object | The ARKit reference object that makes it trackable | | **Video prep** | A video capture finishes uploading | Poster frame, scrubber previews, speech audio | | **Transcription** | Video prep finishes on a capture with no transcript | A timecoded transcript | | **Step extraction** | A transcript becomes available | The capture's procedure steps | | **Pose extraction** | A picture capture is processed | A posed 3D body mesh | | **Document conversion** | A reference PDF is added to a guidebook or procedure | A structured document | | **Procedure extraction** | Someone extracts a procedure from a converted document | A draft procedure with steps and callouts | | **Procedure alignment** | A capture is recorded against a procedure | Corroborated steps, or proposals to review | Where each of these runs, and what leaves your infrastructure, is in [Services and data residency](/foundry/services). ## Job lifecycle Every job carries a status. As a runner picks up the work, the status advances: | Status | Meaning | | - | - | | **Queued** | Waiting for an available runner to claim it. | | **In Progress** | A runner has claimed the job and is working. A progress bar shows how far along it is. | | **Done** | The work finished and its output was uploaded. | | **Rejected** | The request was dismissed before completion. | | **Failed** | A processing error occurred. The job can be requeued to try again. | A job that was claimed but stopped reporting progress — a runner losing power mid-training, say — is automatically requeued so another runner can pick it up. ## What each job shows Selecting a job opens its detail panel, which includes: * The **object** or **capture** being processed, with its thumbnail * Live **progress** and the runner currently handling it, while in progress * **Where the data went**: the provider that did the work and whether anything left your infrastructure * For object training, the **view mode**, **training mode**, and any **objects to avoid** ### Training parameters These apply to object training only. They're chosen when tracking is enabled on an object and describe how the object is handled in the real world, which helps produce an accurate reference object: **View mode** | Mode | Best for | | - | - | | **Front** | Objects viewed only from the front: wall-mounted panels, controls, screens. | | **Upright** | Objects that stay upright and are approached from any side: benchtop equipment, freestanding machines. | | **All angles** | Objects picked up and rotated freely: handheld parts and tools. Trains longer and carries a higher false-positive risk. | **Training mode** | Mode | Typical runtime | Best for | | - | - | - | | **Standard** | \~8 hours | Most objects | | **Extended** | \~18 hours | Small, handheld objects that need extra detail | See [Tracking settings](/dashboard/objects/tracking#tracking-settings) for full guidance on choosing these. ## Managing a job From the queue you can **stop and requeue** a job that's in progress, handing it back for another runner to claim. That's the "my Mac is wedged, give me my job back" case, and it's the one mutation your team needs day to day. The runner honors it on its next check-in and hands the job back cleanly rather than dropping it mid-write. To retry a failed job, or dismiss one that's no longer wanted, [ask us](mailto:hello@holos.io). Training jobs are long-running, so that part of the queue is designed to be checked in on rather than watched. The capture services are quick by comparison — a transcript is usually minutes, its steps seconds — so those rows tend to be finished by the time you look. The Foundry app on the Mac can post a notification when its jobs finish. # Runners Source: https://docs.holos.io/foundry/runners Enroll, monitor, and manage the Macs that do your processing. A **runner** is a Mac enrolled with Foundry that pulls jobs from the queue, does the work locally, and uploads the results. The **Runners** page (**Foundry → Runners** in the [dashboard](https://app.holos.io)) is where you enroll runners, watch what they're doing, and manage their credentials. > [!NOTE] > > Foundry runners are available to select customers only. This page is visible to organization admins and managers whose organization is enabled for its own runners. [Contact us](mailto:hello@holos.io) to set that up. ## Runner status Each runner reports a live status derived from when it was last seen, whether it holds a job, and what it says about its own health: | Status | Meaning | | - | - | | **Training** / **Processing** | Actively working. Its current job and progress are shown inline; the word follows the kind of work. | | **Online** | Healthy, waiting for the next job. | | **Paused** | Online but taking no new jobs, because someone paused it. A job already in flight still finishes. | | **Degraded** | The machine is alive but one of its services isn't reporting. The other services keep working; hover the pill to see which one is unwell. | | **Low disk** | Not claiming jobs because there isn't enough free space. The label names the shortfall. | | **Stalled** | Holds a job whose progress has stopped reporting (a crash or power loss). The job is automatically requeued for another runner. | | **Offline** | Hasn't checked in recently. | For each runner the page also shows its name, host machine, who enrolled it, when it was last seen, and how many jobs it has recently completed. Hovering a runner's name lists the services it serves. You can give each runner a custom icon and color to tell your fleet apart at a glance. ## Enrolling a runner The easiest way to enroll a Mac is from the Foundry app itself — no credentials to copy or paste. 1. Install the Foundry app on the Mac (the Holos team provides it as part of your setup). 2. Open it and **sign in** to your Holos organization. 3. Click **Enroll this Mac**. Foundry mints the machine credentials, stores them securely in the Mac's Keychain, and the runner starts pulling jobs right away. It shows up on this page within moments, permitted to serve the [services](/foundry/services) that build of the app supports. ### Enroll from the dashboard (alternative) If you'd rather create the credentials yourself, you can add a runner from the dashboard and configure the Mac by hand: 1. Click **Add runner** and give it a name. 2. Foundry generates a **client ID** and **client secret**. 3. Configure the runner Mac with the client ID and secret. > [!WARNING] > > The client secret is shown **exactly once**, at creation. Copy it immediately and keep it somewhere safe. Foundry never displays it again. If you lose it, rotate the secret to get a new one. Either way, the runner Mac uses its credentials to mint short-lived access tokens on its own. Nothing about enrollment opens an inbound connection to the Mac: the runner only makes outbound requests to Foundry. ## Setting up the runner Mac The runner software is a small macOS app the Holos team provides as part of your Foundry setup. Once it's enrolled, it runs in the background, polls for queued jobs, and works on one job per service at a time. It updates itself automatically, so the Mac stays current without manual maintenance. The app's **Settings → Services** switches decide what that Mac actually takes on, within what its credentials allow — useful when a machine should train objects overnight but stay out of the capture pipeline, for instance. See [Services and data residency](/foundry/services#choosing-which-services-a-runner-takes). > [!NOTE] > > Object training is disk-intensive: a runner Mac needs roughly 64 GB free to claim a training job, and reports **Low disk** instead of claiming when it doesn't have it. The capture services need far less. ## Managing runners From each runner's menu you can: * **Rotate secret**: issue a fresh client secret and invalidate the old one. Use this on a schedule or if a secret may have been exposed. The new secret is shown once. * **Revoke**: permanently disable the runner's credentials so it can no longer claim jobs. Use this when you retire a Mac. * **Rename** and **restyle**: update the display name, icon, and color. * **Copy client ID**: for matching a dashboard row to a machine. Day-to-day control of a runner lives on the Mac itself, in the Foundry app: **pause** it so it takes no new jobs, choose its **services**, **release the job it's currently working on** back to the queue, **rotate its credentials**, or **remove its enrollment**. Pausing is safe mid-job — the job in flight finishes first. > [!NOTE] > > Changing which services a credential is *permitted* to serve is a Holos-side change: [ask us](mailto:hello@holos.io) and we'll widen or narrow the grant. What the Mac takes on within that permission is yours to set, in the app. > [!NOTE] > > Rotating or revoking takes effect on the runner's next request. A runner mid-job will finish or lose it depending on timing; the job requeues automatically if interrupted. ## Where your data is processed Selecting **Where is my data processed?** on this page opens the full per-service, per-provider table: which services keep everything on the runner, and which hand material to an external provider. It always reflects what your fleet currently runs. See [Services and data residency](/foundry/services) for the same table alongside what each service does. # Services and data residency Source: https://docs.holos.io/foundry/services What each Foundry service does, where it runs, and what (if anything) leaves your infrastructure. A Foundry runner doesn't do one kind of work. Each Mac advertises a set of **services**, and the queue hands it only the job types it advertises and its credentials permit. This page is the reference for what those services are and, for each one, whether the material stays on your hardware. > [!NOTE] > > Foundry is available to select customers only. [Contact us](mailto:hello@holos.io) to discuss dedicated runners and data-residency requirements. ## The services | Service | What it does | Runs on | | - | - | - | | **Object training** | Trains the ARKit reference object that makes a 3D model [trackable](/dashboard/objects/tracking). Needs roughly 64 GB free disk and runs for hours. | Your Macs | | **Video prep** | Derives a video capture's poster frame, scrubber previews, and speech audio track. | Your Macs | | **Transcription** | Turns a video capture's audio into a timecoded transcript. | Your Macs | | **Step extraction** | Turns a capture's transcript into procedure steps. | Your Macs | | **Pose extraction** | Reconstructs a posed 3D body mesh from a picture capture. | Holos servers only | | **Document conversion** | Turns a reference PDF into a structured document its procedures can be extracted from. | Your Macs | | **Procedure extraction** | Turns a structured document into a draft procedure, steps and callouts included. | Your Macs | | **Procedure alignment** | Matches a capture against a procedure, so each recording either corroborates its steps or raises a proposal. | Your Macs | Object training is the one service that *must* run on a Mac: it depends on the Apple toolchain, which is why Foundry is a fleet of Macs in the first place. Pose extraction is the opposite case: it's hosted inference, so it runs server-side and never on your hardware. One service runs only on Holos servers for a different reason. The texture bake, which makes a large 3D scan light enough for a headset to render, needs a Blender build that exists for Linux but not for Apple silicon. It belongs to object processing rather than to a service you switch on, and no Mac is offered the work. When your own runners are offline, Holos capacity picks up the capture services so uploads still finish processing. ## Choosing which services a runner takes Each runner's credentials permit a set of services: a ceiling we set with you when your fleet is provisioned, and widen or narrow on request. Within that ceiling, the Foundry app's **Services** switches (**Settings → Services**) let the person at the Mac decide what it actually takes on: a machine can train objects overnight and stay out of the capture pipeline, for instance. A service its credentials don't permit is marked in that window rather than hidden, so "why isn't this Mac taking transcription jobs" always has a visible answer. > [!NOTE] > > Transcription requires macOS 26 or later, because it uses Apple's on-device speech models. On an older Mac the service is listed as unsupported and those jobs go elsewhere. ## Where your data is processed Some services do all their work on the machine that claimed the job. Others hand the material to an external provider, which is a decision you may need to account for. This distinction is a property of the **provider**, not the service: the same job type can be local or external depending on which provider ran it. | Service | Provider | Where it runs | What leaves your infrastructure | | - | - | - | - | | Object training | CreateML | Foundry runner | Nothing | | Video prep | AVFoundation | Foundry runner | Nothing | | Transcription | Apple Speech | Foundry runner | Nothing | | Transcription | Whisper | [fal.ai](https://fal.ai) | Capture audio | | Step extraction | Apple on-device | Foundry runner | Nothing | | Step extraction | Apple Private Cloud Compute | Apple | Transcript text | | Step extraction | [Anthropic](https://www.anthropic.com) | Anthropic | Transcript text | | Pose extraction | SAM 3D | [fal.ai](https://fal.ai) | Capture photo and/or video | | Document conversion | Docling | Foundry runner | Nothing | | Document conversion | PDFKit | Foundry runner | Nothing | | Procedure extraction | Apple on-device | Foundry runner | Nothing | | Procedure extraction | Apple Private Cloud Compute | Apple | Document text, per section | | Procedure extraction | [Anthropic](https://www.anthropic.com) | Anthropic | Document PDF and text | | Procedure alignment | Apple on-device | Foundry runner | Nothing | | Procedure alignment | Apple Private Cloud Compute | Apple | Procedure and capture step text | | Procedure alignment | [Anthropic](https://www.anthropic.com) | Anthropic | Procedure and capture step text | The same table is in the dashboard, under **Where is my data processed?** on the [Runners](/foundry/runners) page, where it always reflects what your fleet currently runs. The two rows naming [fal.ai](https://fal.ai) are served only by Holos-side capacity, never by your Macs. With your own runners online, transcription is handled by Apple's on-device models and the audio stays on the machine. The three services that read language, step extraction, procedure extraction and alignment, each offer the same choice of provider, and the choice is what decides whether anything leaves. On Apple's on-device models the work happens on the Mac and nothing is sent. Apple Private Cloud Compute and Anthropic are both hosted, so the text goes to them wherever the job was claimed. No provider credentials are ever stored on a runner, and runners only ever make outbound requests to Holos. Every finished job records which provider did the work, so what left is answerable per job rather than in general. The dashboard labels each run in the [job queue](/foundry/jobs). ### How long it stays there Material sent to [fal.ai](https://fal.ai) carries an expiry. Holos asks that the uploaded audio or photograph, and anything the provider generates from it, stop being retained one hour after the job runs. fal documents expired objects as permanently deleted and not recoverable, and a test object stopped resolving within seconds of its stated expiry. Read that as a provider honoring a request, not as a guarantee Holos gives on their behalf. Where a service runs on one of your own Macs the question does not arise, because nothing was sent. > [!NOTE] > > If your organization needs a service pinned to your own hardware, that's a fleet configuration we set up with you. [Talk to us](mailto:hello@holos.io) about residency requirements. # Getting started Source: https://docs.holos.io/getting-started From zero to your first capture in a few steps. ## 1. Create an account Sign up at [app.holos.io](https://app.holos.io/signup) or directly within the Holos Capture or Holos Learn apps. If your organization already has a Holos account, ask your admin to invite you instead — all resources are org-scoped. > **[Sign up](https://app.holos.io/signup)** > > Start your free trial at app.holos.io ## 2. Install Holos Capture Holos Capture is distributed outside the App Store via the Apple Developer Enterprise Program. Your admin will send you either an enterprise install link or a TestFlight invitation. > **[Device setup](/capture/setup)** > > Install Holos Capture and grant the required permissions. ## 3. Prepare objects to track (optional) If the session involves specific equipment or parts, upload their 3D models to [Objects](https://app.holos.io/objects) in the dashboard and [enable tracking](/dashboard/objects/tracking) on them. Trackable objects can then be picked when you start a capture. ## 4. Record a session Open Holos Capture, tap **New Capture**, name the session, select any objects to track, and tap **Start**. When you're done, raise two thumbs up to end the session. > **[Recording a session](/capture/recording)** > > Modality controls, gesture triggers, and what gets recorded. ## 5. Upload and review After recording, open the session in the Capture library and tap **Upload**. Once uploaded, it's available in the Holos dashboard for review, annotation, and linking to training. > **[Uploading](/capture/uploading)** > > Upload completed sessions and review them in the dashboard. ## What's next > **[Tracked objects](/capture/tracked-objects)** > > Configure 3D reference objects so Capture knows what to track during sessions. > **[Holos Learn](/learn/index)** > > Turn approved captures into immersive Vision Pro training courses. # Welcome Source: https://docs.holos.io/index Learn how to capture and deploy the physical processes that happen at your organization. Holos turns undocumented expertise into structured 3D task memory. Capture how the work actually gets done, then deliver it as immersive training for humans or skill libraries for downstream systems. > **[Platform overview](/platform)** > > How Capture, Learn, the dashboard, and the API fit together end-to-end. > **[Holos Capture](/capture/index)** > > Record how the work gets done. Video, hand and object tracking, and transcription, all synchronized in one spatial session. > **[Holos Learn](/learn/index)** > > Deliver captured expertise as immersive Vision Pro training. Learners step through the workflow with the expert's 3D objects. > **[API Reference](/api/index)** > > Build on the capture layer. Manage organizations, procedures, captures, and 3D objects programmatically. # Authoring courses Source: https://docs.holos.io/learn/authoring Create and edit immersive courses on Apple Vision Pro using a four-level Course → Lesson → Stage → Element hierarchy, with manual or AI-assisted authoring. Courses are authored directly in the Holos Learn app on Apple Vision Pro. > [!NOTE] > > Creating and editing courses requires admin or manager access. Content is organized around a four-level hierarchy: **Course → Lesson → Stage → Element**. Build top-down — create the course, add lessons, add stages within each lesson, then populate each stage with elements. ## App tour > **[Course library](/learn/tour-library)** > > Browse and manage your courses from the home screen. > **[Creating a course](/learn/tour-create-course)** > > Create a course manually or generate one with AI. > **[Editing a course](/learn/tour-course-detail)** > > Add lessons, build out stages, and edit course details. > **[Staging elements](/learn/tour-elements)** > > Add 3D objects, video, text, images, and recordings to stages. > **[Editing elements](/learn/tour-edit-elements)** > > Adjust position, rotation, scale, and other properties of staged elements. # Taking courses Source: https://docs.holos.io/learn/courses Browse the course library and work through lessons. ## Course library After signing in, the home screen shows your organization's course library. Courses are displayed as tiles with a thumbnail and title. Tap a course to open its detail view, which shows: * Course description * Lesson list with progress indicators * Estimated duration ## Working through a lesson Tap a lesson to begin. Holos Learn opens an immersive space and loads the first stage. Stages progress sequentially. At the end of each stage, tap **Next** to advance. Some stages may include interactive elements; follow the on-screen prompts. ### Stage types | Type | Description | | - | - | | 3D object | A positioned object in your space. Rotate and inspect it. | | Video | A video panel anchored in your view | | Text | A text panel with instructions or explanations | | Image | A static image panel | ## Progress Your progress is saved automatically. If you leave a lesson mid-way, it resumes from the last completed stage when you return. The course library shows completion status for each course. Lessons you've finished are marked with a checkmark. ## Course authoring Courses are built and published from the Holos dashboard. See [Authoring courses](/learn/authoring) for the full workflow. # Holos Learn Source: https://docs.holos.io/learn/index Deliver immersive, hands-on training on Apple Vision Pro. ![Holos Learn](https://mintcdn.com/holosinc/3qjjKOLHhziCZQk_/images/learn/learn-hero-light.png?fit=max\&auto=format\&n=3qjjKOLHhziCZQk_\&q=85\&s=bff35c15fc0f2b5989b40f5d575b1222) Holos Learn delivers captured expertise as immersive training on Apple Vision Pro. Learners step into the workflow, work through stages, and interact with the same 3D objects the expert used. Courses are authored directly in the app and deployed in hours, not months. Available on [Apple Vision Pro](https://apps.apple.com/us/app/holos-learn-in-3d/id6747681580). iPad coming soon. ## Course structure | Level | Description | | - | - | | Course | A complete training program on a topic | | Lesson | A chapter within a course | | Stage | A single step or scene within a lesson | | Element | A piece of content within a stage (3D object, video, text, image) | ## Next steps > **[Set up your device](/learn/setup)** > > Install Holos Learn and sign in. > **[Taking courses](/learn/courses)** > > Browse the course library and work through lessons. > **[Authoring courses](/learn/authoring)** > > Build courses directly in the Holos Learn app. # Device setup Source: https://docs.holos.io/learn/setup Install and set up Holos Learn on Apple Vision Pro. > [!NOTE] > > These instructions are for Holos Learn on Apple Vision Pro. iPad support is coming soon. ## Requirements * Apple Vision Pro running visionOS 2.0 or later * A Holos account ## Install from the App Store > **[Holos Learn in 3D](https://apps.apple.com/us/app/holos-learn-in-3d/id6747681580)** > > Download on the App Store 1. Open the App Store on your Apple Vision Pro 2. Search for **Holos Learn** or use the link above 3. Tap **Get** to install 4. Open the app from your home screen ## Sign in 1. Open Holos Learn 2. Enter your work email address 3. Check your email for a magic link and tap it to sign in 4. Select your organization Your session persists. You won't need to sign in again unless you sign out manually or your session expires. # Editing a course Source: https://docs.holos.io/learn/tour-course-detail Add lessons, build out stages, and edit course details in Holos Learn. The course detail view is the main editing hub. It shows the course cover image, title, description, and a scrollable list of lessons below. ## Editing course details Tap the **edit** button (top right) to update the title, description, and cover image. ## Lessons Each lesson appears as a row with a horizontal scroll of stage tiles. Use the **New Lesson** button at the bottom to add a lesson. Tap **...** on a lesson row to: * **Rename** — update title and description * **Duplicate** — copy the lesson and all its stages * **Delete** — remove the lesson permanently ### Lesson toolbar Selecting a lesson reveals a toolbar with: | Button | Action | | - | - | | Add Stage | Append a new stage to the lesson | | Shelf | Open the [element shelf](/learn/tour-elements) for the selected stage | | Properties | Open the properties panel | | Save | Save the current lesson state | ## Stages Stage tiles show a thumbnail and title. Tap a tile to load that stage into the 3D viewport. Press and drag tiles to reorder them. Tap **Done** to save the new order. Tap and hold a tile to rename it, change its thumbnail image, or delete it. # Creating a course Source: https://docs.holos.io/learn/tour-create-course Create a new course manually or generate one with AI in Holos Learn. Tap **+** in the course library to open the new course sheet. You have two creation paths. ## Manual Enter a **title** (required) and optional **description**, then tap **Create Course**. The course opens immediately in the [course detail view](/learn/tour-course-detail). ## AI-generated Switch to the AI tab to generate a full course structure from a prompt. Type your own or pick from the preset suggestions, then tap **Generate Course**. Donovan builds out the course — lessons, stages, and suggested content — in about a minute. You can edit anything it creates. > [!NOTE] > > AI course generation is available on paid plans. # Editing elements Source: https://docs.holos.io/learn/tour-edit-elements Adjust position, rotation, scale, and other properties of staged elements in Holos Learn. Tap any element in the 3D stage to select it. The **Properties panel** opens on the right showing the element name and all available controls. Tap **×** to deselect. ## Actions The top of the panel has an action bar with controls that apply to the selected element: | Action | Description | | - | - | | **Lock / Unlock** | Prevent or allow edits to the element | | **Delete** | Remove the element from the stage permanently | | **Duplicate** | Create a copy of the element in place | | **Face** | *(Media only)* Element automatically orients toward the viewer | | **Follow** | *(Media only)* Element moves with the camera rather than staying fixed | | **Group / Ungroup** | *(3D objects only)* Toggle grouping for hierarchical objects | When **Face** is on, rotation controls are disabled. When **Follow** is on, position controls are disabled. ## Move, Rotate, Scale All element types support position, rotation, and scale adjustments via three-axis sliders. Changes apply in real time. ## Text For text elements, the **Text** section lets you edit the content directly. Tap **Update** to apply. The **Format** section controls: * Font (Public Sans, Inter, JetBrains Mono) * Weight, size * Bold, italic, underline, strikethrough * Horizontal and vertical alignment * Text color ## Video controls For video and recording elements: | Control | Description | | - | - | | Play / Pause | Preview playback in the stage | | Loop | Repeat the video continuously | | Start on | **Stage Entry** plays automatically; **Tap** waits for the learner | | Volume | 0–100% | ## Highlight color 3D objects can have an optional highlight color applied as an overlay. Use the color picker to set it, or remove it to go back to the default appearance. ## Animation If a 3D object has animations built into its source model, they appear in the **Animation** section. Tap an animation to activate it and toggle looping. Only one animation can be active at a time. # Staging elements Source: https://docs.holos.io/learn/tour-elements Add 3D objects, video, text, images, and recordings to stages in Holos Learn. The **element shelf** is a browser of all assets available to add to a stage, including 3D objects from your organization's library, as well as ad hoc content like text, images, video, and recordings. Open it from the lesson toolbar using the shelf icon, then tap an asset to stage it. ## Content types | Type | Description | | - | - | | **3D object** | A model from your organization's [object library](/dashboard/objects), placed directly in the learner's space | | **Video** | A video imported from your Photos library or local files | | **Image** | An image imported from your Photos library or local files | | **Text** | A text block with instructions or context | | **Document** | A PDF attached as a reference panel | | **Recording** | A recording of your Spatial Persona with voiceover, captured directly in the app | ## Adding an element 1. Tap **+** in the element shelf 2. Select a content type 3. Choose or capture the content 4. The element saves automatically and appears in the shelf Each shelf tile shows the element type, thumbnail, file size, and processing state. Large video files may take a moment to process after import. ## 3D objects 3D objects are sourced from your organization's [object library](/dashboard/objects). Once placed in a stage, you can adjust position, rotation, scale, and opacity directly in the 3D viewport. # Course library Source: https://docs.holos.io/learn/tour-library Browse and manage your courses from the Holos Learn home screen. The **course library** is the home screen of Holos Learn. It lists all courses available to your organization. ## Course list Each course appears as a card showing the cover image, title, and description. Tap a card to open the course. ## Creating a course If you have admin or manager access, a **+** button appears at the top right. Tap it to open the new course sheet. See [Creating a course](/learn/tour-create-course). ## Access Courses respect your role: | Role | Permissions | | - | - | | Admin / Manager | Create, edit, and delete courses; take courses | | Member | Take courses only | # Platform overview Source: https://docs.holos.io/platform How Capture, Learn, and the API make it easy to capture, quantify, and deploy the physical skills that exist at your organization. Holos turns undocumented expertise into structured 3D task memory. Experts record how the work actually gets done, capturing every motion, decision, and tool interaction. The platform makes that knowledge reusable: as immersive training for human learners, or as structured skill data for downstream systems. ![Holos platform flow](https://mintcdn.com/holosinc/3qjjKOLHhziCZQk_/images/platform/platform-flow-light.png?fit=max\&auto=format\&n=3qjjKOLHhziCZQk_\&q=85\&s=6d4ee36b194b6d94bb73dc99722fa5f3) ## How it works > **[1. Capture](/capture/index)** > > Experts record how the work gets done. Synchronized video, hand tracking, object tracking, transcription, and location, packaged as a single spatial session. > **[2. Manage](https://app.holos.io)** > > The Holos dashboard is where organizations author procedures, review captures, manage 3D objects, and author training outputs. > **[3. Deliver](/learn/index)** > > Approved captures are assembled into structured courses and published to Holos Learn, where learners step into the workflow on Apple Vision Pro. ## Key concepts > **Procedure** > > The canonical step sequence for a task. Written by hand, seeded from a capture, or extracted from a document, then corroborated by every new capture. > **Guidebook** > > The rollup for one physical object: its procedures, technical documents, and 3D model, resolved by drawing number. > **Capture** > > A recorded spatial session from Holos Capture, reviewed in the dashboard and used as evidence for procedures. > **Object** > > A 3D model used for AR tracking during captures or as interactive content in courses. > **Course** > > Immersive training built from captures, organized into lessons, stages, and elements, then published to Holos Learn.