Digital twins: rebuild a robot episode as a MuJoCo scene

A twin is one episode of a robotics dataset rebuilt as an interactable simulation. Kite builds the objects the robot handles with their real size and physics, places the robot in a calibrated copy of the cell, and replays the episode’s recorded motion in it. It then judges the twin against the episode’s video and repairs it for a few rounds, and delivers the best round as a single archive you open in MuJoCo.

Beta. Twins are enabled per account. A call from an account that isn’t enabled returns 403 with code twins_not_enabled. Ask your Kite contact to turn them on.

You give Kite two things: the dataset, and the robot in it. A twin takes a few hours, so the API is asynchronous — create it, then poll it or get a webhook when it’s done.

Check a dataset first

POST /v1/twins/validate runs every check a create runs — the dataset exists and has camera video, which camera will seed the room, and the robot can drive it — and tells you how long a twin typically takes. It creates nothing and costs nothing.

curl https://api.kiteml.com/v1/twins/validate \
  -H "Authorization: Bearer $KITE_API_KEY" \
  -H "Kite-Version: 2026-10-12" \
  -H "Content-Type: application/json" \
  -d '{"source": {"type": "huggingface", "repo_id": "lerobot/svla_so101_pickplace"}, "robot": "so101"}'

{
  "object": "twin_validation",
  "ok": true,
  "source": { "type": "huggingface", "repo_id": "lerobot/svla_so101_pickplace" },
  "cameras": ["observation.images.side", "observation.images.up"],
  "scene_camera": "observation.images.side",
  "wrist_camera": null,
  "episodes": 50,
  "fps": 30,
  "format": "v3.0",
  "estimate": { "duration_s": 17720, "tokens": 5000 }
}

If something is wrong, the error names it and the field to fix. For example, a camera that doesn’t exist lists the ones that do:

{
  "error": {
    "type": "invalid_request_error",
    "code": "camera_not_found",
    "message": "no camera matches 'top'",
    "param": "camera",
    "details": { "cameras": ["observation.images.side", "observation.images.up"] }
  }
}

The robot is checked the same way. Each value the dataset records per frame drives one of the robot’s joints, so a robot that takes a different number is refused, with the ones that fit:

{
  "error": {
    "type": "invalid_request_error",
    "code": "robot_mismatch",
    "message": "lerobot/svla_so101_pickplace records 6 state values per frame and aloha takes 14; robots that take 6: koch, so100, so101",
    "param": "robot",
    "details": { "state_dim": 6, "robots": ["koch", "so100", "so101"] }
  }
}

Create a twin

curl https://api.kiteml.com/v1/twins \
  -H "Authorization: Bearer $KITE_API_KEY" \
  -H "Kite-Version: 2026-10-12" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{"source": {"type": "huggingface", "repo_id": "lerobot/svla_so101_pickplace"}, "robot": "so101"}'

The response is the twin, queued, with 202 Accepted:

{
  "id": "twin_01M3GKYV1Z7A0YV4851EGGQYPP",
  "object": "twin",
  "status": "queued",
  "progress": 0.0,
  "source": { "type": "huggingface", "repo_id": "lerobot/svla_so101_pickplace" },
  "camera": { "scene": "observation.images.side", "wrist": null, "available": ["observation.images.side", "observation.images.up"] },
  "config": { "robot": "so101", "ceiling_m": 2.9 },
  "download": null,
  "created_at": "2026-09-27T05:03:45.347344Z"
}

The dataset is checked before anything starts, so a twin that can’t work is refused at create, for free, with the same errors as validate.

Request body

Field	Type	Description
`source`	object	Required. `{"type": "huggingface", "repo_id": "..."}` — the same shape as an augmentation’s source. A `huggingface.co` URL works as `repo_id` too.
`robot`	string	Required. The robot in the episode: `so100`, `so101`, `koch`, `aloha`, `wx250s`, `wxai_biarm`, `piper`, `xarm7`, `panda`, `panda_ee_state`, or `g1_dex3`. Its recorded joints drive the twin’s robot.
`name`	string	A short name for the twin.
`task`	string	Which objects to build: the ones the robot handles. Defaults to a general manipulation brief.
`prompt`	string	How to describe the room to the reconstructor.
`camera`	string	A substring of the camera that should seed the room. Validate lists the choices.
`wrist_camera`	string	A substring of the wrist camera to use for close-ups.
`ceiling_m`	number	Floor-to-ceiling height of the real room — the scale anchor. Default `2.9`.
`webhook_metadata`	object	Echoed on the twin and on its webhook events.

Idempotency

Send an Idempotency-Key header with every create. A retry with the same key and body returns the same twin — even while the first request is still being processed — so a dropped connection never builds two twins. Reusing a key with a different body returns 409 idempotency_key_reused. One twin builds at a time per account; a second create returns 429 concurrency_limit_exceeded with a Retry-After header.

Track progress

Retrieve the twin with GET /v1/twins/:id. Poll every 30 seconds or so, or register a webhook for twin.completed, twin.failed, and twin.canceled and skip polling.

{
  "id": "twin_01M3GKYV1Z7A0YV4851EGGQYPP",
  "object": "twin",
  "status": "processing",
  "stage": "twin",
  "progress": 0.458,
  "eta_seconds": 9604,
  "status_message": "twin: 65 min in, typically 225 min",
  "stages": [
    { "stage": "ingest", "ok": true, "attempts": 1, "seconds": 8.0, "measurement": "8 probe frames, keyframe t008.png, 0 wrist frames" },
    { "stage": "author", "ok": true, "attempts": 1, "seconds": 2691.0, "measurement": "6 objects authored in 9 turns" },
    { "stage": "refine", "ok": true, "attempts": 1, "seconds": 747.0, "measurement": "MuJoCo-render refinement accepted turn4" }
  ]
}

A twin moves through queued → processing → succeeded. failed and canceled are the other final states, and canceling sits between a cancel and the build actually stopping. Each finished stage reports what it measured about its own output — ingest, author, refine, assemble, dynamics, twin, and publish. twin is the robot calibrated and replayed in the cell, judged against the episode’s video, and repaired, round by round; its measurement is the best round’s score. environment, decompose, and label are reported as skipped: the twin stage builds the room.

Download the scene

Once the twin has succeeded, download describes the whole scene as one .tar.gz:

"download": {
  "url": "/v1/twins/twin_01M3GKYV1Z7A0YV4851EGGQYPP/archive",
  "requires_auth": true,
  "expires_at": null,
  "filename": "twin_01M3GKYV1Z7A0YV4851EGGQYPP.tar.gz",
  "bytes": 126249063,
  "sha256": "6a6f2debbbf17cbd…",
  "scene": "twin_01M3GKYV1Z7A0YV4851EGGQYPP/scene.xml"
}

url is the API’s archive path. Fetch it with your API key: it redirects to a signed link to the archive, valid for an hour, and curl doesn’t send your key on to it. Check the checksum, and extract:

curl -fL "https://api.kiteml.com$DOWNLOAD_URL" -H "Authorization: Bearer $KITE_API_KEY" -o twin.tar.gz
shasum -a 256 twin.tar.gz          # compare with download.sha256
tar xzf twin.tar.gz                # twin_01M3GKYV…/scene.xml and assets/

The folder holds scene.xml — the robot, the room, and the objects; load it with mujoco.MjModel.from_xml_path(...) — and assets/, its meshes and textures. Beside them, real_twin.mp4 shows the episode next to its twin, and verdict.json holds the judge’s score and the differences it found. If no round could be exported as a whole twin, the archive holds the objects alone, and the twin stage says so. The path doesn’t expire: each fetch redirects to a fresh link. To see what’s in the archive without downloading it, list its files:

curl "https://api.kiteml.com/v1/twins/twin_01M3GKYV1Z7A0YV4851EGGQYPP/files?limit=2" \
  -H "Authorization: Bearer $KITE_API_KEY" \
  -H "Kite-Version: 2026-10-12"

{
  "object": "list",
  "data": [
    { "object": "twin_file", "path": "assets/large_cardboard_carton_0.obj", "bytes": 13421, "sha256": "a62c53e278b2…" },
    { "object": "twin_file", "path": "assets/large_cardboard_carton_0.png", "bytes": 47432, "sha256": "aa86d266a19f…" }
  ],
  "has_more": true,
  "next_cursor": "assets/large_cardboard_carton_0.png"
}

Without writing HTTP

The Kite CLI wraps the whole flow — create, wait, download, verify, and extract:

kite twin create lerobot/svla_so101_pickplace --robot so101 --out ./twins

It puts twins in the folder you give, else in $KITE_TWINS_DIR, else in ./twins, and reports absolute paths. --robot needs kiteml-cli 0.6.0 or later. Agents get the same flow as the MCP tools kite_twin_validate, kite_twin_create (with robot), and kite_twin_status, whose next field is the exact download command.

Cancel and resume

POST /v1/twins/:id/cancel stops a twin that’s building. It’s canceling for a few seconds, until the build has stopped, then canceled. A failed twin whose error.resumable is true, or a canceled twin, can continue with POST /v1/twins/:id/resume. Every stage it finished is kept, so only the rest runs again.

Errors

Status	Code	Meaning
`400`	`dataset_not_found`	The dataset doesn’t exist, or isn’t public
`400`	`dataset_gated`	The dataset is gated — accept its terms on Hugging Face, or use a public dataset
`400`	`dataset_has_no_video`	The dataset has no camera video to rebuild the room from
`400`	`camera_not_found`	No camera matches `camera`; `details.cameras` lists the real ones
`400`	`no_suitable_camera`	Every camera is a wrist camera; the room needs one that sees the scene
`400`	`robot_unsupported`	`robot` isn’t one Kite knows; `details.robots` lists them
`400`	`robot_mismatch`	`robot` takes a different number of values per frame than the dataset records; `details.robots` lists the robots that fit
`403`	`twins_not_enabled`	Twins aren’t enabled for this account yet
`409`	`twin_not_ready`	Files or the archive requested before the twin succeeded
`409`	`not_cancelable` / `already_running`	Cancel on a finished twin, or resume on one still building
`429`	`concurrency_limit_exceeded`	A twin is already building on this account — honor `Retry-After`
`429`	`capacity_exceeded`	Kite’s builders are full — honor `Retry-After`

A twin that stops part-way reports status: "failed" with error.code of interrupted, or stage_failed when a stage’s own check didn’t pass. Both are resumable.

The twin object

Field	Type	Description
`id`	string	`twin_` followed by a time-ordered id
`status`	string	`queued`, `processing`, `canceling`, `succeeded`, `failed`, or `canceled`
`progress`	number	`0` to `1`, estimated within the running stage
`stage`	string or null	The stage running now
`eta_seconds`	integer or null	While building: time left, from the stages’ typical durations
`error`	object or null	When `failed`: `code`, `message`, `stage`, `resumable`
`source`	object	The dataset the episode came from
`camera`	object or null	The camera that seeds the room, and every camera the dataset has
`config`	object	The build options you gave at create
`stages`	array	Every finished stage: `stage`, `ok`, `attempts`, `seconds`, `measurement`
`download`	object or null	Once `succeeded`: `url`, `requires_auth`, `expires_at`, `filename`, `bytes`, `sha256`, `scene`
`webhook_metadata`	object or null	Whatever you passed at create
`created_at`	string	ISO 8601, UTC

Get started

Platform API

Tools

Digital twins: rebuild a robot episode as a MuJoCo scene

Check a dataset first

Create a twin

Request body

Idempotency

Track progress

Download the scene

Without writing HTTP

Cancel and resume

Errors

The twin object

​Check a dataset first

​Create a twin

​Request body

​Idempotency

​Track progress

​Download the scene

​Without writing HTTP

​Cancel and resume

​Errors

​The twin object

Check a dataset first

Create a twin

Request body

Idempotency

Track progress

Download the scene

Without writing HTTP

Cancel and resume

Errors

The twin object