i4h Workflow — Convert Dataset

Purpose

Convert an agentic HDF5 recording into a LeRobot dataset (parquet + meta + videos). Use when the user asks to convert HDF5, prepare for training, or export to LeRobot.

Base Code

These steps drive the i4h-workflows base code (the workflows/agentic/ tree). To reuse an existing checkout, set I4H_WORKFLOWS to its path (no clone happens). Otherwise this resolves the current repo, or clones to ~/i4h-workflows — pick that default without prompting. Run every command below from the resolved root:

# Resolve the i4h-workflows base code (provides workflows/agentic/).
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/agentic" ]; then
  ROOT="${I4H_WORKFLOWS:-$HOME/i4h-workflows}"
  [ -d "$ROOT/workflows/agentic" ] || git clone https://github.com/isaac-for-healthcare/i4h-workflows "$ROOT"
fi
export I4H_WORKFLOWS="$ROOT"; cd "$ROOT"

Basics

• Use the same --env that produced the HDF5.
• Env config (source of truth): workflows/agentic/config/environments/<env>.yaml supplies the robot, task, cameras, and dataset.* (action/state names, splits, modality) converter defaults.
• Output goes to HF_LEROBOT_HOME/<repo-id>.

Run

Run the steps below in order. Each step is a separate bash call; variables persist in the local agent's tmux session.

Step 1 — setup and resolve HDF5

REPO_ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"; [ -d "$REPO_ROOT/workflows/agentic" ] || REPO_ROOT="$HOME/i4h-workflows"
ENV_ID=scissor_pick_and_place
RUNS_ROOT="${REPO_ROOT}/workflows/agentic/runs"

# Point HDF5_PATH at a real recording (absolute path). Recordings come from teleop, mimic, or
# validate (which writes data/verify.hdf5 under each runs/eval_* dir). List candidates newest-first:
#   find "${RUNS_ROOT}" -name '*.hdf5' -printf '%TY-%Tm-%Td %TH:%TM  %p\n' | sort -r | head
HDF5_PATH="${HDF5_PATH:-}"
if [ ! -f "${HDF5_PATH}" ]; then
  echo "convert: set HDF5_PATH to an existing .hdf5 (got '${HDF5_PATH:-<unset>}'). Candidates:" >&2
  find "${RUNS_ROOT}" -name '*.hdf5' -printf '%TY-%Tm-%Td %TH:%TM  %p\n' 2>/dev/null | sort -r | head
  exit 1
fi

RUN_DIR="${RUNS_ROOT}/convert_${ENV_ID}_$(date +%Y%m%d_%H%M%S)"
mkdir -p "${RUN_DIR}/logs"
ln -sfn "${RUN_DIR}" "${RUNS_ROOT}/.latest"
export HF_LEROBOT_HOME="${RUN_DIR}/lerobot"

Step 2 — convert

"${REPO_ROOT}/workflows/agentic/dataset/run.sh" \
  --env "${ENV_ID}" \
  --hdf5-path "${HDF5_PATH}" \
  --repo-id "local/${ENV_ID}" \
  --video-codec h264 \
  --overwrite \
  2>&1 | tee "${RUN_DIR}/logs/convert.log"

Notes

• --video-codec h264 is required. The converter's default AV1 codec breaks GR00T's decord video reader at finetune time.
• Scissor SO-ARM generates meta/modality.json from YAML splits and does not need dataset.modality_template_path.
• G1 locomanip and assemble-trocar use dataset.modality_template_path from the env YAML.
• All camera streams are resized to the env YAML policy.image_size (override with --image-size H W), normalizing mixed-resolution cameras (e.g. head cam + overview cam) to the one size the modality config expects.

Verify

• ${HF_LEROBOT_HOME}/local/${ENV_ID}/meta/info.json exists.
• Log reports the saved episode count.
• Per-episode video files are present under ${HF_LEROBOT_HOME}/local/${ENV_ID}/.

Prerequisites

• Workflow set up via [[i4h-workflow-setup]] (the .venv must exist).
• An existing HDF5 recording to convert (set HDF5_PATH to an absolute path; the Run block lists candidates if it's unset or wrong).
• The same --env that produced the HDF5 (its YAML supplies robot, task, camera, modality, and converter defaults).
• HF_LEROBOT_HOME set to the output location for <repo-id>.

Limitations

• --video-codec h264 is required; the converter's default AV1 codec breaks GR00T's decord reader at finetune time.
• All camera streams are resized to the env YAML policy.image_size (override with --image-size H W).
• G1 locomanip and assemble-trocar require dataset.modality_template_path from the env YAML; scissor SO-ARM generates meta/modality.json from YAML splits.

Troubleshooting

• Error: .venv not found / module import fails - Cause: workflow not set up. Fix: run [[i4h-workflow-setup]] first.
• Error: input HDF5 not found - Cause: wrong or missing --hdf5-path. Fix: point HDF5_PATH at an existing recording.
• Error: decord fails to read video at finetune time - Cause: dataset written with the default AV1 codec. Fix: re-convert with --video-codec h264.
• Error: missing/incorrect modality config - Cause: wrong --env, so robot/task/camera/modality defaults do not match the HDF5. Fix: use the same --env that produced the recording.

Final Response

Report source HDF5, dataset path, repo id, episode count, skipped or failed episodes.