顾乔芝士网

持续更新的前后端开发技术栈

2025年09月11日

宇树G1 遥操作采集数据json格式转lerobot实践

参考链接：unitree_IL_lerobot

1. 环境安装

1.1 LeRobot 环境安装

本项的目的是使用LeRobot开源框架训练并测试基于 Unitree 机器人采集的数据。所以首先需要安装 LeRobot 相关依赖。安装步骤如下，也可以参考LeRobot官方进行安装:


# 下载源码
git clone --recurse-submodules https://github.com/unitreerobotics/unitree_IL_lerobot.git

# 已经下载:
git submodule update --init --recursive

# 创建 conda 环境
conda create -y -n unitree_lerobot python=3.10
conda activate unitree_lerobot

# 安装 LeRobot
cd lerobot && pip install -e .

# 安装 unitree_lerobot
cd .. && pip install -e .

1.2 unitree_sdk2_python

针对 Unitree 机器人dds通讯需要安装一些依赖,安装步骤如下:

git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
cd unitree_sdk2_python  && pip install -e .

2. 数据采集与转换

2.1 数据加载测试

如果你想加载我们已经录制好的数据集, 你可以从 huggingface上加载
unitreerobotics/G1_ToastedBread_Dataset 数据集, 默认下载到
~/.cache/huggingface/lerobot/unitreerobotics. 如果想从加载本地数据请更改 root 参数

from lerobot.common.datasets.lerobot_dataset import LeRobotDataset
import tqdm

episode_index = 1
dataset = LeRobotDataset(repo_id="unitreerobotics/G1_ToastedBread_Dataset")

from_idx = dataset.episode_data_index["from"][episode_index].item()
to_idx = dataset.episode_data_index["to"][episode_index].item()

for step_idx in tqdm.tqdm(range(from_idx, to_idx)):
    step = dataset[step_idx]

可视化

cd unitree_lerobot/lerobot

python lerobot/scripts/visualize_dataset.py \
    --repo-id unitreerobotics/G1_ToastedBread_Dataset \
    --episode-index 0

2.2 数据采集

如果你想录制自己的数据集, 可以使用开源的遥操作项目avp_teleoperate 对 Unitree G1 人形机器人进行数据采集，具体可参考avp_teleoperate项目。

2.3 数据转换

使用avp_teleoperate采集的数据是采用 JSON 格式进行存储。假如采集的数据存放在$HOME/datasets/task_name 目录中，格式如下:

datasets/                               # 数据集文件夹
    └── task_name /                     # 任务名称
        ├── episode_0001                # 第一条轨迹
        │    ├──audios/                 # 声音信息
        │    ├──colors/                 # 图像信息
        │    ├──depths/                 # 深度图像信息
        │    └──data.json               # 状态以及动作信息
        ├── episode_0002
        ├── episode_...
        ├── episode_xxx

2.3.1 排序和重命名

生成 lerobot 的数据集时，最好保证数据的episode_0命名是从 0 开始且是连续的，使用下面脚本对数据进行排序处理

python unitree_lerobot/utils/sort_and_rename_folders.py \
        --data_dir $HOME/datasets/task_name


python unitree_lerobot/utils/sort_and_rename_folders.py --data_dir ~/Documents/unitrees/unitree_IL_lerobot/datasets/


2.3.2 转换

转换json格式到lerobot格式，你可以根据 ROBOT_CONFIGS 去定义自己的 robot_type

# --raw-dir     对应json的数据集目录
# --repo-id     对应自己的repo-id 
# --push_to_hub 是否上传到云端 
# --robot_type  对应的机器人类型 

python unitree_lerobot/utils/convert_unitree_json_to_lerobot.py  
    --raw-dir $HOME/datasets    
    --repo-id your_name/repo_task_name  
    --robot_type Unitree_G1_Dex3    # Unitree_Z1_Dual, Unitree_G1_Gripper, Unitree_G1_Dex3
    --push_to_hub


debug

灵巧手，宇树没有给出对应的代码，需要自己自定义转换格式：

调整
unitree_lerobot/utils/constants.py 最后添加：

# 根据实际数据结构调整配置
G1_INSPIRE_CONFIG = RobotConfig(
    motors=[
        "kLeftShoulderPitch",
        "kLeftShoulderRoll",
        "kLeftShoulderYaw",
        "kLeftElbow",
        "kLeftWristRoll",
        "kLeftWristPitch",
        "kLeftWristYaw",
        "kRightShoulderPitch",
        "kRightShoulderRoll",
        "kRightShoulderYaw",
        "kRightElbow",
        "kRightWristRoll",
        "kRightWristPitch",
        "kRightWristYaw",

        # 定义因时灵巧手的 手部电机
        'L_pinky_proximal_joint', 'L_ring_proximal_joint', 'L_middle_proximal_joint',
        'L_index_proximal_joint', 'L_thumb_proximal_pitch_joint', 'L_thumb_proximal_yaw_joint',
        'R_pinky_proximal_joint', 'R_ring_proximal_joint', 'R_middle_proximal_joint',
        'R_index_proximal_joint', 'R_thumb_proximal_pitch_joint', 'R_thumb_proximal_yaw_joint'
    ],
    cameras=[
        "cam_left_high",
        "cam_right_high",
        # "cam_left_wrist", # 缺少腕部相机，暂时去掉
        # "cam_right_wrist",
    ],
    camera_to_image_key={
        'color_0': 'cam_left_high',
        'color_1': 'cam_right_high',
        # 'color_2': 'cam_left_wrist',
        # 'color_3': 'cam_right_wrist'
    },
    json_state_data_name=['left_arm', 'right_arm', 'left_ee', 'right_ee'],  # 对应数据格式修改手部 key
    json_action_data_name=['left_arm', 'right_arm', 'left_ee', 'right_ee']  
)

ROBOT_CONFIGS = {
    "Unitree_Z1_Single": Z1_SINGLE_CONFIG,
    "Unitree_Z1_Dual": Z1_CONFIG,
    "Unitree_G1_Gripper": G1_GRIPPER_CONFIG,
    "Unitree_G1_Dex3": G1_DEX3_CONFIG,
    "Unitree_G1_Inspire": G1_INSPIRE_CONFIG,
}


运行：

python convert_unitree_json_to_lerobot.py --raw-dir ~/Documents/unitrees/unitree_IL_lerobot/datasets/ --repo-id my_test/test01 --robot_type Unitree_G1_Inspire

报错：

==> Cached 1 episodes
Map: 100%|█████████████████████████| 1249/1249 [00:00<00:00, 6208.42 examples/s]
Creating parquet from Arrow format: 100%|████████| 2/2 [00:00<00:00, 597.48ba/s]
  0%|                                                     | 0/1 [00:11<?, ?it/s]
Traceback (most recent call last):
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/utils/convert_unitree_json_to_lerobot.py", line 396, in <module>
    tyro.cli(json_to_lerobot)
  File "/home/ubuntu/anaconda3/envs/unitree_lerobot/lib/python3.10/site-packages/tyro/_cli.py", line 230, in cli
    return run_with_args_from_cli()
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/utils/convert_unitree_json_to_lerobot.py", line 377, in json_to_lerobot
    dataset = populate_dataset(
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/utils/convert_unitree_json_to_lerobot.py", line 351, in populate_dataset
    dataset.save_episode()
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/common/datasets/lerobot_dataset.py", line 879, in save_episode
    video_paths = self.encode_episode_videos(episode_index)
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/common/datasets/lerobot_dataset.py", line 983, in encode_episode_videos
    encode_video_frames(img_dir, video_path, self.fps, overwrite=True)
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/common/datasets/video_utils.py", line 292, in encode_video_frames
    subprocess.run(ffmpeg_cmd, check=True, stdin=subprocess.DEVNULL)
  File "/home/ubuntu/anaconda3/envs/unitree_lerobot/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/ubuntu/anaconda3/envs/unitree_lerobot/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/ubuntu/anaconda3/envs/unitree_lerobot/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg'

解决：

sudo apt update
sudo apt install ffmpeg

重新运行

python convert_unitree_json_to_lerobot.py --raw-dir ~/Documents/unitrees/unitree_IL_lerobot/datasets/ --repo-id my_test/test01 --robot_type Unitree_G1_Inspire


报错

Traceback (most recent call last):
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/utils/convert_unitree_json_to_lerobot.py", line 396, in <module>
    tyro.cli(json_to_lerobot)
  File "/home/ubuntu/anaconda3/envs/unitree_lerobot/lib/python3.10/site-packages/tyro/_cli.py", line 230, in cli
    return run_with_args_from_cli()
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/utils/convert_unitree_json_to_lerobot.py", line 377, in json_to_lerobot
    dataset = populate_dataset(
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/utils/convert_unitree_json_to_lerobot.py", line 351, in populate_dataset
    dataset.save_episode()
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/common/datasets/lerobot_dataset.py", line 879, in save_episode
    video_paths = self.encode_episode_videos(episode_index)
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/common/datasets/lerobot_dataset.py", line 983, in encode_episode_videos
    encode_video_frames(img_dir, video_path, self.fps, overwrite=True)
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/common/datasets/video_utils.py", line 292, in encode_video_frames
    subprocess.run(ffmpeg_cmd, check=True, stdin=subprocess.DEVNULL)
  File "/home/ubuntu/anaconda3/envs/unitree_lerobot/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ffmpeg', '-f', 'image2', '-r', '30', '-i', '/home/ubuntu/.cache/huggingface/lerobot/my_test/test01/images/observation.images.cam_left_high/episode_000000/frame_%06d.png', '-vcodec', 'libsvtav1', '-pix_fmt', 'yuv420p', '-g', '2', '-crf', '30', '-loglevel', 'error', '-y', '/home/ubuntu/.cache/huggingface/lerobot/my_test/test01/videos/chunk-000/observation.images.cam_left_high/episode_000000.mp4']' returned non-zero exit status 1.

从错误信息来看，ffmpeg 命令执行失败，返回了一个非零退出状态。这通常意味着 ffmpeg 在尝试编码视频时遇到了问题。虽然 ffmpeg 已经被找到并且可以运行，但命令中的某些参数或输入文件可能存在问题。

验证 FFmpeg 编码器支持： 你使用的 -vcodec libsvtav1 参数指定使用 SVT-AV1 编码器。确保你的 ffmpeg 安装版本支持此编码器。你可以通过以下命令检查是否支持 SVT-AV1 编码器：

ffmpeg -encoders | grep libsvtav1

如果没有输出，说明你的 ffmpeg 不支持 SVT-AV1 编码器。你可以考虑更换为其他编码器，如 libx264，这是一个广泛支持的编码器。

修改方法：修改 LeRobot 的视频编码逻辑

你需要修改
lerobot/common/datasets/video_utils.py 文件中的 encode_video_frames 函数，将编码器从 libsvtav1 改为 libx264。

def encode_video_frames(
    imgs_dir: Path | str,
    video_path: Path | str,
    fps: int,
    vcodec: str = "libx264",
    # vcodec: str = "libsvtav1",
    pix_fmt: str = "yuv420p",
    g: int | None = 2,
    crf: int | None = 30,
    fast_decode: int = 0,
    log_level: str | None = "error",
    overwrite: bool = False,
) -> None:


重新运行，ok

python convert_unitree_json_to_lerobot.py --raw-dir ~/Documents/unitrees/unitree_IL_lerobot/datasets/ --repo-id my_test/test01 --robot_type Unitree_G1_Inspire

可视化

cd unitree_lerobot/lerobot

python lerobot/scripts/visualize_dataset.py     --repo-id my_test/test01     --episode-index 0


报错

/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/scripts/visualize_dataset.py:150: DeprecationWarning: Use `set_time(sequence=…)` instead.
    See: https://www.rerun.io/docs/reference/migration/migration-0-23 for more details.
  rr.set_time_sequence("frame_index", batch["frame_index"][i].item())
/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/scripts/visualize_dataset.py:151: DeprecationWarning: Use `set_time(timestamp=seconds)` or `set_time(duration=seconds)` instead.
    See: https://www.rerun.io/docs/reference/migration/migration-0-23 for more details.
  rr.set_time_seconds("timestamp", batch["timestamp"][i].item())
  0%|                                                    | 0/40 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/scripts/visualize_dataset.py", line 292, in <module>
    main()
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/scripts/visualize_dataset.py", line 288, in main
    visualize_dataset(dataset, **vars(args))
  File "/home/ubuntu/Documents/unitrees/unitree_IL_lerobot/unitree_lerobot/lerobot/lerobot/scripts/visualize_dataset.py", line 161, in visualize_dataset
    rr.log(f"action/{dim_idx}", rr.Scalar(val.item()))
AttributeError: module 'rerun' has no attribute 'Scalar'. Did you mean: 'Scalars'?
k可视化报错

原因

由于 Rerun 可视化库的 API 从 v0.22 升级到 v0.23+ 后发生了不兼容的变更，导致 lerobot 的 visualize_dataset.py 脚本中调用的旧 API 已被废弃或移除。

替代方案：使用 rr.Scalars（注意是复数）

+++++++++++++++一下是分析过程++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

解决方案：修改 visualize_dataset.py

你需要修改
/unitree_lerobot/lerobot/scripts/visualize_dataset.py 文件中的以下部分。

修改日志记录部分（修复 Scalar错误）

原始代码（出错）：

python


深色版本
for dim_idx, val in enumerate(batch["action"][i]):
    rr.log(f"action/{dim_idx}", rr.Scalar(val.item()))

修改为（使用 Scalars）：

python


深色版本
for dim_idx, val in enumerate(batch["action"][i]):
    rr.log(f"action/{dim_idx}", rr.Scalars([val.item()]))

注意：rr.Scalars 接收一个列表（即使是单个值也要用 [val] 包裹）。

修改时间设置部分（修复弃用警告）

原始代码：

python


深色版本
rr.set_time_sequence("frame_index", batch["frame_index"][i].item())
rr.set_time_seconds("timestamp", batch["timestamp"][i].item())

修改为（Rerun v0.23+ 新语法）：

python


深色版本
rr.set_time_sequence("frame_index", batch["frame_index"][i].item())
rr.set_time_seconds("timestamp", batch["timestamp"][i].item())

等等！看起来代码没变？其实你看到的只是警告（warning），不是错误。但为了兼容未来版本，建议按官方提示更新（虽然目前 set_time_sequence 和 set_time_seconds 仍可用）。

目前可以暂时保留，但未来应迁移到：

python


深色版本
# 新推荐方式（可选，当前非必须）
rr.send_columns(
    "log_time",
    times=[rr.TimeSecondsColumn("timestamp", batch["timestamp"][i].item())],
    sequences=[rr.SequenceColumn("frame_index", batch["frame_index"][i].item())]
)

但为了简单起见，建议先只修复 Scalar 问题。

完整改修建议（只改关键部分）

找到 visualize_dataset.py 中的 visualize_dataset 函数，修改如下：

def visualize_dataset(dataset, num_samples=100, step=1):
    # ... 前面的代码不变 ...

    for i in range(0, min(num_samples, len(dataset)), step):
        batch = dataset[i]

        #  更新时间（保留原样，警告可暂时忽略）
        rr.set_time_sequence("frame_index", batch["frame_index"][i].item())
        rr.set_time_seconds("timestamp", batch["timestamp"][i].item())

        #  修复 action 可视化
        for dim_idx, val in enumerate(batch["action"][i]):
            rr.log(f"action/{dim_idx}", rr.Scalars([val.item()]))  # 使用 Scalars

        # 如果你还可视化 observation，也需类似修改
        if "observation" in batch:
            for dim_idx, val in enumerate(batch["observation"][i]):
                rr.log(f"observation/{dim_idx}", rr.Scalars([val.item()]))

        # 可视化图像（如果有）
        if "observation.images" in batch:
            for cam_name, img in batch["observation.images"][i].items():
                rr.log(f"camera/{cam_name}", rr.Image(img.numpy()))

    # ... 后面的代码不变 ...

+++++++++++++++以上是分析过程++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

解决

    for batch in tqdm.tqdm(dataloader, total=len(dataloader)):
        # iterate over the batch
        for i in range(len(batch["index"])):
            rr.set_time_sequence("frame_index", batch["frame_index"][i].item())
            rr.set_time_seconds("timestamp", batch["timestamp"][i].item())

            # display each camera image
            for key in dataset.meta.camera_keys:
                # TODO(rcadene): add `.compress()`? is it lossless?
                rr.log(key, rr.Image(to_hwc_uint8_numpy(batch[key][i])))

            # display each dimension of action space (e.g. actuators command)
            if "action" in batch:
                for dim_idx, val in enumerate(batch["action"][i]):
                    rr.log(f"action/{dim_idx}", rr.Scalars(val.item()))

            # display each dimension of observed state space (e.g. agent position in joint space)
            if "observation.state" in batch:
                for dim_idx, val in enumerate(batch["observation.state"][i]):
                    rr.log(f"state/{dim_idx}", rr.Scalars(val.item()))

            if "next.done" in batch:
                rr.log("next.done", rr.Scalars(batch["next.done"][i].item()))

            if "next.reward" in batch:
                rr.log("next.reward", rr.Scalars(batch["next.reward"][i].item()))

            if "next.success" in batch:
                rr.log("next.success", rr.Scalars(batch["next.success"][i].item()))

3. 训练

act

cd unitree_lerobot/lerobot

python lerobot/scripts/train.py \
    --dataset.repo_id=unitreerobotics/G1_ToastedBread_Dataset \
    --policy.type=act

标签：traceback 

作者:gpyoja , 分类:技术文章 , 浏览:2 , 评论:0

控制面板
您好，欢迎到访网站！
  查看权限
网站分类
最新留言