Tutorial 1: Control Robot

Objective: Learn how to send control commands to a robot in simulation.

What you’ll learn:

• How to add a robot to your scenario

• How to set joint position targets using set_dof_targets()

• How to step the physics simulation with simulate()

• Action formats (dictionary and tensor)

Prerequisites: Completed Tutorial 0: Static Scene

Estimated time: 20 minutes

---

Running the Tutorial

python examples/1_control_robot.py  --sim <simulator>

you can also render in the headless mode by adding --headless flag. By using this, there will be no window popping up and the rendering will also be faster.

By running the above command, you will give random control actions to the robot and it will automatically record a video.

Examples

IsaacSim

python examples/1_control_robot.py  --sim isaacsim

Isaac Gym

python examples/1_control_robot.py  --sim isaacgym

Mujoco

# For mac users, replace python with mjpython.
python examples/1_control_robot.py  --sim mujoco --headless

Note that we find the non-headless mode of Mujoco is not stable. So we recommend using the headless mode.

Genesis

python examples/1_control_robot.py  --sim genesis

Note that we find the headless mode of Genesis is not stable. So we recommend using the non-headless mode.

Sapien

python examples/1_control_robot.py  --sim sapien3

Pybullet

python examples/1_control_robot.py  --sim pybullet

Newton

python examples/1_control_robot.py  --sim newton

You will get the following videos:

Isaac Sim

Isaac Gym

MuJoCo

Genesis

SAPIEN

PyBullet

Newton

Code Highlights

Robot Control: Use handler.set_dof_targets(actions) to control robot joints. Actions follow the format:

actions = [
    {
        robot.name: {
            "dof_pos_target": {
                joint_name: (
                    torch.rand(1).item()
                    * (robot.joint_limits[joint_name][1] - robot.joint_limits[joint_name][0])
                    + robot.joint_limits[joint_name][0]
                )
                for joint_name in robot.joint_limits.keys()
            }
        }
    }
    for _ in range(scenario.num_envs)
]
handler.set_dof_targets(actions)
handler.simulate()

Key Points:

• Can also use direct tensor actions instead of dictionary format ，Action order matches robot.actuators and robot.joint_limits configuration

• set_dof_targets() only sets targets - call handler.simulate() to step physics

• Unlike Task’s step(), Handler requires separate simulate() call