SELF-PACED WORKSHOP
Explore community movement skills
Explore jumping, ball balancing and skill handoffs using the same observation–action–reward framework.
What else can the same robot learn?
Community projects train jumping and ball balancing using different tasks, rewards and initial states. These demonstrations come from their original simulation environments and do not establish equivalent Newton or hardware performance.
| Skill | Observations and actions | Rewards and completion |
|---|---|---|
| Jumping | Body state and a jump command → joint actions | Height, landing posture and recovery |
| Basketball balance | Robot and ball state → joint actions | Balance, relative position and survival |
| Walking | Body state and velocity command → joint actions | Speed tracking, stability, energy and smoothness |
Isaac Sim supplies props and contact geometry; Isaac Lab organizes observations, rewards and reset distributions. Equal observation dimensions do not guarantee compatibility: the jump policy repurposes a command field as a launch trigger.
Define a skill’s boundaries
Walking continuously accepts velocity commands. A kick is a short skill with a start and an end. Integrating it into an autonomous task requires entry conditions, a defined exit state, and a way to stop when conditions are unsafe.
| Phase | What to check | |
|---|---|---|
| Jumping | Body state and a jump command → joint actions | Height, landing posture and recovery |
| Basketball balance | Robot and ball state → joint actions | Balance, relative position and survival |
| Walking | Body state and velocity command → joint actions | Speed tracking, stability, energy and smoothness |
Why ONNX files are not interchangeable
ONNX is a model representation, not a universal robot-skill protocol. The walking model takes 61 inputs, while the kicking residual takes 62. Both output 14 numbers, yet they are not interchangeable. Input ordering, additional state, action scaling, teacher blending, and termination conditions are all part of the runtime contract.
Loading a model only establishes that its format is readable. Units, joint order and scaling must match the controller. The models page lists the fixed weights and their roles. Chapter 6 shows the complete task assembled from these skills.
Watch the kick in three phases: before it, inspect ball and support-foot positions; during it, identify whether contact comes from the foot or another body part; afterwards, inspect recovery of support. Pause the video in each phase and describe the entry and completion conditions. This is more informative than asking only whether the ball moved.
Describe a skill by its contract
Describe a skill using entry conditions, observations, actions, and exit conditions. A kick requires a useful right-foot ball position, goal alignment, stable support, and fresh target information. Execution combines the kicking teacher with a residual correction. It must then return to a stable standing or walking state; touching the ball alone does not finish the task.
A residual adds a constrained correction to an existing action. This reduces what must be learned from scratch but inherits the teacher’s operating range. Starting a local kick far from the ball does not make it search or approach. Navigation creates a valid handoff state; the skill performs its local action.
Check your understanding
The robot is beside the ball, but its remembered position has expired. Should the kick start immediately?
Show explanation
No. Observe again and confirm the entry conditions. Proximity alone does not guarantee valid inputs.
Executing a skill is not completing a task
We currently tell the robot when to move or kick. Joint and IMU feedback support balance, but the policy does not decide where the ball is or which route to take. Next we review a complete perception/planning pipeline, then consider how learned models can take on more decisions.