Microduck Workshop
简体中文English
GitHub ↗
Microduck Workshop / Course chapters

SELF-PACED WORKSHOP

From engineered pipelines to intelligent robots

See how the engineered football pipeline works, then identify what learned policies and reasoning agents could take over.

Chapter 6 · Estimated learning time: 15 minutes

How the engineered method scores

The earlier chapters provide walking, skating and kicking skills. A complete task must also decide where to look, where to move, when to contact the ball and how to recover. This chapter is a guided overview: no installation, code execution or position calculations are required.

Scroll horizontally to view the full diagram

People design the decision flow; existing policies execute motion.Observe: Camera, ToF and IMU → Locate and plan: Ball, goal and approach route → Align and act: Walking, kicking or skating → Check the outcome: Goal, fall, stopping and recoveryObserveCamera, ToF and IMULocate and planBall, goal and approach routeAlign and actWalking, kicking or skatingCheck the outcomeGoal, fall, stopping and recoveryPeople design the decision flow; existing policies execute motion.Observe: Camera, ToF and IMU → Locate and plan: Ball, goal and approach route → Align and act: Walking, kicking or skating → Check the outcome: Goal, fall, stopping and recoveryObserveCamera, ToF and IMULocate and planBall, goal and approach routeAlign and actWalking, kicking or skatingCheck the outcomeGoal, fall, stopping and recovery
People design the decision flow; existing policies execute motion.

Camera projection and frame transforms relate image targets to the robot. A task prior—stand behind the ball and face the goal—defines the approach target. These calculations translate positions and directions; we omit their derivation here. Walking uses ROS 2 Nav2/Navfn planning with a custom tracker; skating uses visual feedback. Isaac ROS offers integration options for perception, but it is not running in this experiment.

Watch a walking shot

A complete walking development example, including observation, repeated alignment, kicking and recovery. The observer view is re-rendered from recorded physics with the original synchronized camera inset. One success is not a reliability estimate.

The look-down and alignment cycles are real controller behavior. Short turns are followed by settling and fresh near-ball observations; a changed heading error can trigger another cycle. Continuous observation, improved turn braking and separate entry/exit thresholds could reduce switching, but require paired physical tests. The complete real-time sequence remains visible.

Watch a gliding shot

Gliding development example: contact at 6.79 s, whole-ball crossing at 8.13 s, then settling before the goal. The robot stays on its wheels throughout; no leg-kick transition occurs. The re-rendered observer view is synchronized with the original camera.

The decision rules differ: skating requires approach and braking control; walking must enter the kicking policy in a suitable pose. A different task requires new handoff rules.

Separate demonstrations from evaluation

New 10 cm-ball sceneIndependent full tasksObserved limitation
Walking shot0/4No falls; observation or handoff failed
Gliding shot1/4; goals 2/4Stopping or goal-base localization failed

Each group has four predeclared trials, one new seed per floor. Floor and seed change together, so these trials do not establish a causal floor effect. Detection mAP50 is 0.978, yet the full task remains inconsistent: recognition, geometric estimation and execution are different problems. Historical scene scores are kept separate.

Learn actions and how to organize them

Engineered pipelines are interpretable and easy to inspect step by step. They also require people to specify stages, thresholds and recovery. Requests such as moving the ball elsewhere or going around a chair create more integration work. Learned policies and reusable skills can carry more of that experience.

Scroll horizontally to view the full diagram

The next three chapters address action, task organization and variation.GR00T: Learn actions from images, language and demonstrations → ASPIRE: Try, diagnose, revise programs and retain skills → Cosmos + Isaac: Expand data, compose scenes and evaluate generalizationGR00TLearn actions from images,language and demonstrationsASPIRETry, diagnose, reviseprograms and retain skillsCosmos + IsaacExpand data, compose scenesand evaluate generalizationThe next three chapters address action, task organization and variation.GR00T: Learn actions from images, language and demonstrations → ASPIRE: Try, diagnose, revise programs and retain skills → Cosmos + Isaac: Expand data, compose scenes and evaluate generalizationGR00TLearn actions from images, language anddemonstrationsASPIRETry, diagnose, revise programs and retainskillsCosmos + IsaacExpand data, compose scenes and evaluategeneralization
The next three chapters address action, task organization and variation.

Foundation models can reduce task-specific decision programming. They still need action interfaces, underlying movement skills and outcome checks. The following chapters explain the integration design and inspected interfaces; they do not claim a completed GR00T or ASPIRE Microduck football experiment.

References