Zheyu Zhuang 庄哲宇 Roboticist
Behaviour cloning turns demonstrations into a policy, and that policy stays tied to the conditions it was trained under. Change the background, move the object, shift the camera, swap the arm, and it starts to fail. Those are usually treated as four different problems. I treat them as one, and use the structure a task already contains instead of more data.
01 / About
I am a postdoctoral researcher in Robotics, Perception and Learning at KTH Royal Institute of Technology, working with Professor Danica Kragic. I received my PhD in Robotics from the Australian National University with Professor Robert Mahony.
My research asks how visuomotor policies can learn action-relevant representations that transfer across changes in visual appearance, spatial configuration, viewpoint and robot embodiment. By grounding perception in the task, interaction and the robot's own body, I aim to make policies more data-efficient and reliable beyond the conditions in which they were trained.
I approach embodied AI as an end-to-end robotics systems problem. My work spans simulation, real-robot data collection, multimodal sensing and calibration, ROS integration, policy learning, real-time control, evaluation and deployment. This systems perspective helps me connect policy design to physical constraints, diagnose failures across the pipeline, and investigate how embodied agents can learn from limited data and transfer across environments and robot platforms.
02 / Research
A useful representation of the world is not necessarily a complete one. For a robot, meaning comes from what it can do, what it is trying to achieve and how it understands its own body. My research explores how these connections can be learned and how they can endure as the world changes.
Learns where to look directly from action supervision.
Seeker uses the task and robot state to produce task-aware regions of interest, without spatial labels. Instead of treating every pixel equally, it identifies the evidence needed for the next action. The learned attention supports RGB cropping, background augmentation and point-cloud filtering across changing scenes.
Generalises by aligning perception around the robot hand.
PALM focuses on local visual evidence around the robot hand, then connects it with proprioception. The representation stays meaningful across changes in workspace, camera viewpoint and robot embodiment, allowing fine manipulation skills to transfer without relearning the entire scene.
Transfers skills across space through consistent data mirroring.
MirrorDuo reflects images, proprioception and full 6 DoF actions as one consistent transformation. Each demonstration becomes a physically coherent paired example, enabling transfer to mirrored workspaces with little additional data.
The same visual scene produces different representations when the task changes.
When trained end to end, the encoder absorbs the policy objective rather than behaving like a generic visual backbone. Even for similar images, motor supervision changes where it attends and which features it preserves. This reveals task aware visual representations.
Builds visual robustness from the encoder’s own attention.
RoboSaGA uses policy saliency to locate and protect the visual evidence needed for control, while aggressively changing irrelevant appearance during training. This improves robustness to lighting, shadows, distractors, textures and backgrounds without manually defined masks.
Learns visual self identification in multi robot environments.
Robot-centric Pooling connects visual features with proprioception so a policy can recognise the body it controls among similar-looking robots. This robot-centred perception remains reliable around unseen embodiments, self-distractors and large pixel shifts.
03 / Approach
I build robot-learning pipelines end to end, from data collection and ROS-based sensor integration to algorithm design, evaluation and deployment on physical systems.
Integrating hardware and software through ROS, with multi-sensor synchronisation, communication, calibration, control and real-time execution.
Designing, training and evaluating visuomotor and VLA policies across simulation and hardware, with attention to data efficiency, robustness and generalisation.
Combining RGB, depth, event cameras, point clouds and proprioception with visual fundation models such as DINO and SAM.
Designing data collection and evaluation pipelines, deploying policies and analysing the gap between simulated results and physical behaviour.
04 / Systems
Each brought perception, control, software and hardware together around a different robotics problem.
I co-developed GoferBot, a vision-based human–robot interaction system for collaborative furniture assembly. It combines human action recognition, visual servoing and handover behaviour so the robot can coordinate naturally with a person.
Watch GoferBot
I contributed as a member of the ACRV team that built Cartman, combining custom hardware with robotic perception for warehouse picking. The system won the 2017 Amazon Robotics Challenge, and the team leader later spun this work out into Lyro Robotics.
05 / Mentoring
I contribute through mentoring, teaching, editorial work and peer review.
Mentored student projects contributing to CoRL, RA-L, and ICRA publications, and supervised undergraduate work resulting in an IROS paper.
Tutored five graduate courses and one undergraduate course, and led a Robotic Vision Summer School workshop later incorporated into teaching at Monash University.
Associate Editor for IROS 2025 and 2026; reviewer for IEEE RA-L, IEEE T-RO, RSS, CoRL, ICRA, and IROS.
06 / Publications
Before focusing on data-efficient visuomotor policy generalisation, I worked across closed-loop control, 3D perception, human–robot collaboration and integrated manipulation systems.
07 / Off the clock
A few things that get me outside and away from a screen.
Helps me completely empty my head as the survival instinct screams, but oftentimes it's back to the familiar pattern of stopping half way and rethinking my approach.
Often question myself why I put myself through this — days without a shower — but in the end, the cleared mind, the breathtaking views, and being off the grid and off the internet make it all worth it.
Maybe it's the 10th time getting skunked, but all it takes is just one fish to keep me going.