Bilateral Teleoperation for Dexterous Manipulation with Maestro
This project developed bilateral teleoperation capabilities for the Maestro hand exoskeleton, enabling haptic feedback and human-in-the-loop control for dexterous manipulation tasks. The system implemented real-time optimization algorithms to provide intuitive control of robotic hands through human finger motion.
The Maestro hand exoskeleton with a hand visualization in the Unity engine. The forward kinematics model is used to estimate finger joint angles based on measurements from robot joint angles.
The virtual hand in Unity, interacting with virtual objects that impose physics interactions. The expected fingertip forces can be provided as haptic feedback through Maestro's actuation.
The Maestro hand exoskeleton is used to teleoperate the remote robot hand, manipulating a block.
The Maestro hand exoskeleton is used to teleoperate the remote robot end-effector fingers, whereas the paired Lambda haptic device teleoperates the end-effector position and pose.
A visualization of the baseline virtual hand in Unity.
Contributions:
Developed bilateral teleoperation control algorithms in C++ implementing real-time kinematic mapping between human finger motion and 8-DOF robotic hand
Implemented forward kinematics models using four-bar linkage equations and Jacobian calculations for haptic force reflection in Unity VR environment
Designed human-in-the-loop control architecture with real-time optimization algorithms for intuitive multi-finger manipulation tasks
Validated teleoperation performance through controlled experiments demonstrating seamless master-slave control coupling with force feedback
Collaborators: Haiyun Zhang, Job Ramirez, Stefano Dalla Gasperina, Ashish D. Deshpande