Publications

2024

1. 

   Visual Attention Based Cognitive Human–Robot Collaboration for Pedicle Screw Placement in Robot-Assisted Orthopedic Surgery

   Chen Chen, Qikai Zou, Yuhang Song, and 4 more authors

   In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct 2024

   Abs arXiv Bib Website

   Current orthopedic robotic systems largely focus on navigation, aiding surgeons in positioning a guiding tube but still requiring manual drilling and screw placement. The automation of this task not only demands high precision and safety due to the intricate physical interactions between the surgical tool and bone but also poses significant risks when executed without adequate human oversight. As it involves continuous physical interaction, the robot should collaborate with the surgeon, understand the human intent, and always include the surgeon in the loop. To achieve this, this paper proposes a new cognitive human–robot collaboration framework, including the intuitive AR-haptic human–robot interface, the visual-attention-based surgeon model, and the shared interaction control scheme for the robot. User studies on a robotic platform for orthopedic surgery are presented to illustrate the performance of the proposed method. The results demonstrate that the proposed human–robot collaboration framework outperforms full robot and full human control in terms of safety and ergonomics.

   @inproceedings{chen2024visual,
     address = {Abu Dhabi, United Arab Emirates},
     author = {Chen, Chen and Zou, Qikai and Song, Yuhang and Yu, Mingrui and Zhu, Senqiang and Song, Shiji and Li, Xiang},
     booktitle = {2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     doi = {10.1109/IROS58592.2024.10801930},
     isbn = {979-8-3503-7770-5},
     month = oct,
     pages = {7078--7084},
     publisher = {IEEE},
     title = {Visual Attention Based Cognitive Human--Robot Collaboration for Pedicle Screw Placement in Robot-Assisted Orthopedic Surgery},
     year = {2024}
   }

2. 

   A Unified Interaction Control Framework for Safe Robotic Ultrasound Scanning with Human-Intention-Aware Compliance

   Xiangjie Yan, Shaqi Luo, Yongpeng Jiang, and 6 more authors

   In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct 2024

   Abs arXiv Bib

   The ultrasound scanning robot operates in environments where frequent human-robot interactions occur. Most existing control methods for ultrasound scanning address only one specific interaction situation or implement hard switches between controllers for different situations, which compromises both safety and efficiency. In this paper, we propose a unified interaction control framework for ultrasound scanning robots capable of handling all common interactions, distinguishing both human-intended and unintended types, and adapting with appropriate compliance. Specifically, the robot suspends or modulates its ongoing main task if the interaction is intended, e.g., when the doctor grasps the robot to lead the end effector actively. Furthermore, it can identify unintended interactions and avoid potential collision in the null space beforehand. Even if that collision has happened, it can become compliant with the collision in the null space and try to reduce its impact on the main task (where the scan is ongoing) kinematically and dynamically. The multiple situations are integrated into a unified controller with a smooth transition to deal with the interactions by exhibiting human-intention-aware compliance. Experimental results validate the framework’s ability to cope with all common interactions including intended intervention and unintended collision in a collaborative carotid artery ultrasound scanning task.

   @inproceedings{yan24unified,
     address = {Abu Dhabi, United Arab Emirates},
     author = {Yan, Xiangjie and Luo, Shaqi and Jiang, Yongpeng and Yu, Mingrui and Chen, Chen and Zhu, Senqiang and Huang, Gao and Song, Shiji and Li, Xiang},
     booktitle = {2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     doi = {10.1109/IROS58592.2024.10801755},
     isbn = {979-8-3503-7770-5},
     month = oct,
     pages = {14004--14011},
     publisher = {IEEE},
     title = {A Unified Interaction Control Framework for Safe Robotic Ultrasound Scanning with Human-Intention-Aware Compliance},
     year = {2024}
   }

3. 

   Independence in the Home: A Wearable Interface for a Person with Quadriplegia to Teleoperate a Mobile Manipulator

   Akhil Padmanabha, Janavi Gupta, Chen Chen, and 5 more authors

   In Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, Mar 2024

   Best Paper Abs arXiv Bib Code Website

   Best Paper Award in Systems

   Teleoperation of mobile manipulators within a home environment can significantly enhance the independence of individuals with severe motor impairments, allowing them to regain the ability to perform self-care and household tasks. There is a critical need for novel teleoperation interfaces to offer effective alternatives for individuals with impairments who may encounter challenges in using existing interfaces due to physical limitations. In this work, we iterate on one such interface, HAT (Head-Worn Assistive Teleoperation), an inertial-based wearable integrated into any head-worn garment. We evaluate HAT through a 7-day in-home study with Henry Evans, a non-speaking individual with quadriplegia who has participated extensively in assistive robotics studies. We additionally evaluate HAT with a proposed shared control method for mobile manipulators termed Driver Assistance and demonstrate how the interface generalizes to other physical devices and contexts. Our results show that HAT is a strong teleoperation interface across key metrics including efficiency, errors, learning curve, and workload. Code and videos are located on our project website.

   @inproceedings{padmanabha24independence,
     address = {Boulder, CO, USA},
     author = {Padmanabha, Akhil and Gupta, Janavi and Chen, Chen and Yang, Jehan and Nguyen, Vy and Weber, Douglas J and Majidi, Carmel and Erickson, Zackory},
     booktitle = {Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction},
     doi = {10.1145/3610977.3634964},
     isbn = {9798400703225},
     month = mar,
     pages = {542--551},
     publisher = {ACM},
     title = {Independence in the Home: A Wearable Interface for a Person with Quadriplegia to Teleoperate a Mobile Manipulator},
     urldate = {2024-03-12},
     year = {2024}
   }

4. 

   A Complementary Framework for Human–Robot Collaboration with a Mixed AR–Haptic Interface

   Xiangjie Yan, Yongpeng Jiang, Chen Chen, and 4 more authors

   IEEE Transactions on Control Systems Technology, Jan 2024

   Abs Bib

   There is invariably a tradeoff between safety and efficiency for collaborative robots (cobots) in human–robot collaborations (HRCs). Robots that interact minimally with humans can work with high speed and accuracy but cannot adapt to new tasks or respond to unforeseen changes, whereas robots that work closely with humans can but only by becoming passive to humans, meaning that their main tasks are suspended and efficiency compromised. Accordingly, this article proposes a new complementary framework for HRC that balances the safety of humans and the efficiency of robots. In this framework, the robot carries out given tasks using a vision-based adaptive controller, and the human expert collaborates with the robot in the null space. Such a decoupling drives the robot to deal with existing issues in task space [e.g., uncalibrated camera, limited field of view (FOV)] and null space (e.g., joint limits) by itself while allowing the expert to adjust the configuration of the robot body to respond to unforeseen changes (e.g., sudden invasion, change in environment) without affecting the robot’s main task. In addition, the robot can simultaneously learn the expert’s demonstration in task space and null space beforehand with dynamic movement primitives (DMPs). Therefore, an expert’s knowledge and a robot’s capability are explored and complement each other. Human demonstration and involvement are enabled via a mixed interaction interface, i.e., augmented reality (AR) and haptic devices. The stability of the closed-loop system is rigorously proved with Lyapunov methods. Experimental results in various scenarios are presented to illustrate the performance of the proposed method.

   @article{yan2024complementary,
     author = {Yan, Xiangjie and Jiang, Yongpeng and Chen, Chen and Gong, Leiliang and Ge, Ming and Zhang, Tao and Li, Xiang},
     doi = {10.1109/TCST.2023.3301675},
     issn = {1558-0865},
     journal = {IEEE Transactions on Control Systems Technology},
     month = jan,
     number = {1},
     pages = {112--127},
     title = {A Complementary Framework for Human--Robot Collaboration with a Mixed AR--Haptic Interface},
     volume = {32},
     year = {2024}
   }

2022

1. 

   Adaptive Vision-Based Control of Redundant Robots with Null-Space Interaction for Human-Robot Collaboration

   Xiangjie Yan, Chen Chen, and Xiang Li

   In 2022 International Conference on Robotics and Automation (ICRA), May 2022

   Abs Bib Code

   Human-robot collaboration aims to extend human ability through cooperation with robots. This technology is currently helping people with physical disabilities, has transformed the manufacturing process of companies, improved surgical performance, and will likely revolutionize the daily lives of everyone in the future. Being able to enhance the performance of both sides, such that human-robot collaboration outperforms a single robot/human, remains an open issue. For safer and more effective collaboration, a new control scheme has been proposed for redundant robots in this paper, consisting of an adaptive vision-based control term in task space and an interactive control term in null space. Such a formulation allows the robot to autonomously carry out tasks in an unknown environment without prior calibration while also interacting with humans to deal with unforeseen changes (e.g., potential collision, temporary needs) under the redundant configuration. The decoupling between task space and null space helps to explore the collaboration safely and effectively without affecting the main task of the robot end-effector. The stability of the closed-loop system has been rigorously proved with Lyapunov methods, and both the convergence of the position error in task space and that of the damping model in null space are guaranteed. The experimental results of a robot manipulator guided with the technology of augmented reality (AR) are presented to illustrate the performance of the control scheme.

   @inproceedings{yan2022adaptive,
     address = {Philadelphia, PA, USA},
     author = {Yan, Xiangjie and Chen, Chen and Li, Xiang},
     booktitle = {2022 International Conference on Robotics and Automation (ICRA)},
     doi = {10.1109/ICRA46639.2022.9812218},
     month = may,
     pages = {2803--2809},
     publisher = {IEEE},
     title = {Adaptive Vision-Based Control of Redundant Robots with Null-Space Interaction for Human-Robot Collaboration},
     year = {2022}
   }