Sanne van Waveren 🟥

selected publications

1. 

   Mixed-Initiative Human-Robot Teaming under Suboptimality with Online Bayesian Adaptation

   Natarajan, Manisha, Xue, Chunyue,  van Waveren, Sanne, Feigh, Karen, and Gombolay, Matthew

   In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems 2024

   Abs Bib Paper

   For effective human-agent teaming, robots and other artificial intelligence (AI) agents must infer their human partner’s abilities and behavioral response patterns and adapt accordingly. Most prior works make the unrealistic assumption that one or more teammates can act near-optimally. In real-world collaboration, humans and autonomous agents can be suboptimal, especially when each only has partial domain knowledge. In this work, we develop computational modeling and optimization techniques for enhancing the performance of human-agent teams, where both the human and the robotic agent have asymmetric capabilities and act suboptimally due to incomplete environmental knowledge. We adopt an online Bayesian approach that enables a robot to infer people’s willingness to comply with its assistance in a sequential decision-making game. Our user studies show that user preferences and team performance vary with robot intervention styles, and our approach for mixed-initiative collaboration enhances objective team performance (p<.001) and subjective measures, such as user’s trust (p<.001) and perceived likeability of the robot (p<.001).

   @inproceedings{natarajan2024mixed,
     title = {Mixed-Initiative Human-Robot Teaming under Suboptimality with Online Bayesian Adaptation},
     author = {Natarajan, Manisha and Xue, Chunyue and {van Waveren}, Sanne and Feigh, Karen and Gombolay, Matthew},
     selected = {true},
     booktitle = {Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems},
     pages = {1454--1462},
     year = {2024},
     html = {https://dl.acm.org/doi/abs/10.5555/3635637.3663005}
   }

2. 

   Enhancing Safety in Learning from Demonstration Algorithms via Control Barrier Function Shielding

   Yang, Yue*, Chen, Letian*, Zaidi, Zulfiqar*,  van Waveren, Sanne, Krishna, Arjun, and Gombolay, Matthew

   In Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction 2024

   Abs Bib Paper

   Learning from Demonstration (LfD) is a powerful method for non-roboticists end-users to teach robots new tasks, enabling them to customize the robot behavior. However, modern LfD techniques do not explicitly synthesize safe robot behavior, which limits the deployability of these approaches in the real world. To enforce safety in LfD without relying on experts, we propose a new framework, SElding with Control barrier fUnctions in inverse REinforcement learning (SECURE), which learns a customized Control Barrier Function (CBF) from end-users that prevents robots from taking unsafe actions while imposing little interference with the task completion. We evaluate SECURE in three sets of experiments. First, we empirically validate SECURE learns a high-quality CBF from demonstrations and outperforms conventional LfD methods on simulated robotic and autonomous driving tasks with improvements on safety by up to 100%. Second, we demonstrate that roboticists can leverage SECURE to outperform conventional LfD approaches on a real-world knife-cutting, meal-preparation task by 12.5% in task completion while driving the number of safety violations to zero. Finally, we demonstrate in a user study that non-roboticists can use SECURE to effectively teach the robot safe policies that avoid collisions with the person and prevent coffee from spilling.

   @inproceedings{yang2024enhancing,
     author = {Yang, Yue* and Chen, Letian* and Zaidi, Zulfiqar* and {van Waveren}, Sanne and Krishna, Arjun and Gombolay, Matthew},
     title = {Enhancing Safety in Learning from Demonstration Algorithms via Control Barrier Function Shielding},
     selected = {true},
     year = {2024},
     isbn = {9798400703225},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3610977.3635002},
     doi = {10.1145/3610977.3635002},
     html = {https://dl.acm.org/doi/10.1145/3610977.3635002},
     booktitle = {Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction},
     pages = {820–829},
     numpages = {10},
     keywords = {control barrier function, learning from demonstration, safety},
     location = {Boulder, CO, USA, },
     series = {HRI '24}
   }

3. 

   Increasing perceived safety in motion planning for human-drone interaction

   Van Waveren, Sanne, Rudling, Rasmus, Leite, Iolanda, Jensfelt, Patric, and Pek, Christian

   In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction 2023

   Abs Bib Paper

   Safety is crucial for autonomous drones to operate close to humans. Besides avoiding unwanted or harmful contact, people should also perceive the drone as safe. Existing safe motion planning approaches for autonomous robots, such as drones, have primarily focused on ensuring physical safety, e.g., by imposing constraints on motion planners. However, studies indicate that ensuring physical safety does not necessarily lead to perceived safety. Prior work in Human-Drone Interaction (HDI) shows that factors such as the drone’s speed and distance to the human are important for perceived safety. Building on these works, we propose a parameterized control barrier function (CBF) that constrains the drone’s maximum deceleration and minimum distance to the human and update its parameters on people’s ratings of perceived safety. We describe an implementation and evaluation of our approach. Results of a within-subject user study (N=15) show that we can improve perceived safety of a drone by adjusting to people individually.

   @inproceedings{vanWaveren2023increasing,
     title = {Increasing perceived safety in motion planning for human-drone interaction},
     author = {Van Waveren, Sanne and Rudling, Rasmus and Leite, Iolanda and Jensfelt, Patric and Pek, Christian},
     booktitle = {Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction},
     pages = {446--455},
     year = {2023},
     html = {https://dl.acm.org/doi/abs/10.1145/3568162.3576966},
     selected = {true}
   }

4. 

   Correct Me If I’m Wrong: Using Non-Experts to Repair Reinforcement Learning Policies

   Van Waveren, Sanne, Pek, Christian, Tumova, Jana, and Leite, Iolanda

   In Proceedings of the 17th ACM/IEEE International Conference on Human-Robot Interaction 2022

   Abs Bib Paper

   Reinforcement learning has shown great potential for learning sequential decision-making tasks. Yet, it is difficult to anticipate all possible real-world scenarios during training, causing robots to inevitably fail in the long run. Many of these failures are due to variations in the robot’s environment. Usually experts are called to correct the robot’s behavior; however, some of these failures do not necessarily require an expert to solve them. In this work, we query non-experts online for help and explore 1) if/how non-experts can provide feedback to the robot after a failure and 2) how the robot can use this feedback to avoid such failures in the future by generating shields that restrict or correct its high-level actions. We demonstrate our approach on common daily scenarios of a simulated kitchen robot. The results indicate that non-experts can indeed understand and repair robot failures. Our generated shields accelerate learning and improve data-efficiency during retraining.

   @inproceedings{vanWaveren2022correct,
     title = {Correct Me If I'm Wrong: Using Non-Experts to Repair Reinforcement Learning Policies},
     author = {Van Waveren, Sanne and Pek, Christian and Tumova, Jana and Leite, Iolanda},
     booktitle = {Proceedings of the 17th ACM/IEEE International Conference on Human-Robot Interaction},
     pages = {1--9},
     year = {2022},
     html = {https://sannevw.github.io/assets/pdf/vanWaveren_Hri2022_Final_PP.pdf},
     selected = {true}
   }

5. 

   Encoding Human Driving Styles in Motion Planning for Autonomous Vehicles

   Karlsson, Jesper*,  van Waveren, Sanne*, Pek, Christian, Torre, Ilaria, Leite, Iolanda, and Tumova, Jana

   In ICRA International Conference on Robotics and Automation 2021

   Abs Bib Paper Video

   Driving styles play a major role in the acceptance and use of autonomous vehicles. Yet, existing motion planning techniques can often only incorporate simple driving styles that are modeled by the developers of the planner and not tailored to the passenger. We present a new approach to encode human driving styles through the use of signal temporal logic and its robustness metrics. Specifically, we use a penalty structure that can be used in many motion planning frameworks, and calibrate its parameters to model different automated driving styles. We combine this penalty structure with a set of signal temporal logic formula, based on the Responsibility-Sensitive Safety model, to generate trajectories that we expected to correlate with three different driving styles: aggressive, neutral, and defensive. An online study showed that people perceived different parameterizations of the motion planner as unique driving styles, and that most people tend to prefer a more defensive automated driving style, which correlated to their self-reported driving style.

   @inproceedings{karlsson2021encoding,
     title = {Encoding Human Driving Styles in Motion Planning for Autonomous Vehicles},
     author = {Karlsson, Jesper* and {van Waveren}, Sanne* and Pek, Christian and Torre, Ilaria and Leite, Iolanda and Tumova, Jana},
     booktitle = {ICRA International Conference on Robotics and Automation},
     year = {2021},
     selected = {true},
     html = {https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9561777},
     video = {https://youtu.be/z5IXIdPKX_o}
   }

6. 

   Behavioural responses to robot conversational failures

   Kontogiorgos, Dimosthenis, Pereira, Andre, Sahindal, Boran,  van Waveren, Sanne, and Gustafson, Joakim

   In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction 2020

   Abs Bib Paper

   Humans and robots will increasingly collaborate in domestic environments which will cause users to encounter more failures in interactions. Robots should be able to infer conversational failures by detecting human users’ behavioural and social signals. In this paper, we study and analyse these behavioural cues in response to robot conversational failures. Using a guided task corpus, where robot embodiment and time pressure are manipulated, we ask human annotators to estimate whether user affective states differ during various types of robot failures. We also train a random forest classifier to detect whether a robot failure has occurred and compare results to human annotator benchmarks. Our findings show that human-like robots augment users’ reactions to failures, as shown in users’ visual attention, in comparison to non-human-like smart-speaker embodiments. The results further suggest that speech behaviours are utilised more in responses to failures when non-human-like designs are present. This is particularly important to robot failure detection mechanisms that may need to consider the robot’s physical design in its failure detection model.

   @inproceedings{kontogiorgos2020behavioural,
     title = {Behavioural responses to robot conversational failures},
     author = {Kontogiorgos, Dimosthenis and Pereira, Andre and Sahindal, Boran and {van Waveren}, Sanne and Gustafson, Joakim},
     booktitle = {Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction},
     pages = {53--62},
     year = {2020},
     selected = {true},
     html = {https://dl.acm.org/doi/pdf/10.1145/3319502.337478}
   }

7. 

   Gesticulator: A framework for semantically-aware speech-driven gesture generation [Best Paper Award]

   Kucherenko, Taras, Jonell, Patrik,  van Waveren, Sanne, Henter, Gustav Eje, Alexandersson, Simon, Leite, Iolanda, and Kjellström, Hedvig

   In Proceedings of the 2020 International Conference on Multimodal Interaction 2020

   Abs Bib Paper Code

   During speech, people spontaneously gesticulate, which plays a key role in conveying information. Similarly, realistic co-speech gestures are crucial to enable natural and smooth interactions with social agents. Current data-driven co-speech gesture generation systems use a single modality for representing speech: either audio or text. These systems are therefore confined to producing either acoustically-linked beat gestures or semantically-linked gesticulation (e.g., raising a hand when saying “high”): they cannot appropriately learn to generate both gesture types. We present a model designed to produce arbitrary beat and semantic gestures together. Our deep-learning based model takes both acoustic and semantic representations of speech as input, and generates gestures as a sequence of joint angle rotations as output. The resulting gestures can be applied to both virtual agents and humanoid robots. Subjective and objective evaluations confirm the success of our approach.

   @inproceedings{kucherenko2020gesticulator,
     title = {Gesticulator: A framework for semantically-aware speech-driven gesture generation},
     author = {Kucherenko, Taras and Jonell, Patrik and {van Waveren}, Sanne and Henter, Gustav Eje and Alexandersson, Simon and Leite, Iolanda and Kjellstr{\"o}m, Hedvig},
     booktitle = {Proceedings of the 2020 International Conference on Multimodal Interaction},
     pages = {242--250},
     year = {2020},
     selected = {true},
     html = {https://svito-zar.github.io/papers/Gesticulator_ICMI_2020.pdf},
     code = {https://github.com/Svito-zar/gesticulator},
     award = {Best paper award}
   }