Collaborative Human-Robot Exploration via Implicit Coordination

SSRR · 2022

When a human and a robot collaboratively explore a new environment, how can the human implicitly signal the robot to explore a disjoint region of space?

Yves Georgy Daoud Kshitij Goel Nathan Michael Wennie Tabib

This paper develops a methodology for collaborative human-robot exploration that leverages implicit coordination. Most autonomous single- and multi-robot exploration systems require a remote operator to provide explicit guidance to the robotic team. Few works consider how to embed the human partner alongside robots to provide guidance in the field. A remaining challenge for collaborative human-robot exploration is efficient communication of goals from the human to the robot. In this paper we develop a methodology that implicitly communicates a region of interest from a helmet-mounted depth camera on the human's head to the robot and an information gain-based exploration objective that biases motion planning within the viewpoint provided by the human. The result is an aerial system that safely accesses regions of interest that may not be immediately viewable or reachable by the human. The approach is evaluated in simulation and with hardware experiments in a motion capture arena. Videos of the simulation and hardware experiments are available at: https://youtu.be/7jgkBpVFIoE.

Figures

**Human-Robot Cave Exploration** (a) A human-robot team is tasked with exploring a cave. (b) The human implicitly conveys a region of interest to the robot by transmitting their current viewpoint. (c) The robot plans a path to areas of the environment that are occluded to the human.

**Region of Interest from Human FoV** (a) The human's field of view (FoV) is shown in red and used to determine which (b) cells in the global occupancy map are within the ROI (shown in green).

**Information Gain Comparison** Comparison of the information gain objectives using a 2D numerical example. For the environment in (a) and human at (50, 0), the map updated after one zh (Section IIIA) is shown in (b). The CSQMI objective from [18] shown in (c) does not account for zh, while the ROI-CSQMI objective in (d) places higher weights in the occluded region.

**OAVI Objective Heatmaps** Heatmaps for the OAVI objective and its constituent terms (Section III-C) over the 2D map shown in Fig. 3b. Compared to ROI-CSQMI in Fig. 3d, the OAVI objective function in Fig. 4d exhibits a gradient biasing the exploration to focus on the occluded region closer to the human's FoV first.

**Simulation Results** (a)–(d) simulation environments, (e)–(h) ROI entropy plotted as a function of time and (i)–(l) map entropy plotted as a function of time for the CSQMI, ROI-CSQMI, and OAVI exploration variants. 30 trials are run for each exploration variant and simulation environment. Note that ROI-CSQMI and OAVI explore the human's FoV 3× faster than CSQMI while CSQMI reduces the total map uncertainty faster. OAVI reduces the map uncertainty 56% more than ROI-CSQMI.

**Trajectory Snapshots** Top-down snapshots of the trajectory taken by the robot for the three approaches in the two walls environment with the human's FoV drawn in gray dashed lines. CSQMI proceeds to explore the unknown regions outside of the human's FoV, while the ROI-constrained CSQMI and OAVI prioritize the ROI first. As opposed to ROI-CSQMI, the gradient in the OAVI approach (see Fig. 4d) pushes the robot to explore the occluded region closest to the human first.

**Hardware Platforms** (Left) Aerial robot and (Right) helmet for the human partner used in the hardware experiments.

**Motion Capture Experiment** A human-robot team explores an environment inside a motion capture arena, with an obstacle in front the human requiring the robot to provide complementary views.

**Entropy Reduction Results** ROI and map entropy as a function of time for the three approaches. The baseline CSQMI approach minimizes the total map entropy, while its extension ROI-CSQMI prioritizes the ROI. OAVI successfully reduces the uncertainty in the ROI first, followed by an exploratory behavior. A video of the experimental setup and the three exploration approaches can be found at https://youtu.be/7jgkBpVFIoE.

**Reconstructed Map** Reconstructed point cloud map of the 50 m3 environment from the OAVI hardware trial.

BibTeX

@inproceedings{collaborative-human-robot-exploration-2022,
  title={Collaborative Human-Robot Exploration via Implicit Coordination},
  author={Yves Georgy Daoud, Kshitij Goel, Nathan Michael, and Wennie Tabib},
  booktitle={IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 2022},
  year={2022}
}