The main focus of the ROS2AR FTP is the integration of AR facilities into ROS-I. This is done by means of object recognition techniques for robots using smart glasses. The user is assisted in the object detection using landmark and 2D object detection in the HRI context. The related information (e.g. digital twin) about the detected object is gathered and shared to the user using a ROS bridge. Later on, this is visualized in an augmented manner by means of notifications using Android ecosystem with direct application for head mounted devices (such as smart glasses).

This FTP is about ROS robot state visualization for AR smart glasses. ROS-I is integrated with the wide variety of Android based AR emerging devices, with a special focus on head mounted devices.

Through this integration, the user is able to receive real-time notifications about the environment. This information can include the state of the equipment nearby or other object specific data about the robot. This enables the user to operate hands free, while at the same time receiving context relevant information on demand.

This FTP opens the ROS-I community to interface with the popular Android which covers more than 75% of the market. The focus is on smart glasses like visualization devices.

We are addressing the Digital Twin use case in particular. For this the user obtains live context-relevant technical data about an equipment as an augmented representation of its state. Examples include the state of the robots, scheduling information or monitoring info about abnormal or exceptional events.

The code of this FTP is generic enough to enable the community to reuse it.

Please watch also our first demo video: https://vimeo.com/351293877

Have a look at our demo without QR tags!