Robots have long been integrated with various sensors and systems and deployed in conflict areas, for example, sweeping for hidden explosive devices, monitoring air quality and explosives investigations, and removing human (and canine!) operators from the equation in the process. Now, as part of a project to develop a complete telexistence system, integrating telepresence, robotic and haptic technologies onto a single platform, a team from the Advanced Medical Research Centre (AMRC) at the University of Sheffield is exploring stereo vision camera systems to capture medical data in battlefield scenarios.
Funded by the Defence Science and Technology Laboratory and Nuclear Decommissioning Authority through the Defence and Security Accelerator, the project is being led by Professor Sanja Dogramadzi from the Department of Automatic Control and Systems Engineering and David King, Head of Digital Design at the AMRC.
i3D robotics is providing Phobos, our high-resolution stereo vision camera, along with stereo-matching software for data capture and interpretation.
The telexistance system will be deployed into hazardous combat situations and used to locate and reach casualties. Medics trained in the use of a virtual reality headset will then be able to navigate to the casualty, see them, and control the platform to perform a preliminary medical assessment to triage them. To aid this, the system will have the ability to send images and videos via a 5G connection, as well as report back on the casualty’s body temperature and blood pressure, and take samples such as mouth swabs for saliva and blood samples from an arm vein. Each robot is 1 metre wide and just over 1 metre long and has two robot arms, each with a 1-metre reach. Powered by an electric motor, the system is mounted on caterpillar tracks to facilitate movement over disrupted terrain.
“It’s an incredibly versatile and complex system incorporating a range of equipment including a 360o camera plus Phobos, our high-resolution stereo vision camera attached to two robotic arms. One of these arms will also have a custom-built gripper for handling smaller objects with dexterity. Haptic sensors provide feedback on grip, grasp, and handling, whilst our stereo vision cameras provide the data for the VR headsets.” – Dr Richard French, Senior Systems Engineer, i3D robotics
Each complete system costs around £80,000 to produce.
Currently, battlefield medics, similar to paramedics, are sent onto the battlefield to assess and tend to the wounded, putting the medics at significant risk of injury, which could eliminate their specialised and vital knowledge from the arena.
The applications for this type of system extend beyond the reach of traditional battlefields. They could be used in environments where contagious disease is indicated, as well as in environmentally hazardous conditions such as radioactive and chemical contamination. They could also be used in other situations where first responders are required, such as in the aftermath of terror attacks and natural disasters like tsunamis, earthquakes, or volcanic eruptions.
“Phobos was initially developed from an ongoing project we’re involved with at Sellafield for inspecting contaminated internal vaults and cells. Mounted onto a robot and deployed into a radiative environment, the camera system collects data and maps the room into a point cloud. We’ve integrated the system with a sharps database to allow it to locate and identify potentially hazardous objects and place them in a 3D context. Phobos has also been used in the Stereo Theatre project: a collaboration with AMRC’s Design & Prototyping Group (DPG) to develop a proof-of-concept system capable of producing a high-resolution, real-time 3D map of a patient in an operating theatre environment. The project was to enhance the digital operating theatre platform developed by the AMRC DPG.” – Nicholas Bantin, Director, i3D
An ultimate aim of the development of this system is that it will also eventually be used to relieve the pressure on NHS staff with the system triaging patients in hospital scenarios.
