First, a scene is mapped in 3D using a method known a SLAM (Simultaneous Localisation And Mapping). This works by tracking a camera in a scene by tracking important features in the images. The map is created by using a 3D system, which could be a stereo camera or LIDAR, however stereo camera systems are particularly suited to this task as they both generate 3D and provide an image that can be used for tracking.
As the stereo system moves around in the scene, the 3D data is added to a large map. This method works in real-time and can be very useful for robot systems to monitor their surroundings for path planning and collision avoidance. We use our proprietary stereo matching algorithm (I3DRSGM) to provide clear and accurate 3D for the mapping process.
Figure 5: A example Raman spectrum of paracetamol
From this spectrum, Callisto can identify the sample using machine learning which is tagged in a 3D map. Quick identification of different types of samples is easy as each sample is colour coded in the map.
Figure 6: Process for viewing spectral data in 3D map
The system has been demonstrated for the Nuclear Decommissioning sector where the identification and characterisation of radioactive material and other hazardous substances are an essential requirement. Hazardous materials (acids, alkalis, organic solvents) may be derived from multiple sources and may be present in multiple locations. They may be stored in containers, tanks or silos, which have been in operation for several decades and the composition of the waste and the condition of the storage vessels is often unclear or unknown.
Currently, hazardous substances are identified by the visual inspection of an experienced operator, who must then collect samples to be analysed. This is an expensive and potentially hazardous process. Therefore, in-situ identification of unknown samples which may be hazardous or indicative of corrosion without the need for direct human interaction is a clear advantage.
Callisto was used to identify several chemicals of specific interest to the nuclear industry. This included a Magnox slurry simulant material (Versamag), uranyl nitrate, kerosene, tri-butyl phosphate (TBP). Mixed samples of TBP and kerosene were also examined to demonstrate the detection of multiple materials simultaneously. The system mapped the target area and then targeted the Raman spectrometer at the suspicious samples.
We know Callisto is an exciting development, combining advanced 3D vision systems with the latest Raman technology and artificial intelligence. It can be adapted for a range of environments and functions with additional capability added. This might include additional sensors or varying the robotic or visualisation platform.
There are clear applications in a variety of sectors including:
• Nuclear Decommissioning
• Additive Manufacturing and Welding
• Foundation/Process Industries
• Social Care
While this system focuses on bringing together spectroscopy and 3D mapping, the system is built in a modular design so other data sources could be added, such as thermal imaging or radiation sensors.