Intelligent Monitoring using AI Technologies in Hot or extreme environments

iMATH, Intelligent Monitoring using AI Technologies in Hot or extreme environments, taps into the concept of ‘Manufacturing Made Smarter’, using industrial digital technologies to transform the productivity and agility of extreme environment manufacturing. It targeted development and validation of the use of 3D stereo vision and machine intelligence in extreme environments, consistent with work I3DR is doing in nuclear decommissioning. This case brought together I3DR, ABF, a leading Austrian supplier of large data management platforms, VoestAlpine, a global steel manufacturer under the project lead, Joanneum Institute, a leading Austrian research institute in machine vision.

This team represented a wide range of expertise in extreme industries such as nuclear, steel manufacturing, and welding, and I3DR’s expertise in 3D imagine, object recognition, and interfacing with robotic systems. The aim of the project was to produce 3D models of a scene and use intelligent algorithms to identify hazards such as defects. The system would be used as a stand-alone product by operators, or in tele-operations, or be able to be interfaced RAI systems through providing 3D information such as distance and size of objects. Further, it wold produce a 3D stereo system for use in extreme environments in real-time.

Historically, the primary quality assurance process used in most steel mills is by using human inspection after the product as cooled. The cooling process requires time, plus additional time for the inspection process. For high-grade steel, this process can be considerably longer. Discrepancies between customers and mills can arise due to the varied experience level of the engineers, as well as the potential for human error. Human error may also result in plate being rejected that meets standards. The process is lengthy, fallible and potentially wasteful.

The majority of commercial vision-based inspection instruments use side/dark field illumination to produce high-contract images but whilst these are partially successful at detecting defects on the surface, they are prone to false positives is scale is present. Structured light/line scanning techniques to monitor the surface of the steel products through triangulation measurement to monitoring of deviations in the structured pattern have only targeted at smaller straight rolled products less that 1m wide. They do not have the sufficient combination of coverage/resolution for large plates.

For wire rod inspection, no systems were available which could be operated successfully at the required speed and resolution.There were no systems capable of 2D/3D measurements in steel/other industries requiring inspection using machine/deep learning and artificial intelligence, and hence require significant user input to make the robust. This results in a system suitable for a single industry and new defect will be difficult to detect. These difficulties in measurement resulted in manual inspection remaining the most common technique for quality assurance of steel products.

There were a number of challenges including data acquisition in harsh environments and a large amount of processed and stored data. Once scanned, the overall covered surface area is large compared with the small and rarely appearing defects. This made obtaining enough representative names to train iMATH a real challenge on the path towards an inline inspection system.

1. Data Acquisition – For both steel use cases, the minimum size of defects to be detected was as small as 0.2 mm x 0.5 mm which defines the required scan resolution. For plate inspection, a major success factor is to protect scanning equipment from the heat radiated from the huge, 800 degree C surfaces. For wire inspection, the extreme speed of up to 110 m/s requires special high-speed line scan cameras opening at line frequencies of 210 KHz. These conditions required an extremely short exposure time (<1/250 000 s) so extremely strong and focussed lighting was required which could be realised with water-cooled focused LED units.
2. Data handling and data storage – each of the 3 camera systems of the wire inspection system produced 100 MByte/s Data needed to be transferred, lossless compressed and stored in a structured form to allow repeatable and easy access to combined representations of the sample images. The amount of acceptable compression was strongly dependent on surface quality mainly in the non-defect areas. Uncompressed data stream produces around 18 TByte of image data every day for 3 cameras. Data storage was also a problem for monitoring in other steel applications such as plate due to the large number of high-resolution images required. Lossless compression could not reduce the amount of data to an acceptable level. Therefore, the acceptable compression rate needed to be estimated to store training data in a way they were equivalent with direct data stream for inline classification.
3. Training – In normal production, more than 99% of the surface of the workpieces is free from defects, although there may be surface variations which are within the normal appearance and therefore consist of no defects. During iMATH testing, humans would not be capable of labelling enough tiny dents in high-resolution images on several square meters of surfaces or on up to 18km of rolled wires. Therefore, already labelled data sets were used as an initial starting point to initiate an iterative process to generate more data for the training of iMATH. Similarity measures were used in this phase of the training process to generate a broader data base for successful real-world classification. For the wire use case, only textural information was available, 3D defects could only b recognised from a variation of reflected light towards the camera. For the plate application, direct 3D surface reconstruction from stereo images would be used for defect segmentation and characterisation.
4. iMATH used both 2D and 3D data and fed this information to the AI algorithms for decision-making. Therefore, the data from 2D and 3D had to correlate with each other.
5. Implementation Phase – the complexity of the algorithms to deal with the data sets representing highly complex workpieces needed high-performance computers equipped with multiple Graphics Processing Units (GPUs) to reach the goal of immediate inline processing. One option was to perform a pre-segmentation using relatively simple traditional algorithms and only perform a fine classification on small areas, representing better the fact defects normally are found in very small areas.

We had some challenges with this project unforeseen prior to its start, primarily COVID-19 and subsequent lockdowns. and then the war in the Ukraine. Steel production reduced significantly during the pan