POSTED:  21/12/2020

Refining Code - a Collaboration with STFC Hartree Centre

I3DR is working with the Science and Technology Facilities Council (STFC) Hartree Centre on a project to improve data processing speeds for stereo camera technology for application in the field of robotic guidance. 


STFC is one of nine of UK Research and Innovation bodies. They work in partnership with universities, research organisations and businesses, enabling collaborations which develop our understanding across the whole scientific spectrum. Through their £8 million Bridging for Innovators (B4I) programme, STFC provides funding for companies to fast-track solutions to industrial challenges and boost productivity by providing access to their extensive array of technologies and expertise.


The STFC Hartree Centre provides businesses with access to high performance computing (HPC) and complementary software skills to help them solve complex software development challenges. The centre houses Scafell Pike, the first Bull Sequana X1000 in the UK, capable of 4.3 quadrillion calculations per second, alongside leading experts in fields including high performance software engineering and data science.


I3DR’s principal focus is on the development of advanced stereo vision and machine vision technologies. There are many different algorithms being developed to enable the interpretation of data captured by stereo vision cameras. Current semi-global matching (SGM) algorithms are effective over large areas, they fail to provide sufficient resolution when concentrated over a smaller region of interest. Other algorithms provide high resolution images capturing greater detail over a given, narrower region of interest but the quantity of data associated with large area capture causes significant latency in the processing speed. Greater levels of detail are required for sensitive and delicate applications, such as robotic guidance where real-time display of the captured data is essential, but currently, processing speeds of high-resolution data result in latency between interpretation and display. This is especially problematic when incorporating this with other technologies such as robotics, augmented and virtual reality. 


Last year, in a major step forward, Dr Joshua Veitch, PhD student co-funded by STFC CASE and JPM-Mullard, refined a stereo matching algorithm originally designed for high resolution satellite data. The refined code was able to reduce data processing times from a matter of hours to just a few minutes. This new code allows for dense reconstruction of a scene where SGM algorithms would otherwise struggle. Through the B4I programme, I3DR’s Benjamin Knight, Software & Machine Learning Engineer is working with the STFC Hartree Centre to continue to refine that code with the aim of reducing processing speeds to under a frame per second. This will provide direct feedback into an AR or VR applications such as is required for I3DR’s Innovate UK-funded collaboration with AMRC Medical on the Stereo Theatre project.


The Hartree Centre has assisted i3DR and is refining this code, whilst I3DR has built tools to test the algorithm using the popular standard Middlebury stereo dataset. This contains ground truth data against which I3DR can use to benchmark their results and be certain they are always achieving high accuracy as each optimisation step is applied.


“We’ve been taking full advantage of the extensive experience and knowledge of code optimisation that STFC’s personnel has to offer. The progress made to date is incredibly encouraging and we’re hopeful we’re going to achieve near real-time speeds very soon.


The code is being continuously refined with current progress bringing speeds of 10-15 seconds for a 2MP stereo image pair without any loss in quality. We also have a clear path to improve this further in the next three months by moving processing from the CPU to the GPU. This development should give us speeds of less than a second. GPU implementation will allow for making the most of powerful CUDA cores designed for this kind of data processing. This should mean we can unlock the full potential of the high-resolution cameras in our stereo vision systems.” – Benjamin Knight, Software & Machine Learning Engineer, I3DR.


Dr Richard French, Senior Systems Scientist, I3DR, stated “The beauty of this code is how wide the applications will be for it. It can be retroactively applied to historic image data captured to deliver greater resolution, refining existing data sets without having to get new data. It will also represent another step in the journey moving us towards the ability to create a true digital twin of any given environment.” 


The optimised code is due for trialling in on-going projects with Sellafield, the National Nuclear Laboratory and AMRC Medical in early 2021. 


i3D-robotics is a micro-SME based in Tonbridge, Kent, set up in 2013 to develop intelligent vision systems to help solve real-world industrial challenges particularly within Nuclear Decommissioning, as well as Steel, Glass and Ceramics manufacturing, and Construction.