To select which car drivers to nudge for the light nudge, the cars must first be seen. A thermal camera can be used to get images of the actual site, but somehow the cars must be identified from the rest of the image. One cannot detect everything moving between two image stills, as it could be anything from grain flicker to a leaf blowing past. But computers have gotten a lot smarter in the last few years.

Moritz Berghaus from ISAC, Aachen, describes that the software has been taught to ‘think’, similar to the way an automated vehicle has. It all comes from intricate machine learning, in which the software is fed huge amounts of data and then learn by mass exposure and trial-and-error; as a human would.

After this, the complex algorithms will make it possible to find the contours of vehicles and not only detect what they are but also predict where they will be in a few seconds time. The approach works very well and only a few flaws exist. It is for example still difficult for the computer to differentiate between one large vehicle or two small ones close to each other at long distances.

The software does however not only need to identify which pixels are cars, but also measure their speed. To do this, the position and angle of the camera has to be known. A 3d-model of the actual road is then superimposed over the image from the camera and tilted so that it matches reality.

As the distances between various objects along the road are known in the 3d-model, they will also be for the real world images placed below. It is therefore possible to know exactly how far a vehicle has moved between two image stills, and thereby calculate the speed.

As several images are taken every second, it is possible to get a more or less continuous speed plot. So it is not only possible to see the cars and identify them digitally but also to note their speed. But then their speed has to be assessed to see whether the light nudge should be activated or not. And that is a totally different story.