„Nowadays, a lot of attention is paid to machine learning, and in particular deep learning-based software products. Not only in the field of intelligent video surveillance, but in almost every aspect of modern software, consumers are demanding some kind of artificial intelligence. Such algorithms are very powerful approaches to detecting patterns in all kinds of video images. For example, an object classification neural network can recognise the content of an image and communicate to the user: "There's a cat in this image." Such an algorithm works in the desired environment with an accuracy of over 90%. What exactly is a desired environment is defined by a data engineer. Various conflicts can arise in the process.

The challenges for real-world applications start with unexpected objects or environments that are very difficult to model. To train neural networks, you need vast amounts of images. These are publicly available and easy to obtain, but they usually do not correspond to realistic data. For example, if you try to build a model for recognising people in a typical video surveillance application using publicly available images, this model may only know people standing upright in the typical field of view of a mobile phone camera.

Illustration 1: This crawling person is not recognized by common networks on cameras
[Bildquelle: Securiton Deutschland]

Such a model will have difficulty recognising people in camera images. when the camera is mounted on a six-metre high camera pole. This lack of data causes much greater problems if the data engineer cannot predict or describe what the object will look like. If there are no images available to train an object detection model that include people crawling, for example, the trained detector will most likely not detect a person crawling on the ground.

Illustration 1 shows this example behaviour. Here, a crawling person is not detected by a camera analysis based on Machine Learning.

So is there an opportunity to benefit from Machine Learning or Deep Learning in video surveillance applications? In my opinion, yes! But the approaches and algorithms have to be very intelligent and the use very resourceful. On the one hand, the applications could be very limited. For example, there is a possibility that an analysis will only work if only upright people are expected in the application. In this case, a data engineer can predict what objects will look like.

On the other hand, we could use machine learning approaches to detect anomalies - rather than pre-trained objects. For example, in Figure 2, someone is trying to hide behind a mirror and enter unnoticed. This represents unexpected behaviour that cannot be modelled by a data engineer. However, an algorithm developed and trained to detect anomalies is able to detect such an attempted intrusion. When an abnormal event occurs, the operator is notified by the system at that moment.

Such algorithms are not yet operational, but will be in the future. They can be trained to recognise what is both normal and abnormal - that is, new after the training phase.

Illustration 2: Person hiding behind a mirror
[Bildquelle: Siemens AG]

Currently, data scientists are working on algorithms to solve the problem of unexpectedly detecting anomalies in a video stream. Using such algorithms in high-quality video analytics will be the next step in terms of high accuracy and high sensitivity at the same time.“