BestSens insights: Data evaluation and data visualisation

Three quick questions to our data expert Markus

» How do we collect data?

That depends on what the data is used for. Basically, there are no restrictions for us regarding the data sink. For example, the data can be stored as local logs on an external storage medium or in a database. The latter offers various possibilities, also with regard to the protocols to be used and on-premise or on-demand requirements – this can be individualised for each customer.

For evaluations to improve existing algorithms or develop new ones, there are various longevities of the data. For this, additional information is often required, for example, from a trial plan and trial protocol.

There are a lot of adjusting screws and variables. When you get data from the customer, it is important that you define and know the variables exactly, so that you can finally check the expectations with the results.

» And how does the AI then come into play?

Simplified, we give the AI the data and it evaluates it. The training is done with labelled data. Where these come from is irrelevant to us and the AI. However, there are often cases where it is not possible to obtain labelled data. Be it because of complex test set-ups and environments or because there is too little existing data for it to be of statistical significance. In such cases, it is still possible for us to obtain various information from the data, including anomaly detection.

If, as in the case of anomaly detection, the data no longer falls within a predefined framework, we can recognise other correlations with the help of AI and, if necessary, determine reasons for the deviations.

» What is more important: data evaluation or data processing?

They both go hand-in-hand. Without meaningful pre-processing of the data, even more complex deep learning methods can fail. However, this does not mean that an AI cannot also assist in pre-processing. Ultimately, preprocessing is always a healthy mix of automation and manual processing through specific domain knowledge.

In principle, however, we tend to use the data as unchanged as possible. Our specially developed hybrid data evaluation models are capable of recognising even complex structures and correlations in the data and guarantee robust recognition even in the case of dynamic changes.