Deep Learning despite sparse data

Search millions of images within fractions of a second for a keyword such as "beach" or use speech recognition to have a song played for you on your streaming service. Deep learning - a branch of artificial intelligence - makes such things possible. With the help of huge amounts of data, algorithms learn; they classify, sort and filter data. The technology is used in many areas, such as medicine, agriculture and robotics.

However, this has not yet been the case for processes in the chemical industry. "There is much less data here than, for example, in an image search on the web, and in some cases there is no data at all or companies do not make it available," says Professor Dr. Marius Kloft, who heads the Intelligent Systems department and the Machine Learning working group at TU Kaiserslautern and is the spokesman for the new DFG research group. "In addition, often all data also looks the same, which makes learning here much more difficult." For example, in a chemical plant, the same process is always run under the same conditions over a long period of time, such as the conversion of raw materials into products. Sensors then always measure the same temperature, pressure and so on.

It is precisely in such areas that the new research group nevertheless wants to apply deep learning methods. The group benefits from its interdisciplinary approach. Otherwise, the development of such new methods would not be possible. In addition to computer scientist Kloft and computer science professor Dr. Heike Leitte, the group includes TUK process engineering professor Dr. Hans Hasse, as well as junior professor Dr. Fabian Jirasek, who conducts research on machine learning in process engineering, and junior professor Dr. Sophie Fellenz, who also works on machine learning. What is special about Fellenz and Jirasek is that the Carl Zeiss Foundation is funding their junior professorships as a tandem with 1.5 million euros; their common goal is to bring together machine learning with physical modeling. The new DFG research group is now also benefiting from this. As part of a Mercator Fellowship, the project will also involve computer science professor Dr. Stephan Mandt from the University of California at Irvine, whose research includes applications of machine learning in the natural sciences.

"For us, the focus is less on the number of data and more on the quality of the data," Kloft continues. The team is taking two approaches. First, it performs chemical processes in the lab itself to collect experimental data; second, it also generates synthetic data. "Each experiment is complex and expensive. We are therefore primarily concerned with collecting the right data, i.e., the data from which the deep learning methods can learn the most," adds Jirasek. For this purpose, a so-called batch distillation plant is used on the Kaiserslautern campus to collect data from dynamic processes. Furthermore, at the project partner, the Technical University of Munich, a continuous production plant for synthetic fuels is operated on a pilot scale at the Straubing site.

"These plants are equipped with a variety of sensors to record a wide range of data, such as pressure and temperature, but also videos of the inner workings or the composition of the mixtures used," the junior professor continues.

In addition to the data from the laboratory, further data is generated with the help of physical simulations and artificial intelligence methods. A particular challenge here is to make these as realistic as possible. "We are taking two approaches to this," explains Assistant Professor Fellenz. "First, we are developing methods to learn the style of experimental data and then apply it to simulation data." Such methods already exist to change the tonality of texts. In the research group, they are now being applied to the time-series data of chemical processes, which can be understood as sequential data just like words in a text. "On the other hand, we incorporate physical laws, for example from thermodynamics, directly into our models, so that we can also generate realistic data from areas where we have no measurements," the computer scientist continues.

The goal for the next four years is to use the new deep learning methods to detect anomalies or faults in chemical plants at an early stage, but also to identify suitable countermeasures. This is of high practical relevance, since any failure of a plant is at least expensive, and in the worst case represents a danger to people and the environment.

The examples are numerous, such as in 2013, when an explosion occurred in a chemical plant in Geismar, Louisiana, USA, killing one person and injuring many. The cause was a fault in a heat exchanger. "Especially in such plants, anomalies in the process can have dire consequences," Kloft continues. Deep Learning methods could help here to automatically detect such faults in order to issue a warning message in time. "Our anomaly detectors should be much more sensitive than conventional techniques thanks to Deep Learning."

The object of the work is also to explain the information, visualize it and present it clearly. They need to be quick and easy to understand so that professionals in the chemical industry can use these techniques and respond to appropriate recommendations from the technology. The team will also work on verifying its methods. This will involve using mathematical methods to verify that the algorithms are working correctly. Kloft continues, "With our work, we are generating a completely new data set that we will make available to others."

Professor Dr. Werner R. Thiel, Vice President for Research and Technology at TUK, emphasizes the importance of the new DFG research group: "We have been able to lay the foundation for this success with our two tandem junior professorships funded by the Carl Zeiss Foundation. We are thus contributing our expertise from computer science and process engineering to the project. Without this combination, such a project would not be possible. This is an advantage of our research location, which is interdisciplinary in many areas. This is now paying off again. I warmly congratulate all those involved."

The Rhineland-Palatinate Minister of Science, Clemens Hoch, also congratulates: "I am very pleased with the success of the researchers from Kaiserslautern and their cooperation partners from Oldenburg and Straubing. The DFG-funded research group is proof of the TU's research strength in this highly topical field of research and fits perfectly into the state's AI strategy. This project again clearly demonstrates that the key technology of artificial intelligence can make positive, performance-enhancing contributions in a wide range of scientific disciplines and applications. Therefore, I am very pleased that we in Rhineland-Palatinate are fortunate to have leading AI researchers, such as Professor Kloft and his colleagues, in our institutions whose innovative projects are developing new possibilities for the use of artificial intelligence in Rhineland-Palatinate and beyond."

The long-term goal of the research group is to develop methods for the autonomous operation of plants in the chemical industry. With its work, the group also wants to advance process simulation in process engineering by developing new types of tools and integrating data types into them that are currently not even considered.

In addition to the working groups of the TU Kaiserslautern, the teams of Dr. Michael Bortz of the Fraunhofer Institute for Industrial Mathematics ITWM in Kaiserslautern, Professor Dr. Jakob Burger of the TU Munich and Professor Dr. Daniel Neider of the University of Oldenburg are involved in the project. In addition, Professor Dr. Stephan Mandt from the University of California in Irvine is involved as a Mercator Fellow.