Artificial Intelligence for Digitising Industry – Applications

Front matter

= = = = = pl E: ra =m E ra =

Artificial Intelligence for Digitising Industry Applications

RIVER PUBLISHERS SERIES IN COMMUNICATIONS

Dedication

“Action is the real measure of intelligence.”

- Napoleon Hill

“Intelligence is the ability to adapt to change.” - Stephen Hawking

Preface

Editors Biography List of Figures List of Tables

List of Abbreviations

1 AI Automotive

1.0 AI Reshaping the Automotive Industry Daniel Plorin 1.0.2 AI Developments and Future Trends for AI Technologies . .

1.1 AI for Inbound Logistics Optimisation in Automotive Industry Nikolaos Evangeliou, George Stamatis, George Bravos, Daniel Plorin and Dominik Stark 1.1.3 Data Flow Principles and Architecture of the MPDSS ... .

vil xix

Xxi

XXV xxxiii

XXXV

PU

1.2 State of Health Estimation using a Temporal Convolutional Network for an Efficient Use of Retired Electric Vehicle

Batteries within Second-Life Applications 21

Steffen Bockrath, Stefan Waldhór, Harald Ludwig, and

Vincent Lorentz

1.2.1 Retired Electric Vehicle Batteries for Second-Life Applications s i 2403.4. eta ca shes ooh id ke 22

1.2.3 Data Measurement Using the Open-Source BMS foxBMS 25

1.2.4 Temporal Convolutional Neural Network for State of Health PregictiOn: ¿40d id ta Shinde co 27

1.245: RESULTS: ra ok edhe are eg Bad Beh Pa Gobo 29

122;6;Conclusion® ai ad dopa 2S AA A eS Ad 31

1.3 Optimising Trajectories in Simulations with Deep Reinforcement

Learning for Industrial Robots in Automotive Manufacturing 35

Noah Klarmann, Mohammadhossein Malmir, Josip Josifovski,

Daniel Plorin, Matthias Wagner and Alois C. Knoll

EI Introduction. cae sea ees oc ot dy, a oh ee See 36

1:3:3 Methodology a e ca e go a we ce a aa ag a 39

1.4 Foundations of Real Time Predictive Maintenance with Root

Cause Analysis 47

Franz Wotawa, David Kaufmann, Adil Amukhtar, lulia Nica,

Florian Kliick, Hermann Felbinger, Petr Blaha, Matus Kozovsky,

Zdenek Havranek and Martin Dosedel

1.4.3 Related Research i ece oi ee e eie r E E EN a 54

1.4.4 Conclusion .. co 55

Contents 1x

1.5 Real-Time Predictive Maintenance — Model-Based, Simulation- Based and Machine Learning Based Diagnosis 63 Franz Wotawa, David Kaufmann, Adil Amukhtar, lulia Nica,

Florian Kliick, Hermann Felbinger, Petr Blaha, Matus Kozovsky, Zdenek Havranek and Martin Dosedel

1.5.2 Application of Diagnosis Systems Based on Simplified DC 1.5.2.1 Simplified DC e-Motor Model With Fault Injection

1.5.2.2 Model-based Diagnosis for Simplified DC e-Motor . 67 1.5.2.3 Simulation-Based Diagnosis for Simplified DC

CEMOLOI aio avn sin th, i i aca aa cal, aaa 71 1.5.2.4 Machine Learning for Diagnosis of Simplified DC e- Motor ie ee, Bes tel lime ae sd oi aa ră ad Ang 75 153° Conclusions tia e eons a i 80

1.6 Real-Time Predictive Maintenance — Artificial Neural Network Based Diagnosis 83 Petr Blaha, Matus Kozovsky, Zdenek Havranek, Martin Dosedel,

Franz Wotawa, David Kaufmann, Adil Amukhtar, lulia Nica, Florian Kliick and Hermann Felbinger

1.6.1.2 Artificial Intelligence in Vibration Diagnosis .... 85

1.6.2.1 Acausal e-Motor Model with Faults Injection

Capability ani. ware oO we Ware i ÎI pg e fa 86 1.6.2.2 Artificial Neural Network for Inter-turn Short Circuit Detection e iat coe ce ad za 87 1.6.2.2.1 Selection of suitable condition indicator

Xx Contents

1.6,4 Conclusion. seci ae ca e a i tah aoe We Sa ea an hats

2 AI Semiconductor

2.0 AI in Semiconductor Industry

Cristina De Luca, Bernhard Lippmann, Wolfgang Schober,

Saad Al-Baddai, Georg Pelz, Andreja Rojko, Frédéric Petrot,

Marcello Coppola and Reiner John

2.0.3 Future Trends for AI Technologies and Applications in

2.1 AI-Based Knowledge Management System for Risk Assessment and Root Cause Analysis in Semiconductor Industry Houssam Razouk, Roman Kern, Martin Mischitz, Josef Moser, Mirhad Memic, Lan Liu, Christian Burmer and Anna Safont-Andreu 2.1.2.Research Areas s socso 244 cuca o ad es 2.1.2.1 FMEA and FMEA Consistency Improvement . . . . 2.1.2.2 Causal Information Extracting from Free Text . . . . 2.1.2.3 Failure Analysis Process, Failure Analysis Reports, 2.1.3; Reflections: în ecs co a ce ye Oe a Bk ae ed Ma oe a 2.154: Conclusion. drea a eS ae a eae, în Ao ss

2.2 Efficient Deep Learning Approach for Fault Detection in the Semiconductor Industry Liliana Andrade, Thomas Baumela, Frédéric Pétrot, David Briand, Olivier Bichler and Marcello Coppola

Contents Xi

2:22 Related Works: i a ee ee ee 133

2.2.4.1 Industrial HW/SW System for On-Device Inference 137

2.2.4.2 Neural Network Building and Training Using N2D2 139 2.2.4.3 Neural Network Export and FPGA Implementation

22-3 CONCUSSION tc a eae cau Ma dCs) eee 143

2.3 Towards Fully Automated Verification of Semiconductor Technologies 147 Matthias Ludwig, Dinu Purice, Bernhard Lippmann,

Ann-Christin Bette and Claus Lenz

2.3.2.1 Methodology: The Integrated Analysis Process . . . 152 2.3.2.2 Example Analysis: From the Image to the Feature

EXTACILON ze e mb 4 setae at low dt ele 156

2.4 Automated Anomaly Detection Through Assembly and Packaging Process 161 Saad Al-Baddai, Martin Juhrisch, Jan Papadoudis, Anna Renner, Lippmann Bernhard, Cristina De Luca, Fabian Haas and Wolfgang Schober

3 AI Industrial Machinery 177

3.0 AI in Industrial Machinery 179 xii Contents

Giulio Urlini, Janis Arents and Antonio Latella

3.0.2 AI Developments and Future Trends in Industrial Machinery

3.1 AI-Powered Collision Avoidance Safety System for Industrial Woodworking Machinery Francesco Conti, Fabrizio Indirli, Antonio Latella, Francesco Papariello, Giacomo Michele Puglia, Felice Tecce, Giulio Urlini and Marcello Zanghieri 3.1.4.4 Latency, Energy and Memory Footprint on S EM32H 74320 aa a ba ok Eee be BA 3:13 Conclusion: aes. A Aia at A ae es

3.2 Construction of a Smart Vision-Guided Robot System for Manipulation in a Dynamic Environment Janis Arents, Modris Greitans and Bernd Lesser 32.272 Operability toa id 3.2.3 Requirements on a en E e e r 000000000 0G

Contents xiii

Hans Cappelle, Ali Gorji Daronkolaei, Ing Jyh Tsang, Björn Debaillie and Ilja Ocket

3.3.2 Gesture Recognition Using a Machine Learning Approach . 223 3.3.3 Gesture Recognition Using a Spiking Neural Network . . . . 228

3.4 Touch Identification on Sensitive Robot Skin Using Time Domain Reflectometry and Machine Learning Methods 239 Pawel Kostka, Anja Winkler, Adnan Haidar, Muhammad Ghufran Khan, Rene Jăkel, Peter Winkler and Ralph Miiller-Pfefferkorn

3.42 State Or the Att 1. pr eats Aone hee ea tai ca $ 240 xiv Contents

ZA GResulis: A Ske eee de tan BE oe el os 245

4 AI Food and Beverage 249 4.0 AI in Food and Beverage Industry 251

Rachel Ouvinha de Oliveira, Marcello Coppola and Ovidiu Vermesan

4.0.4 Al-Based Applications .. . soaa .. 256

4.1 Innovative Vineyards Environmental Monitoring System Using Deep Edge AI 261 Marcello Coppola, Louis Noaille, Clément Pierlot, Rachel Ouvinha de Oliveira, Nathalie Gaveau, Marine Rondeau, Lucas Mohimont, Luiz Angelo Steffenel, Simone Sindaco and Tullio Salmon

4.1.1 Introduction .. cc 262 471.2) Related Work ie. snie e AA Be es 263 4:1:6:°Concluston aia ie eA ee he ae ge od Se ad 276

4.2 AI-Driven Yield Estimation Using an Autonomous Robot for Data Acquisition 279 Lucas Mohimont, Luiz Angelo Steffenel, Mathias Roesler, Nathalie Gaveau, Marine Rondeau, Francois Alin, Clément Pierlot, Rachel Ouvinha de Oliveira and Marcello Coppola

4.2.1 Introduction .. . 2... 2. a 280 4.2.3 Towards an Automated Protocol for Yield Estimation . . . . 282 4.2.4 Assessing the Vine Vitality Using an Embarked LIDAR . . . 285

ADO CONCIISIONS 0 9 Sink BO A ee EO E 286

Contents XV

4.3 AI-Based Quality Control System at the Pressing Stages of the Champagne Production 289 Lucas Mohimont, Mathias Roesler, Angelo Steffenel, Nathalie Gaveau, Marine Rondeau, Francois Alin, Clément Pierlot, Rachel Ouvinha de Oliveira, Marcello Coppola and Philipe Doré

4:3.2 Methodology 1.00.04 22 Ad puede ea be tt e aS 291 4:3:4. CONCIUSION: sys. apt a da Sa e eed alae 296

4.4 Optimisation of Soybean Manufacturing Process Using Real-time Artificial Intelligence of Things Technology 301 Ovidiu Vermesan, Jóran Edell Martinsen, Anders Kristoffersen, Roy Bahr, Ronnie Otto Bellmann, Torgeir Hjertaker, John Breiland, Karl Andersen, Hans Erik Sand, Parsa Rahmanpour and

David Lindberg 4.4.3 Overall Manufacturing System Architecture and Platform . . 306 4.4.5 Edge Processing and Al-based Framework for Real-Time Monitors eii Se a ee ee ae es 310 4.4.6 Embedded Intelligent Vision and Multi-sensors Fusion Approach: ni s dg. iT a ghd GO We otk aa a de 311

4.5 Al and IloT-based Predictive Maintenance System for Soybean Processing 327 Ovidiu Vermesan, Roy Bahr, Ronnie Otto Bellmann, Jóran Edell Martinsen, Anders Kristoffersen, Torgeir Hjertaker, John Breiland, Karl Andersen, Hans Erik Sand, Parsa Rahmanpour and David Lindberg

4-5.) Intro dUCHON cri a a da a 328

4.5.4 Soybean Production Process and Maintenance Policies ... 334 xvi Contents

4.5.5 Al-based Predictive Maintenance Framework

Methodology. a în e iara tace pla ae Gee ae Sh 4.5.6 Industrial Integrated System for Soybean Production

EgUIpImenț mamai a a ei wk we cae a des

5 AI Transportation

5.0 Applications of AI in Transportation Industry Mathias Schneider, Matti Kutila and Alfred Hof 5.0.3 Future Trends for Applications in Transportation Industry . .

5.1 AI-Based Vehicle Systems for Mobility-as-a-Service Application

Mikko Tarkiainen, Matti Kutila, Topi Miekkala, Sami Koskinen, Jokke

Ruokolainen, Sami Dahlman and Jani Toiminen

5.1.2 AFBased 3D Object Detection and Tracking for Automated DEVIN 8 e on araka a ake o e ba e A oh bl ai 5.1.2.3 Evaluation of the Algorithm and Vehicle Integration

5.1.3 Autonomous Control Prototyping in Simulated Environments 5.1.3.1 Reinforcement Learning Control for Mobile Vehicles 5.1.3.2 The Architecture — Immediate Actions Time-Horizon

LA Conclusion: < ss cd ati a eps aS, A a de E aa

5.2 Open Traffic Data for Mobility-as-a-Service Applications — Architecture and Challenges Mathias Schneider, Mina Marmpena, Haris Zafeiris, Ruben Prokscha, Seifeddine Saadani, Nikolaos Evangeliou, George Bravos and Alfred Hop

Contents xvii

5.2.2 Data Acquisition ... ce 376 5.2.2.1 Bus Traces: s ca ma ee e e a are oi 376

3.2.3:2. Bus OPS Trace: mitral it 380

Index 393

Preface

Artificial Intelligence for Digitising Industry — Applications

Industry 4.0 has revolutionised the manufacturing sector by integrating several technologies, including cloud computing, big data, and cyber-physical systems. The goal of Industry 4.0 is to make the manufacturing industry “smart” by integrating machines and equipment that can be monitored and controlled throughout the life cycle.

Industry 5.0 extends technological advances to further facilitate intelligent machine-machine and human-machine collaboration. The goal is to combine the speed, precision, repeatability, and replicability of the operation of machines with the vision, decision-making, and critical and cognitive thinking of human beings. Industry 5.0 can significantly increase the efficiency of manufacturing by extending the use of AI technologies to create a versatile connection between humans and machines, enabling constant monitoring and interaction. This collaboration will enhance the speed and the quality of production by assigning repetitive tasks to intelligent robots and other machines and fostering critical thinking by human beings. Industry 5.0 is characterized by the convergence of technologies and integrates the industrial internet of things (IoT) with Al-based solutions and digital twins to connect physical and virtual manufacturing environments. This convergence makes possible physical and virtual simulations and operating environments in which models based on predictive analytics and managed intelligence enable faster, more accurate and precise, and more reliable decisions. This approach may also provide greener solutions than those of current industrial facilities: end-to-end, environmentally friendly manufacturing solutions with a minimal CO footprint.

Al is transforming industrial environments. Edge-based AI technologies mitigate operational risk, improve the safety and efficiency of manufacturing, optimise processes, and form more reliable and sustainable manufacturing facilities. Adopting AI technology across industrial sectors enables more accurate prediction of anomalies and malfunctions, better management of

XIX

Xx Preface resource consumption, and optimising of manufacturing processes. Artificial intelligence is expected to significantly impact global manufacturing and industrial development. Integrated with other technologies - like intelligent sensors, IoT, digital twins, edge computing, augmented reality, intelligent wireless and cellular connectivity - AI optimises production in real time and facilitates vertical, horizontal, and end-to-end integration.

AI industrial applications harness artificial intelligence to enhance efficiency and sustainability while expediting digital transformations. By applying AI, machine learning, and deep learning, manufacturers can advance operational efficiency, dynamically control, and adapt product lines, customise product designs, and plan technological developments.

This book explores the research, practical results, and exchange of ideas between the representatives of forty-one organisations participating in the AI4DI project to develop the technological community. The concepts presented herein reflect interaction with other European and international projects addressing the research, development, and deployment of AI, HoT, edge computing, digital twins, and robotics in industrial environments to strengthen and sustain a dynamic AI technology ecosystem. These concepts and research results shed light on steps in the evolutionary transition to Industry 5.0. The focus is on five industries: the automotive, semiconductor, industrial machinery, food and beverage, and transportation industries.

The AI4DI project is part of the Electronic Components and Systems for European Leadership Joint Undertaking (ECSEL JU) programme, and the applications and technologies developed by the project partners support the digital transformation of the industry. They are aligned with the priorities of the new European partnership for Key Digital Technologies (KDT). KDT aims to provide innovative electronic components and systems, software, and smart integration to digital value chains, providing secure and trusted technologies tailored to the needs of user industries and citizens to support and reinforce Europe’s potential to innovate. The goal is to develop and apply these technologies to address significant global challenges in mobility, health, energy, security, manufacturing, and digital communications.

The alignment between research, innovation, and industrial policies by using collaborative approaches in mastering the drivers of innovation contributes to and strengthens Europe’s scientific and technological bases.

Editors Biography

Dr. Ovidiu Vermesan holds a PhD degree in microelectronics and a Master of International Business (MIB) degree. He is Chief Scientist at SINTEF Digital, Oslo, Norway. His research interests are in the area of smart systems integration, mixed-signal embedded electronics, analogue neural networks, artificial intelligence (AI) and cognitive communication systems. Dr. Vermesan received SINTEF’s 2003 award for research excellence for his work on the implementation of a biometric sensor system. He is currently working on projects addressing nanoelectronics, integrated sensor/actuator systems, communication, cyber-physical systems (CPSs) and Industrial Internet of Things (IoT), industrial AI with applications in green mobility, energy, autonomous systems, and smart cities. He has authored or co-authored over 85 technical articles and conference papers. He is actively involved in the activities of the Electronic Components and Systems for European Leadership Joint Undertaking (ECSEL JU) and involved in technical activities to define the priorities for the new European partnership for Key Digital Technologies (KDT). He has coordinated and managed various national, EU and other international projects related to smart sensor systems, integrated electronics, electromobility and intelligent autonomous systems such as E*Car, POLLUX, CASTOR, IoE, MIRANDELA, IoF2020, AUTOPILOT, AutoDrive, ArchitectECA2030, AI4DI, AI4CSM. Dr. Vermesan actively participates in national, H2020 EU and other international initiatives by coordinating and managing various projects. He is the coordinator of the IoT European Research Cluster (IERC) and a member of the board of the Alliance for Internet of Things Innovation (AIOTI). He is currently the technical co-coordinator of the ECSEL Artificial Intelligence for Digitising Industry (AI4DI) project.

Reiner John received his degree in Electrical Engineering from the Fachhochschule des Saarlandes (Germany) in collaboration with the University of Metz / Perpignan (France). In 1984 he started his career with the Siemens Semiconductor Group in Munich, where he worked in

XXI xxii Editors Biography automatic test system development. In 1989 he was responsible for the consultation and application of embedded control development tools in the Siemens Automation Group. After joining Siemens Corporate Research and Development in 1991, Reiner John researched knowledge-based embedded systems within the Fuzzy group. Moving to Regensburg to work for the Siemens Automotive Division three years later, he developed concepts and implementations for a real-time operating system to manage and control the engine and transmission system. In 1996 joined Siemens Semiconductors, the later IPO of Infineon Technologies, where he served in several management positions in the Quality and Production Department of the company. In 2000, he further pursued his career in Taiwan, where he set up and managed the Infineon Silicon Foundry Taiwan Office as the Head of Department for seven years. At present, Reiner John is working in AVL List GmbH, Austria, where he oversees the coordination of public-funded R&D projects in the area of trustable AI for industrial and electromobility applications.

Dr. Cristina De Luca received the Laurea degree in statistical and economic science from, University of Padova (Italy) and the PhD degree in mathematics from, University of Klagenfurt (Austria), 2003. She joined Infineon Technologies Austria AG in 2002. She has worked on a wide range of R2R control applications for lithography, CMP and CVD semiconductor production processes and rollout in Regensburg (Germany), Kulim (Malaysia) and Villach (Austria) and contributed to research on R2R for the epitaxy process. Her research interests included advanced process control, automation and statistical data analysis, production automation, predictive maintenance, virtual metrology and industry4.0 automation, model predictive control for semiconductor manufacturing. She was an external professor for statistical quality control at the "Fachhochschule Kárnten" for 2004-2008 in cooperation with Infineon Technologies Austria AG. She is certified in Project Management since 2008. In 2009, she became project manager for European projects, first ENIAC and then ECSEL JU. She followed projects at different levels and contributed to their preparation, implementation, and coordination. To cite some of the projects: IMPROVE, EPPL, EPT300, SemI40, PRODUCTIVE4.0, Arrowhead Tools, AI4CSM. She is currently the coordinator of the ArchitectECA2030 project (Automotive) and AI4DI project (artificial intelligence). In 2019 she joined Infineon Technologies AG, Munich (Germany), where she is Senior Manager Funding Projects and Coordination.

Editors Biography xxiii

Marcello Coppola is technical Director at STMicroelectronics. He has more than 25 years of industry experience with an extended network within the research community and major funding agencies with the primary focus on the development of break-through technologies. He is a technology innovator, with the ability to accurately predict technology trends. He is involved in many European research projects targeting Industrial IoT and IoT, cyber physical systems, Smart Agriculture, AI, Low power, Security, 5G, and design technologies for Multicore and Many-core System-on- Chip, with particular emphasis to architecture and network-on-chip. He has published more than 50 scientific publications, holds over 26 issued patents. He authored chapters in 12 edited print books, and he is one of the main authors of “Design of Cost-Efficient Interconnect Processing Units: Spidergon STNoC” book. Until 2018, he was part of IEEE Computing Now Journal Technical editorial board. He contributed to the security chapter of the Strategic Research Agenda (SRA) to set the scene on R&I on Embedded Intelligent Systems in Europe. He is serving under different roles numerous top international conferences and workshops. Graduated in Computer Science from the University of Pisa, Italy in 1992.

Figure 1.1.1 Figure 1.1.2 Figure 1.1.3 Figure 1.1.4 Figure 1.1.5 Figure 1.1.6 Figure 1.2.1

Figure 1.2.2 Figure 1.2.3 Figure 1.2.4 Figure 1.2.5 Figure 1.2.6 Figure 1.2.7

List of Figures

Further development of core functions in the processes of the automotive industry through AI AI4DI — Development of AI solutions addressing different approaches.

Multilayer perceptron neural network results.

Random forest results.

Retired electric vehicle (EV) battery packs prognosis in GWh per year.

Measurement and data pipeline and the feedback loop into the BMS.

Comparison of a standard 1D convolution and a causal convolution.

Dilated convolutions visualised.

SOH prediction using a TCN with a reference measurement for the whole lifetime of a LIB.

(a) Manually predefined geometric model of the seam, (b) the path editor and the robot jog in tecnomatix process simulate for manual trajectory programming, (c) multiple training environments running in parallel and optimizing the trajectory for a car door, and (d) the robot following the learned trajectory.

XXV

6 13 15 17 17 17 18

22 26 27 28 29 30 xxvi List of Figures

Figure 1.4.1 Figure 1.4.2 Figure 1.5.1 Figure 1.5.2

Figure 1.6.7 Figure 1.6.8 Figure 1.6.9 Figure 2.1.1

Figure 2.1.2 Figure 2.1.3 Figure 2.2.1 Figure 2.2.2 Figure 2.2.3 Figure 2.2.4

DRL agent in the MDP formalization for optimising a TCP trajectory by minimizing the distance to a Four levels of maturity in predictive maintenance. . Simplified DC e-motor diagnosis observations used Simulation-based diagnosis algorithm description. . Simplified DC e-motor diagnosis observation with Normalized confusion matrix - model testing/ verification. .. osa ce es ee ee ee Winding equivalent for extended motor model. . . . Simulated/Emulated faults in extended motor model Motor currents in dq coordinates (healthy/with TERR 8A is alee eed ii dai ati că Filtered data in af coordinates (healthy/with

fail): otet ep het ir a E Oe tS MNN structure used for inter-turn short circuit detection: «+ e e e ee ea ee ee Mean square error of MLP during training phase.

Al-based knowledge management concept system for risk assessment and root cause analysis in System design and optimization process using

Resource occupation (FF and LUT) for 42x42 wafer MADS:. OS weil o e RR

List of Figures xxvii

Abstraction layers in typical computing systems, ranging from software, over hardware design, to the Equally scaled scanning electron microscope images

of semiconductor device cross-sections, showing a 28nm, a 40nm, a 65nm, and 150nm process node. . 150 Example of a cross-section image which shows already interpreted objects on the left side and part

Overall framework of the project. Domain knowledge

and AI methods were the enabler for the use-cases

that are facilitated through the automated SEM A. Normalized histogram per class. B. Various

zoom levels of the same image, magnified 4310, Exemplified overview of the segmentation pipeline. 154 Examples of segmented images with yellow illustrating metal components, and green illustrating

VIAS eto ewe. deo hos dns act ee anne 155 Example features of different semiconductor technologies. The four dimensions were arbitrarily chosen from more than hundred possible attributes defining a semiconductor front-end technology. .. 156 Example SEM CS with the grey-scaled SEM image

Left: Distribution of average curves distance to other samples. Right: the results are showed in left Outliergram, an example of feature for device current traces. Outliers are detected by inspecting xxviii List of Figures

Figure 2.4.9 Figure 3.1.1 Figure 3.1.2 Figure 3.1.3 Figure 3.1.4 Figure 3.1.5 Figure 3.2.1 Figure 3.2.2 Figure 3.2.3

Figure 3.2.4 Figure 3.2.5 Figure 3.2.6

Shows samples of OOI use case. Top left: particle in lower right corner (bottom side). Top right: particle in centre of image (top side). Bottom left: particle in centre of image (top side). Bottom right: scratch in upper area of heatsink (top side). Note that Bottom Show an example of outliers (anomaly) cluster which is clearly inconsistent with the rest of the dataset: sap. yesa e od taa Manca Shows the process flow for the whole process including PDA and supervised learning applied on Anomalies exist at the marked area. In this study, anomaly detection with pre-trained algorithm Shows an example of clustering anomalies units.

Left: shows the clustering according to PAD.

Middle: shows clustering after review process by an expert. Right: shows an example of defect units which recognized as a tick by PAD. As it shown, names represent the real names of classes of labelled Shows the process flow of wire bonding use case. . Visualizations of selected samples of the dataset.

Validation metrics at different augmentation factors.

ROC curves at different augmentation factors. . . . Renderings with different light power levels and Object detection and pick and place operation.

Scene including the plastic bottle- and the reconstructed metal can 3D models (middle) together with the and corresponding depth image

Figure 3.3.4 Figure 3.3.5 Figure 3.3.6

Figure 4.2.1 Figure 4.2.2 Figure 4.2.3 Figure 4.2.4 Figure 4.2.5 Figure 4.3.1

List of Figures Human-robot interaction concept using 60 GHz

Signal processing pipeline to provide input to the feature generator.

Feature extraction for the random forest classifier. . Range vs Doppler (left), flattened range-Doppler vs frames (right): ho dks, ema whee) Se ww ae LSM network with trainable output layer.

Confusion matrix for a 90%-10% learning (1620 samples) and test (180 samples) split of the dataset. Proof of concept radar/robot interface block diagram. oe a eo ee Pe Ta tuia, i Overview of the HMI principle: Basic components of the sensing, computing and control of touch VES MOR ote ane ia d a m ta Network performance during the training: a) accuracy of the force identification and b) loss of Comparative range, data rate, energy efficiency characteristics of communications technologies. . . Operation of the different classes of a

LoRaWAN.

Example of grape segmentation. On the left: the original image. On the right: the segmented image.

Correlation between visible grapes and the total Biomass estimation and vine cane diameters (a) Vineyard robot BakusO, (b) PhysiocapO LIDAR Example of the image processing steps.

STM board and TPU accelerator used in this work.

230 231

293 294

Figure 4.3.4 Figure 4.4.1 Figure 4.4.2 Figure 4.4.3

Figure 4.5.10 Spectrum analysis Wi-Fi 2.4GHz.

xxx List of Figures

The electromagnetic spectrum - regions of interest Boundary tracking for samples with impurities and split soybeans (left) and cleaned soybeans and crushed fractions (right).

Image detection segmentation and processing for samples of cleaned soybeans (left), soybeans with impurities (middle) and crushed fractions (right).

Soybeans images processed by a binary image filter.

13306

Maintenance types. Adapted from EN Standard.

Failure and maintenance timing.

Comparison of maintenance cost and frequency of maintenance.

Parameters monitored during equipment life-time operation. Adapted from STMicroelectronics. . . . AlI-based predictive maintenance framework.

Industrial integrated system for equipment maintenance.

Soybean production predictive maintenance system demonstrator.

Figure 4.5.11 MQTT subscriber with Node-RED flow (vibration sensor per topic).

Figure 4.5.12 MQTT subscriber with Node-RED results (vibration

Figure 4.5.14 Real-time data from the three-axis MEMS vibration

330 332

336 340

342 343 345 346 347 348

348 349

Figure 5.1.2 Figure 5.1.3 Figure 5.1.4 Figure 5.1.5 Figure 5.1.6 Figure 5.2.1 Figure 5.2.2

List of Figures

3D object clustering using a point cloud and 2D

Immediate action control research architecture.

Open data tampere: design for data acquisition.

Bus GPS trace, Line 32 Ranta-Tampella to TAYS AEVO DE a a A ol Traffic cameras and their field of view in Tampere. . Loop detectors for traffic amount measurements Traffic object detection (left) and hourly car Refinement of GPS bus traces: (a) Raw GPS [blue] and planned bus route [green] (b) Snapped bus route to OSM road network [black] (c) Partitioned route according to bus stop vicinity [yellow/purple] (d) Data storage and processing in the cloud. The processed features can be retrieved and visualized as time-series and used for training Al prediction models: so on pl et een Deg eee aide & Weekly data from left to right, top to bottom: traffic amount, congestion, vehicle counts derived from traffic-camera images, queue length and travel-time durations for the segmented bus routes (bus-links).

XXX1

List of Tables

Table 1.5.1 Simplified DC motor component state description. .. 67 Table 1.5.2 Diagnosis results obtained using model-based diagnosis. < o ee we ee ee ee ai tă 71 Table 2.3.1 Framework of the cross-section interpretation with the

Table 2.3.2 Results of measured features of the VIAs. In the right column, the deviation between the automated and the

Table 3.2.1 First object detection results, where the model has

Table 3.3.1 Chirp/Frame (a) and scene (b) radar parameters. . . . 227 Table 3.3.2 Recall and precision statistics of the machine learning

Table 3.3.3 Chirp/Frame (a) and scene (b) radar parameters. . . . 232 Table 3.3.4 Recall and precision statistics of the SNN based

detector. oi a ee ca 234 Table 4.3.1 Results obtained with the STM32MP157C-DK2

Board: woes spy Bag hey alae ae ade a tă 296 Table 4.4.1 Normal parameters measured on whole or cracked

Table 4.4.2 Normal parameters measured on expanded soybean

Table 5.1.1 Percentage of estimated object cluster means correctly xxxiii xxxiv List of Tables

Table 5.2.1 Statistics for open traffic data in tampere (2021-5-31). (*) Traffic cameras images are available starting from November 2019 but are not stored in the

Table 5.2.2 Errors-to-Data Ratio (EDR) for five categories of traffic data collected for the week of February 19 to 25, 2021. EDR is given as a percentage before and after the first step of data cleaning, which involves

List of Abbreviations

Al Artificial intelligence

AI4DI Artificial intelligence for digitizing industry

ANN Artificial neural networks

API Application programming interface

APMF Approximate progressive morphological filter

ASIC Application-specific integrated circuit

BLE Bluetooth low energy

BMS Battery management system

BRAM Block RAM

CatE Computing-at-the-Edge

CBM Condition-based maintenance

CM Confusion matrices

CNN Convolutional neural networks

CPU Central processing unit

CUDA Compute unified device architecture

DC Direct current

DFB Data fusion bus

DL Deep learning

DMA Direct memory access

DNN Deep neural network

ECSEL JU Electronic components and systems for european leadership - joint undertaking

EDR Errors-to-Data Ratio

EIS Electrochemical impedance spectroscopy

EMF Electro motive force

EOL End of life

ERP Enterprise resource planning

ETL Extract, transform, load

EV Electric vehicle

FF Flip-flop

XXXV

XXXVI List of Abbreviations

FFT FoV FPGA GPS GPU HDMI HIL HLS HMI HTTPS HW IDE IloIT IloT IMU ITS JSON LIB Lidar LoRa LoRaWAN LTE LSQ LUT MaaS MBD MIR

Fast fourier transform

Field of view

Field programmable gate arrays Global positioning system

Graphics processing unit High-definition multimedia interface Hardware-in-the-Loop

High level synthesis

Human machine interface, human machine interaction Hypertext transfer protocol secure Hardware

Integrated development environment Industrial internet of intelligent things Industrial internet of things

Inertial measurement unit

Intelligent transport systems JavaScript object notation Lithium-ion battery

Light detection and ranging

Long range (modulation technique) LoRa wide area network

Long-term evolution (standard for wireless broadband) Learned step size quantization Lookup table

Mobility as a service

Model-based diagnosis

Multimedia information retrieval Machine learning

Multilayer perceptron

Material planning decision support system Message queuing telemetry transport Mean squared error

Neural network design & deployment Non-functional requirements

Neural networks

Original equipment manufacturer Open neural network exchange

Open platform communication

OPC UA OpenDDS OS

OSM PdM PLC PMSM PTQ PvM QAT R2F RAM RDBMS RESTful

TCN TensorRT

TPU UI YOLO

List of Abbreviations xxxvii

OPC unified architecture

Open data distribution service

Operating system

Open street map

Predictive maintenance

Programmable logic controllers

Permanent magnet synchronous motor

Post training quantization

Preventive maintenance

Quantization aware training

Run to failure

Random access memory