In their empirical study, Bauer et al. [1] found that the lack of sufficient employees (with ML knowledge) and limited budget are part of the most frequent significant challenges for SMEs. This can lead to rushed approaches, which end in a database with low-quality data. However, an essential requirement for a successful application of AI in the industrial context is a database with high-quality data, e.g., from production and testing processes. The practical application of AI algorithms often fails due to

as shown in [9]. Furthermore, industrial data are typically acquired continuously without saving relevant metadata. In addition, this often leads to a brute force approach, which tries to use all acquired data. Large data sets are subsequently challenging to manage, and their use is computationally expensive. A knowledge-driven approach can efficiently use resources and increase the information density within the data, e.g., by reducing the amount of used sensor data due to process knowledge. By recording data in a targeted manner, redundancies can also be avoided. The necessary process knowledge to analyze data, especially in SMEs, is often limited to a few employees and cannot be easily accessed by colleagues. Those specialists might also not be willing to share their knowledge in fear they lose their distinctiveness against other employees [10]. In the worst case, the (process) knowledge is lost if the specialist leaves the company.

Due to the specific challenges in the analysis of industrial data mentioned by Wilhelm et al. [9], the accessibility of the data itself and the accessibility of domain-specific knowledge play a crucial role in obtaining robust ML models or further insights and knowledge about, e.g., products or processes. Module 1 of PIA, therefore, consists of a UI to access easily (meta)data and knowledge as well as a knowledge base in the form of a lessons learned register. M1 can be seen as a complementing add-on component to internal knowledge repositories like company wikis [11] and databases like InfluxDB [12] or MongoDB [13]. Guidance on recording and structuring data and metadata can be found in the checklist [4] presented in chapter 2.3 as well.

To learn from previous projects and retrieve knowledge to use in future projects, lessons learned are a well-established method in project management [14]. In the context of the framework PIA, the lessons learned register is organized and structured like the checklist presented in M2 and contains high-quality lessons learned for each specific chapter of the checklist. Rowe et al. [14] structure the formulation of lessons learned into the five subsequent steps: identify, document, analyze, store, and retrieve. Moreover, they describe these steps in more detail and provide a template for lessons learned. The technical standard DOE-STD-7501-99 [15] suggests that each lesson learned should contain the following five elements:

Additionally, Patton [16] distinguishes lessons learned into lessons learned hypothesis and high-quality lessons learned. While a lessons learned hypothesis is a lesson learned with one supporting evidence, high-quality lessons learned could be approved in multiple projects. To ensure the quality of the lessons learned, Patton [16] further formulated ten questions in his paper for generating such high-quality lessons learned. Moreover, he recommended reviewing lessons learned periodically regarding their usefulness and sorting out obsolete lessons learned to maintain high quality.

Figure 2 shows the approach proposed in this contribution. After analyzing a given use case, specific results were achieved. The whole project is evaluated in a retrospective, and lessons learned are formulated according to the five steps of Rowe et al. [14]. If the lessons learned (hypothesis) can be validated in further projects, they get added to the lessons learned register. The lessons learned register is reviewed regularly to ensure relevance and actuality.

The foundation for successfully applying AI in industry is a high quality of the underlying data. The research field measurement and data planning can be seen as an early part or requirement of data mining, which is the process of extracting knowledge from data sets using computational techniques [17]. For industrial data mining, the Cross-Industry Standard Process for Data Mining (CRISP-DM) established, which divides data mining into the six non-sequential and independent phases: business understanding, data understanding, data preparation, modeling, evaluation, and deployment [18]. Since data mining consists of several disciplines, each of which has its own research area, inexperienced users can feel overwhelmed and demotivated mainly because the industry’s focus is not to record high-quality data but to use data to increase efficiency and margin. A guide or checklist can help users to orientate and access the field of data mining. Here several approaches can be found in literature like, e.g., A Checklist for Analyzing Data of [19] or the Analytical Checklist – A Data Scientist’ Guide for Data Analysis [20]. The checklists mentioned are universally applicable but lack information for the realization of a data analysis project and assume that data sets are already recorded. Other concepts, like, e.g., FAIR of Wilkinson et al. [2] ensure high-quality data and offer practicable solutions like The FAIR Cookbook that guide new users but are primarily focused on research data.

The Checklist - Measurement and data planning for machine learning in assembly of Schnur et al. [3] tries to find the sweet spot between having a broad scope, transferring knowledge from research data management but still being clearly structured and not overwhelming for an inexperienced user. Within PIA, the checklist enables the users of PIA to perform a machine learning project from the beginning to the end and record high-quality FAIR data. It covers the following chapters:

Each chapter begins with a short introduction, followed by checkpoints that guide the user. Here, two types of checkpoints exist: necessary and best-practice checkpoints. While the best-practice checkpoints are optional but highly recommended, the necessary checkpoints must be executed. Furthermore, the checklists provide tips and notes derived from high-quality lessons learned from previous ML projects and further literature suggestions. The checklist is based on a revised version of the CRISP-DM mentioned above. Therefore, some parts of the checklist are iterative.

The checklist was initially published in German on the file-sharing platform Zenodo and has been translated for integration into PIA to English, which also increases accessibility and re-usability [4].

In nowadays industry, an extensive range of tools and software for data analysis exists. Some well-known and established examples are the software library Pandas for Python [21], Power BI [22], the Statistics and Machine Learning Toolbox of MATLAB ^® [23] or the open platform KNIME [24]. All the solutions mentioned are powerful data analysis tools and offer a broad spectrum of algorithms.

Within the concept of PIA, the focus lies on time-continuous data. However, potential users can implement every ML tool or algorithm that suits their use case and data into M3, starting from traditional approaches like feature extraction, selection, and classification/regression to modern approaches like deep learning with neural networks. However, it could be shown by Goodarzi et al. [36] that traditional approaches could perform similar to modern approaches while being less complex and have a higher interpretability. To cope with the heterogeneous data sources in industrial applications, especially as an inexperienced user, a set of different feature extraction and selection methods can be beneficial [37]. For the implementation of PIA within this study, the authors use the existing automated ML toolbox for time-continuous data of Dorst et al. [6] and Schneider et al. [7]. This toolbox automatically tests different combinations of feature extraction and feature selection methods with linear discriminant analysis and Mahalanobis distance as the classifier. This automated ML toolbox combines five complementary feature extraction methods with three feature selection methods, as shown with their corresponding literature in Figure 3. A 10-fold cross-validation automatically determines the best of the resulting 15 combinations by ranking the combinations according to their resulting cross-validation error on the test data [38], [39]. Due to the different focus of each algorithm (shown in Figure 3), the toolbox could achieve good results in a broad application range ([37], [40]).

Users can perform a first ML analysis with the toolbox by running five lines of code:

For further analysis or regression tasks, the methods can be modified, changed, or applied separately.

As a validation use case for this contribution, an assembly line with two stations was chosen (Figure 4a) that produces a device holder (Figure 4b). In the first station, a robot picks up the individual parts of the device holder from a warehouse and places them on a workpiece carrier. The product is transported to station 2 by a belt conveyor for the next step. There, a worker assembles the two components by a bolting process. In addition, the device holder can be produced in another variant (Figure 4c). The use case is presented in more detail in [41].

The combination of two different stations with different processes and different degrees of automation, as well as the opportunity to produce a second variant of the device holder, make this assembly line a good use case for demonstrating the flexibility of PIA while keeping the complexity low (compared to more extensive assembly lines).

The Angular framework, an open-source single-page web application framework, has been chosen to demonstrate the concept of PIA. Angular 13.3.4. allows fast development of the demonstrator and gives the possibility that, once the demonstrator is hosted on the web, PIA can be easily accessed from any device within the intranet.

Figure 5 shows a schematic representation of the development environment of PIA. For simulating the user experience, the type-2 hypervisor Oracle VM VirtualBox (https://www.virtualbox.org/) from Oracle Corporation was used with Ubuntu as the guest operating system (OS). However, in general, Angular can also be used on Microsoft Windows or Apple macOS.

Table 1 gives an overview of the used environments and Table 2 of the used libraries, each with their corresponding sources.

Besides Angular (E1), the environments Node JS (E2) and Node Package Manager (E3) are used. Angular’s primary architectural features are a hierarchy of components. Using this structure, the various PIA functionalities have been separated into components for ease of use and reuse. Table 3 provides an overview with a short description of the components used in PIA. This structure also allows to easily add new components to the application without interfering with existing ones. The Angular Material library (L2) provides a consistent experience across the website. Specific dynamic components have also been made responsive using Bootstrap and Flex-Layout libraries (L3, L4). To make it easier for future developers to add new information to the website, data about each process has been saved in JSON format and then queried to display the relevant information in the UI. Users or developers can easily add more plants or tools to the application by editing the relevant JSON file, which will be dynamically displayed in the UI.

Figure 6 shows the landing page of PIA. Over a menu, the user can navigate through the four menu points:

The implementation of M1 contains two parts, accessibility of data and metadata (menu-point: Plant) and a lessons learned register (Knowledge base). The plant module displays information, data, and metadata about various plants. Figure 7 shows an example flow-through of the use case in this study. After clicking on the Plant button, the available stations of the plant are displayed: Gripping Process and Bolting Process. After selecting a process (Figure 7, green box), the user can select between the following options:

All information is contained in an array of JavaScript objects. Therefore, a new plant, station, or resource can be easily included by adding new objects to the array in the same format and assigning it on the front-end. Here, Angular material cards (L2) are used to display further information, e.g., process resources. An example of the basic structure of the array of objects for a Station with one process which includes a robot and relevant metadata, e.g., technical data or technical drawings, is shown in list. 2 (Section A).

Further instances of the resources, e.g., a gripper for the robot, can be easily added by creating a new object with ID, name, and paths corresponding to the documents and images in the assets folder and providing the relative paths to the corresponding documents. The new instance will automatically be displayed in the UI after recompiling. Furthermore, the button Knowledge base (Figure 7, blue box) contains specific knowledge about each resource.

The second part of the knowledge base contains the lessons learned register and a simple example of the link to general knowledge. The general knowledge was implemented illustratively as a graphical representation of the assembly processes in the form of a tree. Here, the user can expand the tree by selecting the respective nodes to access the sub-nodes that describe the next steps of the process described in the parent node. The information component has been integrated with the nodes, which can provide further descriptive information about each node.

The implementation of the lessons learned register is shown in Figure 8. In the suggested version of a lessons learned register, each lesson learned is generated by the process shown in Figure 2 and grouped by their respective project step (chapter) of the checklist (Figure 8, blue box). After selecting a chapter, the lessons learned appear on the right-hand side (Figure 8, red box). Users can add criticism to existing lessons learned, lessons learned hypotheses, or additional files in the Comment Section (Figure 8, green box). The Comment Section is reviewed at regular intervals and, if necessary, transferred to the Lessons Learned Register.

Using the presented structured, high-quality lessons learned register allows the user to easily access domain-specific knowledge in a target-orientated manner and to avoid previously made mistakes in earlier projects. The presented GUI enables users to find relevant information intuitively rather than searching for information in multi-layered folder structures with restricted accessibility.

The checklist implementation uses the tree component of the Angular Material (L2) library, which allows the present hierarchical content as an expandable tree (Figure 9, blue box). Each node of this tree displays information about itself and additional tips or hints (Figure 9, red box). Furthermore, comments and files can be added to each checkpoint (Figure 9, green box). This helps employees who are new to the project to catch up and comprehend past steps. When the user ticks through all the sub-nodes, the primary process node is automatically ticked, indicating that all the sub-processes have been completed.

The Checklist for Measurement and Data Planning guides and supports users holistically through a data analysis project while highlighting trip points. Using the checklist in an implemented version rather than the printed version, the status and progress of ongoing projects can be tracked and understood by non-involved users in case a worker gets sick or leaves the company.

Since PIA is implemented as a front-end demonstrator with no back-end, the data analysis is carried out in MATLAB^® Online™. Figure 10 shows the results of the data analysis with the ML toolbox for data of the example use case provided in [6]. The toolbox can be directly connected to GitHub into MATLAB^® Online™ (Figure 10). As shown in the blue box of Figure 10, the user can access other algorithms by clicking through the folder structure. After executing the code (Figure 10, green box), the user can plot the results (Figure 10, red box). For interpreting the results, the user can follow the subsequent steps of the checklist in M2 while using the knowledge, data, and metadata provided in M1.

The implemented toolbox can be used by users with little programming experience while covering a broad range of algorithms for analyzing industrial time-continuous data with ML. However, the users can also embed other, e.g., use-case-specific algorithms.