Overview
Toshiba Corporation has developed an unsupervised image classification AI that significantly improves the accuracy for benchmark images from 27.6% to 83.0%. This is achieved by utilizing our proprietary deep-learning-based unsupervised AI, which classifies types of defects and failures in inspection images accumulated at the manufacturing site without any manual labeling tasks.
Unsupervised image classification AI had a general problem that the AI recognized features of background normal product patterns that are unnecessary for defect and failure classification, resulting in lower accuracy. However, this AI ignores unnecessary features from the background pattern, learns only the important features related to product defects and failures, and classifies images with high accuracy. This AI is effective for inspection images with complex product patterns, such as those in semiconductor manufacturing field, reducing the time-consuming manual work, thereby accelerating the analysis process for quality improvement.
Toshiba will present the details of this technology at the international conference PAKDD2024 (Pacific – Asia Conference of Knowledge Discovery and Data Mining) (*1) to be held from May 7 to 10 in Taiwan.
Development background
In recent years, there has been a growing need for highly accurate image classification AI that can classify inspection images in the manufacturing industry to quickly identify the occurrence of defects and failures and improve productivity. There are two types of AI: supervised AI, in which the target to be classified are manually given to the AI in the form of image labeling, and unsupervised AI, in which the AI learns features of the target by itself. In general, higher classification accuracy can be expected from supervised AI, but the high cost of manual labeling work is a barrier to the use of AI in manufacturing. Unsupervised AI is easy to use and has the advantage of being able to learn image features that humans do not expect. While it is expected to be able to classify unexpected defects and failures caused by changes in manufacturing processes, improving classification accuracy has been an issue for practical application of unsupervised AI.
There have been many works and proposals to improve the accuracy of unsupervised AI. However, conventional unsupervised image classification AI has difficulty distinguishing important features that affect grouping from background features that are unnecessary in product inspection (Figure 1). For example, while the effect of a simple background, such as the color of the sky, can be ignored, in the case of a complex background, such as a semiconductor circuit, the classification accuracy of the defects or failures will be reduced by the effect of background pattern.
(a) Normal product pattern (unnecessary background features)
(b) Product pattern with defects
Features of the technology
Toshiba has developed an unsupervised image classification AI that identifies only important features such as defects and failures from the target image containing the classification target by suppressing the learning of unnecessary features from the background pattern image (Figure 2). This AI consists of a new “background feature extraction network” that learns unnecessary features in background patterns using deep learning AI and an “target feature extraction network” that learns features in the target image that contain defects or failures. This enables efficient extraction of necessary features from the target image while ignoring unnecessary features in the background pattern, and highly accurate identification and classification of defects and failures. We evaluated this AI using benchmark images that simulate actual product patterns.
This technology can be used efficiently in the manufacturing process by using the inspection images of good products accumulated in the manufacturing line inspection as background patterns. When the classification accuracy was verified with benchmark data of background images with stripes and target images with handwritten digits on the striped background, the classification accuracy was significantly improved from 27.6% to 83.0%, that is sufficient for practical use in the manufacturing field. With conventional AI, groups were divided by the combination of handwritten digits and background due to the stripe pattern, but with this AI, groups are divided by the target feature of the handwritten digits only, improving classification accuracy (Figure 3). Application of this AI to inspection processes at manufacturing sites is expected to greatly improve the accuracy and efficiency of quality control. The automation provided by this technology also allows engineers to focus on more advanced analytical processes, leading to overall yield improvement in manufacturing.
Figure 2: Overview of Developed Unsupervised Image Classification AI that Suppresses the Effect of Unnecessary Features
Future developments
Toshiba plans to introduce this AI into the Group's semiconductor factories and apply it to various inspection processes and products to verify its performance. We will continue to improve the performance and system, aiming for early practical application.