The Problems with Determining the Grades of Steel and Other Materials via Visual Inspection

Steel products are processed goods into shapes such as plates, rods, and tubes used as materials of buildings, machines, and other objects. Steel product has rigorous requirements for strength, resilience, and other properties. Its quality is affected by the presence of minute non-metallic impurities (non-metallic inclusions) that get into the steel product during its manufacturing processes. Non-metallic inclusions such as oxides and sulfides are difficult to remove completely. Depending on their structure, size, and amount, they can create a risk of structural cracking or product breakage.

Thorough metallographic structure inspections are performed in the quality management sites of steel manufacturers and component manufacturers -- suppliers that process steel to manufacture components such as automobiles and aircraft --. Samples cut out of steel product are inspected by inspection personnel using microscopes. They visually determine properties such as the types and amounts of non-metallic inclusions in the steel, steel grain size, and the like. Normally, grain size is compared against JIS*-defined standard drawings and the grain number is used to determine the grade of the steel.

* JIS: Japanese Industrial Standard

This inspection process, which might appear at first glance to be rote work, requires a great deal of experience. It is greatly affected by the experience and sensitivity of individual inspectors, so uneven evaluation results are a concern. Furthermore, performing inspection work for long period of time places much strain on inspectors, which leads to problems such as grading errors or errors in the entry or transcription of quality data. Furthermore, it is difficult to secure sufficient personnel with extensive experience, and for them to pass on their abilities to others. Training a new inspector to the level required for full-fledged inspection takes years, and is costly. In our modern age, in which personnel shortages are rapidly growing in the industrial field, these problems present serious challenges for improvement of steel product reliability, maintaining of competitiveness, and further development.

METALSPECTOR II-C, the Product of 30 Years of Toshiba Experience

In 1988, Toshiba Digital Solutions (then Toshiba Corporation) released a non-metallic inclusion measurement device that used image sensors to tackle the issues of metallographic structure inspection. At the time, the image processing used monochrome images, but, nonetheless, it was an innovative product that helped improve the efficiency of quality inspection sites by automating processes and producing results that were closer to those of the visual inspections by metallographic structure inspection personnel.

The non-metallic inclusion measurement device used a camera in place of an inspector's eye and a controller in place of the mind used to determine quality. We continued to refine the image processing technologies used to determine quality by processing image data taken with cameras, releasing advance after advance. Through this process, we built up a wealth of expertise. METALSPECTOR II-C is the embodiment of this expertise.

METALSPECTOR II-C supports not only measurement methods that conform with steel product quality assurance standards, but also manufacturers' own proprietary measurement methods. It became possible to combine our own unique measurement algorithms and autofocus functions to process single samples using the ASTM* method in roughly six minutes, and to use JIS point calculation method in roughly two minutes. Furthermore, it is capable of highly accurate measurement, with a deviation of just ±0.25μm when measuring the widths and lengths of inclusions identified through visual inspection using an electron microscope. When there are multiple types of inclusions, such as sulfides and oxides within those sulfides, it can identify each separately, performing image processing at a level unrivaled in the past. It is currently in use in many quality inspection sites (Fig. 1).

* ASTM: American Society for Testing and Materials

Integrating Image Inspection Expertise and AI and Offering Them as a Managed Service

We are now developing services that leverage their fullest the vast amounts of metallographic structure inspection data we have accrued by developing and supplying METALSPECTOR II-C, as well as our rich expertise regarding quality inspection operations. As a partner to our customers, we have observed steel product quality closely on-site, and, in October 2019, we began providing Albacore, Toshiba Managed Services. We have integrated our image processing technologies, in which we have an extensive track record in steel manufacturer and component manufacturer worksites, with advanced deep learning technologies. We provide AI-based, high accuracy image analysis in the form of AI Image Processing Service for steel products, a SaaS service that is easy to use in sites where metallographic structure inspection is performed.

We have verified the effectiveness of the learning models that form the heart of this service in actual worksites. The service has been confirmed to produce results with a level of recognition accuracy close to that of experienced inspectors. It has been recognized as a technology ready to be put into immediate use in workplaces where rigorous quality management is essential.

This Steel AI Image Processing Service is integrated with METALSPECTOR II-C.

Steel product micro-structure image data acquired using METALSPECTOR II-C is used by cloud-based AI to automatically determine and analyze quality feature quantities of steel product. A model was automatically generated for determining the grades of steel grain sizes. Grade inference results are then reported to a worksite operation terminal (Fig. 2).

This is how the system performs metallographic structure inspection using a uniform standard, without using a program approach, that is capable not only of standard measurement methods, but micro structure inspection as well. In this way, not only standard measurement methods but also metal structure inspections of the same standard corresponding to microstructural inspections are realized without programing. Furthermore, a consistent process automation up to the report writing corresponding to various standards is available.

The Advantages of Managed Services

One of the advantages of managed services is making it possible to use the latest AI analysis continuously, even without specialized knowledge concerning AI. The more it is used, the more accurate the analysis becomes. AI learns and develops autonomously, continuously developing its skills, just like a veteran inspector. Inspection results are reported without errors or omissions, and stored quality data is robustly managed, so there is no need to worry about user audit trail management or security. If quality information comes to be shared across organizations and a uniform quality standard is developed within the industry, the AI Image Processing Service for steel products will evolve into a AI platform used by every company.

In the future, Toshiba Digital Solutions will release managed services that use AI and cloud technologies to perform image inspection to detect problems with PTP sheets, which are used in pharmaceutical manufacturing to package pills and capsules, and to detect problems in the surfaces of web products such as film, paper, and non-woven cloth. These image inspection solutions, which are already in use in a wide range of fields, will be evolved into AI image processing services.

Toshiba Managed Services, which overcome the difficulties involved in passing on techniques and solve the social problem of labor shortages to help improve manufacturing quality, will continue to grow and evolve.