Toshiba Develops Acoustic Wear-estimating AI Technology for High-accuracy Analysis of Equipment Operating Sounds and Detection of Signs of Equipment Wear

TOKYO—Toshiba Corporation (TOKYO: 6502) has announced its development of “Variational Autoencoder-based Deterioration Estimation (VAE-DE),” an acoustic wear-estimating AI that analyzes equipment operating sounds and detects signs of wear with high accuracy. This AI allows early detection of signs indicating equipment wear without false positives, even in the presence of ambient noise from the environment or electrical noise from circuits. To verify the effectiveness of this AI, Toshiba used simulation data based on operating sounds from cooling fans actually used in power facilities for several years and noise collected at power facility installation sites. By investigating correlations between the “estimated wear value,” a value obtained by estimating a device’s extent of deterioration based on its operating sounds, and the “deterioration trend,” which indicates the actual state of equipment wear, Toshiba has found significant improvement in the correlation coefficient, which increased from 0.144 to 0.905. Those results confirm that the developed method can estimate wear with high accuracy while suppressing false positives due to noise, a feat that has been difficult to achieve in the past.
This AI makes it possible to determine the extent of equipment deterioration based on its operating sounds and to perform maintenance at appropriate times. Toshiba will present the details of this technology at the 9th International Conference on Condition Monitoring and Diagnosis 2022 (CMD2022), an international conference related to diagnostic technologies for power facilities, to be held from November 13 to 18 in a hybrid format, online and in Kitakyushu, Fukuoka Prefecture, Japan.

In recent years, there has been growing interest in “condition-based maintenance,” where in place of the conventional approach of maintaining equipment through periodic inspections by humans, AI is used to monitor the condition of equipment and perform maintenance when needed. The sounds of operating devices provide important information for monitoring equipment. For equipment that operates continuously or with repetitive motions, monitoring of operating sounds is an effective means of measuring its condition, because any abnormalities or deterioration will become evident. Microphones for collecting operating sounds also have the advantages of low cost and few installation restrictions.
There have been advances in the development of AI-based technology for using operating sounds to monitor equipment status. While learning normal operating sounds allows AI to detect sounds that suggest equipment abnormalities or deterioration with high accuracy, an outstanding issue has been that, at actual worksites, systems can pick up ambient noise such as that from other devices or air-conditioning systems, causing the AI to incorrectly judge such noise as an abnormality.
In particular, devices that operate over long periods generally deteriorate gradually, even over a period of several years, and abnormalities appearing in their operating sounds are faint at first. Conventional AI techniques are thus more sensitive to ambient noise when attempting to detect wear at an early stage, leading to more false positives. Under such circumstances, condition-based maintenance requires both early recognition of deterioration trends and fewer false positives due to noise.

To facilitate distinguishing between normal operating sounds and sounds indicative of wear, Toshiba has developed VAE-DE, an AI technology that uses a neural network, which is an AI computational model that mimics the structure of the human brain. This AI learns the characteristics of the operating sounds of equipment and combines high performance in detecting minute changes in operating sounds with robustness against ambient noise. This AI uses a variational autoencoder (VAE) network, a deep-learning method that automatically finds and learns data features, using unique criteria designed to separate normal sounds from sounds indicative of wear (Fig. 1). Conventional VAE rarely produces false detections even when normal sounds are mixed with noise, but there is still a risk that weak signs of wear will be overlooked as normal sounds (Fig. 2). The newly developed AI is based on a VAE method that is less prone to false positives due to noise and performs learning based on new proprietary criteria that separate normal sounds from sounds indicative of wear, so that only those faint sounds indicative of wear are judged as being outside the normal range of sounds.
Cooling fans were used to verify the effectiveness of this AI. Cooling fans are installed in many devices and are essential for their proper operation. These fans constantly rotate at high speeds, so they often deteriorate faster than other parts of the device they are installed in. However, the rate of deterioration greatly varies depending on the individual fan and the conditions under which the device is used. Condition-based maintenance is therefore desirable, so that cooling fans can be replaced before they fail and stop working.
By first recording and analyzing sounds from operating cooling fans, that was in actual use in power facilities for over several years, it was confirmed that deterioration trends occur probabilistically and can be detected as a slight increase in high-frequency sounds that are inaudible to humans. Next, the developed AI was applied to simulated data that reproduced sounds indicating wear, together with electrical noise generated from power equipment. Regarding the correlation between the deterioration trend, which indicates the actual state of equipment wear, and the “estimated wear value,” which is the value estimated from the operating sounds, the correlation coefficient was greatly improved from 0.144 to 0.905, indicating an extremely strong correlation. This confirms that the developed AI can infer the presence of wear with high accuracy while suppressing false positives due to noise, a feat that has been difficult to achieve in the past (Fig. 3).
The development of this AI will aid in the realization of effective condition-based maintenance from the onset of wear, the timing of which differs by device, by capturing early signs of equipment abnormalities or wear while minimizing false positives due to ambient noise.

Fig. 1: Overview of the proposed method.

Fig. 2: Detection of weak deterioration trends by the proposed method.

Fig. 3: Relationship between extent of deterioration and estimation of wear, as seen in evaluations. Error bars indicate variation in estimates.

Toshiba aims to apply the developed AI technologies to cooling fans for power supply equipment as quickly as possible. Another aim is to extend the application of this system to other devices that operate continuously for long periods of time in factories and IT facilities, both inside and outside the company, thereby contributing to the realization of high-precision condition-based maintenance.