Corporate Research & Development Center

TOKYO—At a time of growing concern about social distancing and public health and safety, Toshiba Corporation (TOKYO: 6502) has developed a PC-based, high-speed, high accuracy image-analysis AI that monitors crowds and congestion by estimating the number of people in camera images using its unique deep learning algorithm.

Unlike many other solutions, Toshiba’s AI does not require a high-end graphics processor unit (GPU)^{(Note 1)} and can run on a PC with a typical central processing unit (CPU)^{(Note 2)}, and handle high-speed analysis of a total of 180 images a minute. Even so, the new AI both reduces the cost of estimating crowd numbers and improves on accuracy—compared to other approaches, Toshiba’s new AI reduces estimation error per image from 16.0% to 14.7%, confirming its world-leading accuracy^{(Note 3)}.

In today’s “new normal,” when people are equally concerned about being able to go about their daily lives and the heighten risk of infection from being in crowds, Toshiba’s new AI is a public health and safety tool that delivers the ability to quickly identify and manage crowds and congestion. It makes it possible to carry out crowd analysis in many different kinds of location with only minimal computing resources.

Toshiba will present this technology at the 26th Symposium on Sensing via Image Information (SSII2020) held from June 10 to 12, 2020.

As more security cameras are installed, with estimates of 1-billion worldwide by 2022, their usefulness in monitoring the movement of traffic and individuals has made them important tools for public safety. The Covid-19 pandemic has also shown how monitoring crowds can contribute to public health by identifying congestion hot spots. Advances in deep learning AI, an area where Toshiba cultivates expertise, now make it possible to estimate crowd numbers and density at high speed with high accuracy.

Until now, cost has been a hurdle to the wider use of technology for estimating crowd size, according to Yuto Yamaji, an AI researcher at Toshiba’s corporate R&D Center. “Current approaches need high-end hardware, a powerful GPU to handle the scale and complexity of the computing. Installing technology like that across large facilities is expensive,” he explains. But that’s not the only concern. “There’s also an accuracy problem. As the result of applying a fixed scale to image analysis, it’s harder to realize accurate detection if the apparent size of a person changes, depending on distance from the camera.”

Toshiba approached the first problem by developing a deep network that uses only the CPU, the core component on any PC, for all high-speed processing. Toshiba has confirmed that the AI is able to provide high-speed analysis that can handle one image from three cameras every second, a total of 180 images a minute with a single PC.

Alongside this, the company developed an algorithm to boost count accuracy. It compensates for perspective by creating a network structure that allows multiple groups to be identified and analyzed, regardless of size, as shown in Figure 3.
Evaluated against images from a public dataset^{(Note 4)} of densely populated locations, Toshiba’s new AI reduced estimation error for a single image from the 16.0% that has already been achieved to 14.7%, confirming its world-leading accuracy.
It extracts data from the images, and visualizes it as a density map overlaid on images, it is possible to display areas of congestion and slow-moving flows of people.