Development of a technology for high-performance measurement of group sizes

Overview

We have developed a new technology for precisely calculating the number of persons and vehicles in a wide-angle video shot from a single camera. (Ref. 1)

This technology is applicable to even videos where target objects overlap or are extremely small, and publicly released evaluation image data^{(Note 1)} shows the highest performance yet achieved in terms of calculation error. We will present the results of this technology on 15 December at ICCV 2015 (Santiago, Chile), the world's largest computer vision conference.

Development Background

There are currently around 28 million security cameras installed worldwide, and because of their utility in helping to solve crimes, the need for further installations continues to grow. Image processing technologies such as face recognition and human detection are also being implemented with the goal of creating a safer and more secure society and providing more advanced security solution services. Human and vehicle traffic density and congestion are factors that frequently contribute to problems, so in regard to smoothing and improving the efficiency of administrative work through labor savings and improved visualization, there is utility in more accurately calculating the congestion level of persons in a group. Current technologies for group measurement estimate group's sizes by using a measurement index called a density map^{(Note 2)}. However, density maps estimate the existence probabilities of persons or objects based on the brightness levels of all pixels in the target image, meaning that numerous calculations are required, and performance improvements generally require the use of large amounts of memory.

Features of the Technology

In contrast to conventional methods for directly estimating a group density map from images, we have developed a unique machine learning method called Co-voting an Uncertain Number of Targets (COUNT). Whereas conventional technologies learn existence probabilities to represent whether a human is present in each pixel, our method avoids such per-pixel decomposition and instead statistically learns the number and position of persons in an image, thereby estimating their offset position vectors. These estimated positions are used to calculate a group density map by superimposing previously created single-person density maps following a normal distribution^{(Note 3)}. This enables learning based on group sizes and locations, and as a result, the number of persons and objects can be estimated with less memory use and higher precision compared with conventional methods (Ref. 2).

Previously published best-performance data^{(Note 4)} had an average absolute error^{(Note 5)} of 3.43 persons, but our method has improved upon this, with an average absolute error of 2.50 persons. For another set of published data, the average absolute error was 2.10 persons by a conventional method but 1.59 persons by our method, which used only one-thirtieth of the memory.

Future Developments

By 2016, we hope to incorporate this technology into our RECAIUS cloud service, which allows IT activities while seeing, hearing, and speaking with others. This technology allows balancing and energy reduction through discovery and notification of locations where many people are congregating. Moreover, visualization of such locations allows for consideration of congestion mitigation measures. The method can be further expanded to allow for simultaneous calculation of the number of pedestrians and vehicles at locations such as congested intersections.