Corporate Research & Development Center

Tokyo—Toshiba Corporation (TOKYO: 6502), an innovator in technologies for advanced driver-assistance and autonomous driving systems has developed AI, that will improve traffic safety. Motion Estimation AI uses on-board cameras and inertial sensors that measure acceleration and angular velocity to deliver highly accurate estimates of the movement of a drone or a vehicle, while Motion Prediction AI predicts the movements of other vehicles on the road in a wide range of traffic conditions. Tested against a public data set, they both achieve the world’s highest accuracy^{(Note 1)}. Details of the technologies were presented at International Conference on Robotics and Automation (ICRA), streamed online on June 1, 2020.

The many advances that electronics and AI have brought to road vehicles have reduced accidents and the loads imposed on drivers—and are also very popular. In 2018, the car industry sold 20 million vehicles with advanced driver-assistance systems, and the figure is expected to pass 80 million in 2030^{(Note 2)}. The ultimate goal is autonomous vehicles.

A safe driving system must be able to simultaneously track the position of the vehicle it is mounted in, and the surrounding environment: the real-time movement of other vehicles, pedestrians, potential hazards and obstacles. Technologies that use sensors like LiDAR^{(Note 3)} or GPS have been developed to do just that, but the former is expensive, and shadows from tall buildings can block the latter. This has prompted growing interest in Simultaneous Localization and Mapping (SLAM) technology, which uses images from on-board camera to estimate a vehicle’s location and to make three-dimensional spatial maps around the location, and motion prediction technology, which uses past-to-present movements of other vehicles to dynamically estimate their position over time.

To be really applicable, technologies like these must meet the test of being usable in all kinds of road conditions and complex traffic flows, not only on expressways where vehicle movement is relatively simple and predictable.

Toshiba’s new AI solutions advance the accuracy of SLAM and motion prediction, and will contribute to the realization of autonomous driving.

Motion Estimation AI combines data from in-vehicle camera images and inertial measurement unit (IMUs), including angular velocity and acceleration sensors, to estimate vehicle movement, and performs robustly in all kinds of traffic conditions. However, there is a problem with this approach^{(Note 4)}, in that the data from the inertial sensor can be affected by the velocity of the vehicle—its acceleration and deceleration—which adversely impacts on movement accuracy.

Toshiba has developed a method that constantly estimates vehicle motion by determining the usefulness of the data from the camera image and the IMUs and using only the relevant data. Motion Estimation AI can estimate the motion of cars that travel at a relatively constant pace, with limited acceleration and deceleration. Tests using a public driving data set^{(Note 5)} confirmed that the AI produces 40% fewer errors than the current method of estimates based on all of the data obtained from the camera and IMUs, and 82% fewer errors than when only the camera was used. As the figure shows, the estimated track of the Motion Estimation AI is almost perfectly aligned with the actual track.

Application of Motion Estimation AI is not confined to the road. The technology can also estimate the motion and much larger acceleration and deceleration of drones. This capability that advances the goal of realizing autonomous drones that are able to fly a predetermined route and respond to conditions around them at the same time.

Motion Prediction AI estimates the movements of other vehicles on the road in a wide range of traffic conditions. Conventionally, realizing highly accurate prediction of future movement requires predictive AI models for road configurations, including the number of lanes and road curvature. The large number of models needed to accommodate the many different possible configurations make it difficult to apply to a wide range of real-world conditions.

Toshiba’s Motion Prediction AI uses a deep neural network to input generalized geometric features, including road curvature, and realizes predictive AI that can accommodate a wide range of road configurations. There are two stages in system operation: first the AI predicts the movement of other vehicles in each lane independently of road shape, and then predicts which lane each individual vehicle will select, enabling highly accurate prediction for many different road configurations. Tested against a public domain dataset, the Motion Prediction AI achieved the world’s highest accuracy^{(Note 6)}, reducing the error in predicting the future position of other vehicle in four seconds’ time by more than 40%.