We live in a world of dramatic change. Companies and societies are trying to tackle increasingly complex issues, and they are turning their attention to quantum technologies. Although it is expected to take some time before we see full-fledged quantum computers that utilize quantum phenomena, Toshiba is already working on quantum-inspired optimization technologies. We are applying our own in-house combinatorial optimization technologies for producing optimal results from massive numbers of options, providing them in the form of the quantum-inspired "SQBM+" optimization solution. This solution uses existing computers to produce highly accurate approximate solutions in short amounts of time. In this running feature, we will explain quantum-inspired optimization technologies.

Parts 1 through 4 presented an overview of quantum-inspired optimization technologies, outlining their objectives and potential applications across various domains. Parts 5 and 6 focus on the practical implementation of these technologies, particularly in systems that require rapid and reliable decision-making—commonly referred to as real-time or mission-critical systems. In Part 5, we introduced Field-Programmable Gate Arrays (FPGAs), essential devices for building real-time systems. In Part 6, we explain how FPGA-implemented SQBM+ is applied to real-time systems, using examples from the automotive and financial sectors.

When you hear "real-time system," what comes to mind?

Driver assistance systems, automated trading systems in finance, air traffic control systems, facial recognition systems in surveillance cameras, and medical systems such as ventilators are all representative examples of real-time systems. Academically, a real-time system is defined as "a system subject to time constraints, in which processing must be completed within a specified deadline" (Fig. 1). This definition can also be interpreted as referring to systems that must guarantee response times shorter than a specified time constraint. For example, the surrounding environment of a moving vehicle and the best bid and ask prices in the stock market are constantly changing. Real-time systems can be understood as systems designed to respond to or control constantly changing environments and information.

The term "real-time system" also evokes the image of an extremely high-speed system. However, its academic definition does not include any speed-related requirements, such as "the response time must be less than N seconds." In reality, the response times required of real-time systems (their time constraints) can vary significantly depending on the specific application.

In the engineering field, there are systems known as "high-speed real-time systems." This typically refers to systems that guarantee response times of several tens of milliseconds or less. For example, typical video cameras operate at a frame rate of 30 frames per second (FPS). This corresponds to a frame rate of 30 frames per second, meaning one frame is captured approximately every 0.033 seconds (33 milliseconds). If each captured image is analyzed, all processing—including the analysis—must be completed within 33 milliseconds. In another example, on the Tokyo Stock Exchange, the best bid and ask prices for many stocks typically update at intervals of several tens of milliseconds. However, a phenomenon known as a "rush" frequently occurs, during which prices fluctuate at sub-millisecond intervals. The rush is the result of chain reactions of buying and selling triggered by specific news, trading actions, or similar events involving various traders. Such trading behavior involves automated trading systems with response times of less than one millisecond. In comparison, human response times to information received through the eyes or ears are said to be around 300 milliseconds (or around 200 milliseconds in the case of athletes). Even spinal reflexes, which bypass the brain, are reported to occur in just under 100 milliseconds. Thus, high-speed real-time systems can be considered systems that achieve response times far faster than those of humans.

Furthermore, real-time systems not only imply extremely high speed, but also evoke the concept of edge systems. The term "edge" (in systems or computing) is commonly used in contrast to "cloud" computing, highlighting the difference in where data processing occurs. Typical internet connections used to access cloud services have a communication delay (latency) on the order of several tens of milliseconds. In applications where low latency is critical—such as web conferencing or online gaming—a latency of 15 milliseconds or less is considered ideal. However, this level of performance is often difficult to achieve in practice.

Moreover, due to its design, the modern internet cannot guarantee upper bounds on latency—meaning worst-case delays are unpredictable. As a result, high-speed real-time systems are often implemented as edge systems that can make autonomous decisions without accessing the cloud.

This article focuses on real-time systems designed for combinatorial optimization. We have often found that many engineers and researchers, especially those specializing in real-time systems and mathematical optimization, are surprised by the idea that combinatorial optimization can be performed by a real-time system. This is because combinatorial optimization problems involve computations that are difficult to perform in realistic time. Even with significant advancements in real-time system capabilities, meeting the speed requirements for these calculations remains challenging. As a result, such real-time computations have been virtually unprecedented.

Due to strict timing constraints, many conventional high-speed real-time systems are designed to execute predetermined responses based on relatively simple condition checks (trigger logic). Such systems, which monitor the timing for executing a single predetermined response—typically with only one available option—are often referred to as "watchdog systems." This system can be configured with multiple pairs of trigger conditions and response actions, enabling it to perform a variety of responses. However, as the number of possible response actions increases, a correspondingly larger number of system configurations may be required.

In contrast, with high-speed combinatorial optimization technologies that can be embedded in a single edge system, it becomes possible to realize a real-time system that selects the optimal response from a vast array of options in accordance with continuously changing external conditions, and immediately executes the selected action. The system that enables this functionality—referred to as instantaneous adaptability—will hereafter be called the “Real-Time Combinatorial Optimization System” (Fig. 2, left side). Toshiba’s quantum-inspired optimization technologies are ideally suited to meet the demanding requirements of realizing such real-time adaptive systems.

Let us examine the key requirements of quantum-inspired optimization technologies that are essential for developing real-time combinatorial optimization systems (Fig. 2, right side).

First, the solution time for combinatorial optimization problems must be both short and deterministic. As we covered in Part 5, FPGA-implemented SQBM+ achieves shorter solution times than conventional methods. However, when applied to real-time systems, it is essential to guarantee a maximum (worst-case) solution time. Ideally, solution times should be fixed and free from deviation. Toshiba's Simulated Bifurcation Algorithm (SB Algorithm) simulates time-evolution processes*, including the bifurcation of a many-body oscillator system (or many-body spin system)  with interactions corresponding to the problem to be solved.　The algorithm is designed to efficiently search for high-quality solutions to such problems. (Fig. 2, graph at top right). The number of time-evolution steps used to simulate the bifurcation process until completion can be predetermined according to the specific application. Therefore, during the operational phase, the time required for a single solution does not vary.

Second, the combinatorial optimization module can be integrated into the edge system with the other modules. This is one of the key requirements for minimizing the overall system latency. The FPGA implementation of SQBM+ is a dedicated hardware circuit that performs ultra-parallel processing of the Simulated Bifurcation Algorithm (as explained in Section 5), making it well-suited for integration into edge systems. This represents a key distinction from quantum computers and other quantum-inspired technologies that require specialized auxiliary equipment such as cryogenic coolers or lasers.

Third, the combinatorial optimization module can be equipped with interfaces tailored to specific applications. Similar to the second point, this is also a requirement for reducing overall system latency. As illustrated in Figure 1, a real-time system is not a self-contained system. Rather, it acquires information from the external environment and responds to it with concrete actions. Therefore, an interface for input and output with the external environment must always be present.

In real-time systems, the bottleneck is often not the time required for data processing itself, but rather the latency in data transmission between interfaces and modules, or between modules themselves. To minimize communication latency, the combinatorial optimization module must be equipped with interfaces tailored to the specific application. In the case of FPGA-implemented SQBM+, dedicated interface circuits for each system can be independently designed and directly connected to the specialized hardware circuits that execute the SB algorithm.

*A time-evolution process involving the bifurcation of a many-body oscillator system (or many-body spin system)
Spin refers to an internal degree of freedom of a particle in quantum mechanics (angular momentum). The spin state has two possible orientations, ↑ or ↓, which can be mapped to binary values of -1 and 1. A model consisting of multiple spins (N spins), where each spin interacts with others, is referred to as a many-body spin system. The interaction coefficients between spins can be configured so that the solution to a combinatorial optimization problem corresponds to the minimum energy state of a many-body spin system (i.e., mapping the problem onto a many-body spin model). The SB algorithm developed by Toshiba represents spins as oscillators ranging from -1 to 1 and simulates the time-evolution process of a many-body oscillator system to identify its minimum energy state, which corresponds to the solution of the optimization problem. Each oscillator starts near zero and gradually diverges toward either -1 or 1 (bifurcation). Each oscillator influences the others, so the behavior of multiple oscillators becomes extremely complex (see the top-right graph in Figure 2).

This section presents overviews of two real-time combinatorial optimization systems—one in the automotive field and the other in finance—that have been published in academic papers.

We begin with an example from the automotive field. Autonomous vehicle control systems cyclically execute a sequence of processes, including sensing, recognition planning, and control. Each sequence is performed more than 10 times per second, which means the system's overall latency must be 100 milliseconds or less. The FPGA implementation of SQBM+ satisfied this requirement and enhanced the functionality of a multi-object tracking module (Fig. 3).

Object tracking is performed after detecting vehicles and pedestrians in the current video frame. An essential part of the tracking process is matching the objects detected in the current video frame (detected objects) with those currently being tracked by the system (tracked objects). Under normal conditions, a one-to-one correspondence between detected objects and tracked objects is assumed. However, when objects get crossed or one object overtakes another—as in frame (k–1) of Fig. 3— it may be more appropriate to match a single detected object with two tracked objects.

In addition to standard one-to-one correspondences, one-to-many matches may also occur, as in this case. Selecting the most plausible correspondence from these possibilities poses a complex combinatorial optimization problem. In articles ^[1][2], we demonstrated that by implementing SQBM+ on a Vehicle-mountable FPGA board, it is possible to perform high-functionality object tracking—enabled uniquely by SQBM+'s matching algorithm—at a frequency exceeding 20 times per second.

Next, we turn to an example from the financial sector. At the Tokyo Stock Exchange, a system was demonstrated that continuously monitors changes in the best quotations of multiple stocks and, based on combinatorial optimization, determines the timing of buy and sell orders as well as which stocks to trade (Fig. 4). Here, we consider the discrete portfolio optimization problem, which involves selecting a smaller set of stocks from a large number of tradable stocks by maximizing an evaluation metric designed to maximize expected return and minimize expected risk. When mutual dependencies—such as trade-offs—exist between stocks, the problem becomes a complex combinatorial optimization challenge due to the second-order interactions.

Article ^[3] reports on a high-speed trading system characterized by its ability to solve second-order discrete portfolio optimization problems each time the best quotation changes. The system demonstrated a total latency of 164 microseconds (0.164 milliseconds) and successfully executed orders at the intended price and quantity. The system performs combinatorial optimization over five million times per day. It was also reported that the system operated continuously for 42 days (252 hours) without any observed malfunctions.

In this article, we introduced a high-speed real-time system that makes rational decisions based on combinatorial optimization, using examples from automotive and financial applications. We, Toshiba, are committed to applying SQBM+ not only to enhance the functionality of existing real-time systems, but also to create new value across a wide range of fields. We encourage you to keep a close eye on SQBM+ as we continue to unlock its future potential.

Reference materials
[1] K. Oya, H. Fujimoto, Y. Hamakawa, M. Yamasaki, and K. Tatsumura, "Proposal and Prototyping of Automotive Computing Platform with Quantum inspired Processing Unit," Transactions of Society of Automotive Engineers of Japan, Vol. 54, No. 6, pp. 1216-1221 (2023). https://doi.org/10.11351/jsaeronbun.54.1216
[2] K. Tatsumura, Y. Hamakawa, M. Yamasaki, K. Oya, H. Fujimoto, “Enhancing In-vehicle Multiple Object Tracking Systems with Embeddable Ising Machines,” arXiv:2410.14093 (2024). https://doi.org/10.48550/arXiv.2410.14093
[3] K. Tatsumura, R. Hidaka, J. Nakayama, T. Kashimata, and M. Yamasaki, “Real-time Trading System based on Selections of Potentially Profitable, Uncorrelated, and Balanced Stocks by NP-hard Combinatorial Optimization,” IEEE Access 11, pp. 120023 - 120033 (2023). https://doi.org/10.1109/ACCESS.2023.3326816

• Quantum-Inspired Optimization Solutions “SQBM+”