Vol.23 Communication AI for knowledge processing with speech and image data The business innovation produced by collaboration between people and AI


#02 Dialog and knowledge platform of RECAIUS Gathering findings from the field and using them as knowledge Takeshi Nagai Chief Specialist
Business & Service Development Dept. RECAIUS Business Development Div. Toshiba Digital Solutions Corporation, Hideki Morikane Group Manager Business Group Business & Service Development Dept. RECAIUS Business Development Div. Toshiba Digital Solutions Corporation

Toshiba's communication AI "RECAIUS", which is responsible for person-related digitization, is a service that provides support for communication and knowledge usage between people and things, and between people and people. In order to solve the real, serious issues faced by customers, RECAIUS not only requires individual Artificial Intelligence (AI) engines such as speech recognition and synthesis, dialog, and knowledge retrieval engines, but also a system for providing total support for communication and knowledge usage and updating, defined on an individual usage purpose and field basis.

With RECAIUS, we have developed a platform for using speech, images, and knowledge to provide greater customer satisfaction for customer support services and greater field worker productivity. We believe the platform will enable strategic collaboration with system integrators and service providers, making it possible to provide solutions tailored to customers' various objectives in various industrial fields. Let's look at an overview of the RECAIUS service platform, in which agent and knowledge functions are mutually linked, and some use cases.

Person-related digitization
Conveying implicit knowledge as codified knowledge

Today Japan, especially the industrial world, is facing a crisis. The number of experts is falling and coming aging society are producing shortages in workforces, so the implicit knowledge which has in the past been passed from person to person is in danger of disappearing. This implicit knowledge is the key for customer satisfaction, productivity, safety, and security for the real, serious world. The knowledge, put into words, would take the form "when this happens, you should do this," but using this knowledge to properly share with others is not an easy feat. This is because it is difficult for someone, without sharing the same concrete experiences as an expert with a wealth of experience, to understand these "when this happens" phenomena in the same way as the expert. Even for expressions which seem at first to be clear-cut, like "When this happens three times" or "When this turns red," aligning decision-making sensibilities can be difficult, such as whether to count something as having happened once, or deciding whether a certain color should be considered "red".

The difficulty in obtaining knowledge primarily in word form was already pointed out during the AI boom of over three decades ago. However, the advent of the powerful data-driven machine learning methods later developed, such as deep learning, which is currently evolving at breakneck speed, showed the possibility of overcoming these knowledge transfer limitations. It has become possible to use large amounts of recognition and identification results to extract and learn feature quantities effective for decision-making (expression learning), and to create an extremely accurate decision-making mechanism (inference model).

These new AI methods can be used to enable systems to inherit the implicit knowledge of experts, without expressing it in words, provided a large amount of input and output teaching data indicating what the experts saw and what decisions they made. In situations in which defect judgements are made visually, it has become comparatively simple to use images as input and have systems learn pass/fail criteria.

However, the digitization (codification) of experts' implicit knowledge still presents difficulties. For example, it is difficult to make decisions regarding (1) whether correct decisions were made (2) in situations in which the data used to make decisions is difficult to identify or measure, (3) when the knowledge concerns a phenomenon that happens extremely rarely (there is no concrete data), or (4) when the decision-making involves specific timing or decision-making based on a number of smaller decisions. The difficulty in actually transferring this implicit knowledge from person to person is believed to be achievable through the use of specific experiences, creativity, and communication abilities.

Click here to move to the top of this page.

Communication and field experience that brings knowledge to business

Takeshi Nagai

Overcoming the difficulties involved in transferring knowledge from people to systems – that is, digitization – may require the same processes used to pass on implicit knowledge from person to person. We want to apply advanced AI in environments in which it is possible to build up capabilities by first making decisions based on simple criteria, and then in cases where decision-making is difficult, or incorrect decisions are made, gaining insights from experts, applying similar experiences to new situations, and creating records when experiencing whether decision-making was successful or not.

Until now, this "when this happens, you should do this" knowledge has been put into words and recorded in documents such as manuals, to be referred to and utilized consciously by those who need it. The information system model was one of storing knowledge in a knowledge database and providing search functions to users. However, one of the problems with this model is that it is only useful when people actively seek out knowledge. Another problem is that it is possible for necessary knowledge to go unregistered, without anyone noticing it.

Potential methods for preventing this are (1) actively informing users that this knowledge is available, (2) when useful knowledge is missing, drawing attention to lack of the knowledge to those who could supplement the knowledge, and (3) making it easy to supplement knowledge through dialog-form interaction. Truly leveraging digitized knowledge requires a method (agent) for active communication (dialog) with field personnel. Opportunities for learning the results of the actual use of knowledge by field personnel and sharing experiences such as supplementing knowledge by learning it from others are essential for learning by both people and AI.

Click here to move to the top of this page.

Agents and knowledge
Dialog and knowledge platform of RECAIUS

In tackling the social challenge of passing on knowledge in the real, serious world, RECAIUS doesn't merely supply individual AI engines such as speech recognition and synthesis engines, dialog engines, and knowledge retrieval engines. Instead, it aims to provide full-fledged knowledge utilization services that provide value to those in the field by fusing communication support and knowledge management.

To do so, we have been developing platforms that bridge the gaps between field operation systems and AI engines. These are the "agent platform" and "knowledge platform" (Fig. 1).

Fig.1 Communication AI project overview

The knowledge platform consists of a collection of "when this happens, you should do this" knowledge. On its own, it offers the following functions: (1) a function for determining whether specific conditions satisfy the "when this happens" condition of data required for some knowledge, (2) a function for linking this to "you should do this" information, and (3) a function that supports the updating of this linkage itself. These evaluation and linking functions can be combined in complex forms and utilized. The system makes it possible to search for relationships in a diverse set of data, such as linked signal data and words. For example, it can be used to search for reports about past situations which have been determined to be related based on sensor data.

The goal of the agent platform, on the other hand, is to make it possible to rapidly create a dialog service with which people can easily obtain information from an information database through dialog interaction, and which actively asks users for lacking information. In the past, this required dialog scenarios to be created for individual purposes and fields, and because multiple stored dialog scenarios could not be linked and used as-is, required entire dialog scenarios to be rebuilt from scratch. In this platform, however, typical dialog scenarios for retrieving knowledge from a database, or adding knowledge to it, are modularized, and can be combined together.

We believe these two platforms, the agent platform and the knowledge platform, can be combined to promote the activation (both utilization and updating) of knowledge obtained from people, making true knowledge utilization possible (Fig. 2).

Fig.2 RECAIUS's two service platforms

Click here to move to the top of this page.

Field knowledge utilization has already begun

Hideki Morikane

Production sites such as factories can create systems in which data obtained from equipment-mounted sensors is used to detect changes in worksite conditions, while simultaneously providing worksite personnel with appropriate knowledge based on those conditions.

Jinya, a traditional hot spring inn located in Kanagawa Prefecture, uses RECAIUS in a system for sharing information between staff members. Instead of using transceivers, commonly used in the past, tablet devices connected via a mobile network, together with compact earphones, have made it possible for information – not only speech but also text created through speech recognition – to be shared with all staff members simultaneously. This system has prevented staff members from missing information and has taken information about what types of hospitality and care have been effective for which types of customers and converted into knowledge to be used by other staff members. Trial use has already begun for further leveraging the knowledge, such as identifying changes in workplace and customer conditions and automatically issuing instructions to staff using voice synthesis.

We will use the know-how we accrue through many test cases in our RECAIUS product development, creating effective AI which energize industries in the real, serious world. Our efforts will bring on an even wider world of AI services which reflect Japanese distinctive virtues, such as diligence, care, thoughtfulness, and hospitality.

* The corporate names, organization names, job titles and other names and titles appearing in this article are those as of October 2017.