Experienced manufacturing site personnel are aging, and companies face the challenge of passing on their techniques and expertise to future generations as soon as possible. One way of doing this is by documenting operational expertise. While hopes have been high for this approach, creating documentation places a heavy work burden on personnel who aren't document creation experts, and the documentation process seldom goes smoothly. We'd like to introduce AR Package, which makes it easy to immediately start creating documentation using augmented reality (AR) . We'd also like to present some of its usage cases.


Leveraging AR to Solve the Problem of Skill Transmission


In the manufacturing industry, attention is being turned to measures which use digital technologies such as IoT and AI to improve worksite operation efficiency. These measures are part of companies' digital transformation (DX) measures, but they are no easy matter. They require site operations to be clarified and standardized, and for systems to be integrated, taking a bird's eye view of all company systems. The observations and judgments of experienced technical personnel who are well-versed in operations are often unconscious, based on implicit knowledge that has been developed through their years of experience. This makes it difficult to visualize their know-how in the form of text and numbers.

Furthermore, as these skilled experts grow older and the population continues to shrink, companies are facing shortages of new personnel to whom these skills can be passed on. In the past, experts passed on their skills through their day-to-day work, gradually teaching junior colleagues the operational know-how and intuition they had developed based on worksite decision-making standards and personal experience. However, it has now become difficult to take the time to pass these on to the next generation of workers.

To address this issue, in recent years, many companies have taken to organizing the know-how of these experts and codifying them (knowledge creation). Expectations have grown for the documentation of operational expertise to make it possible to achieve uniform operation quality, regardless of who is performing the work. However, creating easy-to-understand documentation, from scratch, that covers the extensive, complex operations knowledge that has previously been conveyed orally, on-site, is no easy matter. It places a tremendous burden on workers who also have to handle their usual business duties, and the process of documentation has not gone as smoothly as hoped.

One of the promising technologies people hope will help eliminate this document creation hurdle is AR.


The Growing Use of AR Technology


AR overlays virtual information on top of the real world. It virtually augments the world in front of us. Recently, it is being used more widely, not only in the entertainment and advertising worlds, but also in industrial fields.

AR technology can be used in a wide range of applications and areas. For example, when performing device maintenance work in factories, AR technology can be used to view actual control panels and overlap virtual information simultaneously so that work can be performed smoothly. In warehouse picking work, in which selected articles are gathered, AR can be used to display the routes to article locations by overlaying them on the real world. This has the potential to cut the number of work mistakes and reduce movement times, increasing work productivity.

Using AR to display three-dimensional content in real space can also help provide a sense of how large something would feel and how it would look when put in position.


Making It Easy for Anyone to Create AR Content


Toshiba Digital Solutions supplies AR Package, a worksite operation digitalization solution that uses AR technology. AR technology can help improve worksite operation efficiency and quality, such as by shortening on-site work times, reducing the number of operation errors, creating and storing work history data, and assisting experienced personnel to transfer their skills to junior colleagues. It is an AR solution that is easy to use, even in industrial fields, led by the concept of "AR that Can Be Created by Anyone, Anywhere, Right Away" (Fig. 1).

Generally speaking, superimposing virtual information over the real world using technologies such as AR requires development work. Engineers and designers with expert knowledge are often believed to be essential to performing this development work, which is assumed to take a great deal of time and money.

However, this approach does not make it possible to flexibly document massive, complex operations, nor to flexibly maintain this documentation on an ongoing basis. AR Package has made the creation of AR content intuitive, simple, and easy to understand.

Its AR content creation tools are tool is simple and easy to use. Simply drag and drop text, images, videos, and other content placing them as desired, as if you were making a slide for a presentation. In addition to standard AR objects that are provided by default by AR Package, such as arrows, users can also use original AR objects that they have prepared themselves, providing greater flexibility when creating content.

Ordinary AR requires markers such as QR codes (black-and-white rectangular markers) to be placed to position where AR content is to be displayed. These markers must therefore be prepared, affixed to surfaces where AR content is to be displayed, and maintained in appropriate condition. This takes time and effort, and makes it difficult to use AR in locations where markers cannot be affixed.

AR Package eliminates the need for special markers. Users can simply take a photo of the site where they wish to display AR content and designate whichever points on the photograph they wish to be AR markers. Then, by holding the smart device over the actual site, the camera will recognize the points designated as AR markers so that AR contents will be displayed.

AR Package consists of Story Designer, which is used to create and edit AR content, and Story Viewer, which is used to display AR content. Both can be used with ordinary computers and smart devices. They are also native apps local applications, so there's no need to set up complex systems or networks. Content can be created on a computer in which Story Designer has been installed and then transferred directly to a physically connected smart device to be used on-site (Fig. 2).

It also functions as a standalone application, making it tremendously effective when used in industrial fields. For example, it can be used in locations where network connections are hard to establish, such as underground, where radio signals don't reach, or in places with strong electromagnetic fields. Of course, it can also flexibly meet various other customer needs: it can support the sharing of data via the cloud, and it can be integrated with legacy systems.


AR Package Use Cases


We tried out AR Package in a department responsible for maintaining equipment in a Toshiba Group factory. Within the department, it was normal for know-how to be conveyed through word of mouth. The department was struggling to create manuals and codified knowledge that covered its complex operations, and it found itself unable to devote sufficient time to training the next generation of department personnel.

Veteran personnel tried using AR Package to create AR content, and then inexperienced new personnel used the AR content to perform actual equipment maintenance. They found that the AR content enabled them to perform the work confidently, without requiring assistance. Even personnel who were creating AR content for the first time were happy to find that they intuitively understood the tools by using them and were able to create an AR operation manual of the entire series of complex operations involved in the work. The manual was visually easy to understand

Through the efforts by personnel to document operation manuals, we confirmed in an actual worksite environment that AR Package is an effective solution for both worker training and operation standardization.

Now let's look at an example from outside the Toshiba Group of how customers are using AR Package to create AR content and pass on skills. One company uses a variety of testing equipment whose operation requires both experience and skill. Previously, experienced operators passed on their skills by training junior operators one-on-one, using printed manuals with separate operating procedures for each and every device.

However, there was seldom time to provide training due to the everyday workload. Furthermore, when they did perform training, passing on skills took a tremendous amount of time and effort, because the skills to be learned were so advanced and involved making judgments and operating equipment while making inferences regarding the functioning of highly specialized equipment.

When they used AR Package to create an operation manual that leveraged the potential of AR, they found that even people operating devices for the first time were able to carry out the defined operations without the need of any assistance from experts. They were also very happy with the intuitiveness and ease of use, because instructions and other data were displayed immediately when a smart device was pointed at a target object with designated AR markers. They have also begun trying new techniques to create even easier to understand operation manuals, such as including videos.

The AR content the customer created themselves is improving the efficiency of skill succession and shortening training times, so hopes are high for using this technology with other test devices and for new employee training, as well.

There are also several global companies that would like to use AR Package as a digital manual creation tool for their overseas locations. Many manufacturers with overseas locations translate their Japanese manuals into the local language for use in their overseas plants. This translation takes both time and money.

AR content conveys things visually, which makes it possible to reduce the amount of text needed for explanations and to create manuals that are largely unaffected by language differences. This reduces the amount of translation that is needed, and can in some cases eliminate the need for translation altogether. Furthermore, videos, images, and figures can be used to clearly convey exact operation points and movements, helping reduce the amount of direct training necessary at sites. Not only does AR Package reduce costs and increase efficiency, but it also has the potential to accelerate business operations.


The Promise of the Future Evolution of AR Package


AR Package is just part of our Digital Manufacturing Solutions. It can, of course, be used as a standalone solution, but it can also be combined with technologies and solutions such as IoT data and knowledge databases to further refine worksite operations.

What's more, we also plan to offer optimized solutions to an even wider range of customers, such as by providing AR content creation support services in collaboration with companies that specialize in document creation. We aim to make further refinements and improvements to AR Package's AR usage, focusing on ease of use in customers' usage scenarios. For example, we plan to offer support for wearable devices.

We also plan to further advance it as an open platform by enhancing integration with other operation systems, creating an environment that is even easier for customers and partners to use.

AR Package gives users the power to decide what kind of content to create and how they want to use it. No matter your business or industry, if you're interested in creating or using digital content, including AR, we at Toshiba Digital Solutions are waiting to hear from you.

  • AR Package is not currently available for purchase outside Japan.
  • The corporate names, organization names, job titles and other names and titles appearing in this article are those as of September 2021.

>> Related information

Related articles