Can a table-tennis robot serve up future of human-machine collaboration?

In the popular imagination, humans and machines have not always gotten along. But what was once speculative fiction is becoming a present-day reality. Roboticists are examining the challenges of human-robot interaction and which adaptations will be necessary as artificial intelligence (AI) dramatically changes how people work and live. This exploration could be painful, or it could be fun.

Japanese company OMRON, a sensing and control instrument manufacturer serving industries such as factory automation and health care, chooses fun—but with a deeper purpose. In 2013, as the company celebrated its 80th anniversary at a private exhibition in China, it unveiled the table tennis–playing robot FORPHEUS, a bellwether for human-machine collaboration.

OMRON uses industrial automation to make equipment for Japan and 120 countries globally. “OMRON’s core technologies are ‘Sensing & Control +THINK,’” says Masamune Nakayama, the development leader for FORPHEUS at OMRON’s Technology and Intellectual Property Headquarters. “We applied these technologies to create a robot that could play table tennis, a popular sport globally and especially in China. In this way, we could show our vision for the future relationship between human and machine.”

According to Nakayama, “Sensing” refers to the robot’s ability to detect the position and movement of the ball and the human player; “Control” uses control technologies to facilitate the accurate movement of the robot. And “+THINK” uses data from human thought processes. “Some describe table tennis as ‘playing chess while running a 100-meter dash,’” Nakayama says. “We wanted to know what top players and coaches are thinking during a match and incorporate that information using ‘+THINK’ technology.”

Initially, OMRON’s main challenges were developing the base technologies required for a table-tennis robot to see the ball, predict where it is going, and swing the paddle in response. Next, the team developed methods to gauge a player’s position and the ball’s motion, predict the ball’s path, and determine optimal contact for returning the ball—all at a high level of precision. OMRON followed up with a second-generation robot that displayed the information it gathered to the player, making it easier to continue a rally. In 2016, Guinness World Records recognized it as the “first robot table-tennis tutor.”

Nakayama says the turning point for further development was introducing AI. “In our initial work, we focused on creating a robot with robust table-tennis skills,” he says. “But as we entered the third generation, we developed ‘the harmony between human and machine for bringing out human capabilities and possibilities.’ We began creating technologies so that the robot could see the ball and the player’s movements, as well, and then match that player’s level.”

Another development was in the robot’s mechanics, featured in the fifth-generation model that debuted at the 2019 CES trade show. “By improving the robot’s software, we were able to significantly improve its table tennis–playing skills,” Nakayama says. “However, the parallel link construction of the arm unit has a limited range of motion. No matter how smart our algorithms are, in certain cases, the robot simply cannot reach the ball. Facing this problem, we incorporated a mechanism that corresponds to a human’s elbow, which dramatically improved the robot’s ability to return balls.”

Soon this highly polished FORPHEUS robot became popular on video-sharing platforms, and at exhibitions, people vied for a chance to put their table-tennis skills to the test. As the robot’s skills continued to improve, the team faced a new challenge. “The racket is held at the end of a carbon shaft attached to the parallel link arm,” Nakayama says. “But after a full day of rallies, the shaft begins to wear down, which ultimately leads to it breaking.”

To resolve these issues, OMRON began developing a new arm with a high-strength shaft that’s 10 centimeters longer but still weighs the same. Collaborating with Hisashi Kataoka of Kyoso Technology Co. led to a breakthrough. “We designed the latest, sixth-generation arm,” Kataoka says. “In doing so, we were able to reduce the weight of the arm’s tip quite a bit. We had exhausted the usual approaches toward lightening the components; with conventional methods, it would be difficult to further lengthen the arm or change its shape without increasing the weight.”

This joint project began in January 2020, around the same time that Kyoso Technology was considering providing generative design solutions as a service, which presented an opportunity for both companies. “We had been working on FORPHEUS from the mindset of open innovation, forming new relationships with universities and other companies to develop technologies through joint research,” Nakayama says.

Kataoka says that to develop this arm, Kyoso Technology worked with Mitsuo Nakatani at MAKErs SENSE and companies such as Applied Technology, Nisso Kogyo, and EOS Japan to determine the direction for the end product. They used Autodesk Fusion 360 for modeling and generative design, Inventor Nastran for simulations, and Netfabb for preprocessing 3D printing and additive manufacturing data.

“When we saw the actual result of generative design, it was surprising, to say the least,” Kataoka says. “The shape I saw was like nothing I had seen or that my internal logic considered a ‘good’ design. At first, I had a strong negative reaction to it. But I couldn’t deny it based solely on my own experience—that would make it impossible for us to approach new technologies.”

Kataoka says that when determining how to lighten a part, current design methods and engineering practices are limited. “That is why it was so startling how easily generative design surpassed my expectations,” he says. “Additive manufacturing has not really taken off yet in Japan, so I think our work—challenging ourselves to use techniques like a titanium lattice structure—will place us ahead of the market. I think our approach of working with highly specialized companies was a really important factor in our success.”

Nakayama was equally impressed. “I had been looking at on-screen simulations that used generative design, but when I hold the actual part in my hand, the reality of how incredible the design is really hit me,” he says. “At first, I was unsure such a delicate structure could hold up in actual use, but once we mounted it to the robot, there were no problems whatsoever. One of my bosses even commented that it was like a piece of art.”

Nakayama studied mechanical engineering and materials processing at university, so he understands how the organic structure of generative design could create stronger, more novel designs. “But the technology still seemed like a far-off dream to me,” he says. “Now I have this real part in front of me that I’ve been able to use in my work—the robot can send back volleys like never before. I feel like I’m witnessing the progress of technology, and the world we live in is getting more and more interesting.”

The new arm was fabricated using a titanium alloy in the EOS M400-4 metal additive manufacturing system, with support from the EOS Corporation. The arm has been mounted on the robot for testing, and after the robot’s parameters are fine-tuned, the part may be used in future exhibitions.

The project also advances the team’s broader AI-related research. In December 2019, a joint research project with Square Enix was announced for developing personalized, motivational coaching by combining OMRON’s human emotion- and ability-reading AI with Square Enix’s AI used in its electronic games for eliciting emotional responses with the way the game unfolds. OMRON is also looking to apply its technology across industries to realize the harmony between human and machine. For example, a machine at a factory could sense the skill level and expressions of a worker to provide additional assistance. Such a solution and others like it could promote the benefits of AI.

“In the future, when additive manufacturing and generative design have become commonplace, I think robots will evolve from shapeless metal boxes to become more organic and lifelike in appearance,” Nakayama says. “To me, it seems that a robot’s appearance is very important if you want people to feel at ease around it.”