The hitter, a Kuka KR10 robot, stares down Lin Zhang, the pitcher.”
First published: September 28, 2017
URL to live article: https://polytechnic.purdue.edu/newsroom/baseball-cartoon-inspires-research-robots-smart-enough-be-human-assistants
“Touch,” a manga (cartoon) which was adapted into a popular animated television series in Japan, is the story of twin brothers who play high school baseball and the neighbor girl who is their close friend. Manga are popular in Japan with people of all ages, and many are translated into other languages.
While studying automation control science at the Harbin Institute of Technology in northeast China, Lin Zhang, a postdoctoral research associate in the School of Engineering Technology, became a fan of the show. The twins’ love of baseball inspired Zhang’s current research into how robots perceive human motion.
“In baseball, the pitcher is going to throw the ball to a certain area, trying to deliver a pitch either inside or outside the strike zone,” Zhang said. “The hitter must decide whether or not to swing the bat.”
The more skilled the pitcher, the more difficult a task for the hitter. Now imagine trying to teach a robot to be the hitter.
“You have to give the robot tons of information like the speed and direction of the ball and the velocity and angles of the bat,” said Zhang. “But it can only accomplish the task of hitting the ball if the information you give it is very specific and very accurate to a tiny point.”
In other words, no matter how challenging it is for a human player to connect bat to ball, it’s currently a lot more difficult for a robot to do the same thing.
For his doctoral research at New Mexico State University, Zhang’s research subjects were more conventional.
“We used a motion capture system to record the gait pattern of older people,” he said. “With a bunch of infrared cameras, we built up models of the human body in motion. Through those models, we extracted data to identify the features of a person’s manner of walking which might predict a fall.”
Lin Zhang
Zhang used machine learning techniques to do that predictive work. The process of making computers more able to simulate the functionality of the human brain is called deep learning, he said, and the techniques are helping make robots more intelligent.
“Human brains process information with layers and layers of neurons,” said Zhang. “Basically, deep learning stacks multiple layers of artificial neurons, creating artificial neural networks which can help machines or robots make decisions. In effect, it makes them smarter.”
The computational capabilities of graphic processing units (GPUs) have been growing quickly in recent years, so computer scientists are now using GPUs for deep learning research.
“The technology is amazing. Before deep learning, people programmed robots with instructions, and the robots just followed them very strictly,” Zhang said. “I’m trying to make the interaction between humans and robots more fluid. There are growing needs for robots to be more flexible, able not just to follow specific instructions but to observe, perceive the environment, and react according to their observations and perception.”
That’s the general idea, Zhang said. He’s using the baseball scenario as a means to that end. Zhang is the pitcher, and he wants a Kuka KR10 robot to be the batter. “It’s precise and fast,” he said, and it has the range of motion and speed he needs for this research. “If the robot could be programmed with the same abilities as a human, to observe how the pitcher throws the ball, it could determine if and how to hit the ball itself.”
The first step is to provide the robot with eyes. For that, Zhang is currently working to program a Microsoft Kinect — the same motion-sensing input device you might buy for an Xbox video game console.
“It has the ability to record videos, measure the environment, and record the distance of objects from its camera,” said Zhang. “Using the data it provides, I want to teach the robot to learn the pitcher’s throwing patterns.” He will connect the Kinect to the Kuka robot in the next phase of his research.
But the real aim of Zhang’s research is not to create robots smart enough to play baseball.
“Since my research used to involve studying how older people walk, it made me want robots to be more intelligent so they could assist people,” said Zhang. “Robots help humans in many fields, but they still aren’t very smart. People suffer through diseases and other problems. They need better access to nursing. If robots can better understand human behavior, they might become better assistants.”
