Robot Soccer Offers a Peek Into AI’s Future

42 min read

Messi Vs. Ronaldo …or Mr. Roboto?

On a hot Central Texas afternoon in the middle of June, members of SparkCognition’s marketing and communications team were spectators during soccer practice at the University of Texas at Austin.

The players weren’t running up and down the field and they hadn’t broken a sweat — they weren’t even suited up. They were robots, and they were preparing for the year’s most prestigious international competition, the RoboCup.

That’s right, the RoboCup.

It’s not your typical soccer tournament. Hosted in Leipzig, Germany, the RoboCup is an “international scientific initiative with the goal to advance the state of the art of intelligent robots.”

The RoboCup competitions provide a channel for the dissemination and validation of innovative concepts and approaches for autonomous AI robots under challenging  athletic conditions. When established in 1997, the original mission was to field a sports team of robots capable of winning against the human soccer World Cup champions by 2050.

And while we may be far from that reality, the University of Texas at Austin RoboCup team is competitively advancing this mission, led by one of the top coaches in the league.

On any given day one will see autonomous robots roaming the hallways by Peter Stone’s office, and the Robotic Soccer Lab.

Dr. Stone, or on this particular afternoon, Coach Stone, is the David Bruton, Jr. Centennial professor of computer science at UT. Having received his Ph.D. in 1998 and his M.S. in 1995 from Carnegie Mellon University, both in Computer Science, Dr. Stone is a leading researcher and professor in the field of artificial intelligence.

His focus areas include planning, machine learning, multi-agent systems, robotics, and e-commerce. His long-term research goal is to create robust, autonomous agents that can learn to interact with other intelligent agents in a complex and dynamic environment.

As a lifelong soccer player with a passion for the sport — at one point in his career, almost making it to the major league — Coach Stone is pursuing his research goals by scoring athletic goals on the soccer field, and he is making major progress.

Making Robots Competitive

Since 2002, Coach Stone has led the UT Robotics Team to six RoboCup championships in the simulation challenge (computer software), and one in the standard platform challenge (physical robots).

As a champion competitor, Coach Stone leads a team of student researchers who share his interest for the sport and robotics.

When Cognitive Times caught up with Dr. Stone, he was leading a meeting of his RoboCup team, checking in on the state of their robots. This meeting was akin to a practice.

Coach Stone began with a status check of where his team members were with their code, and then had them “drill” their software.

The practice resembled that of young children first learning to play soccer. Some robots made their way to the ball while others wandered the field. Some fell down and others stood with their heads on a swivel, gauging where on the field they were in relation to the ball. Like parents of young children, the human team members followed the robots around the field, ensuring that they were on track and safeguarding them from hard falls.

All the while, Coach Stone questioned his team (the humans) on approach and reasoning. He guided them with solutions to improve their programs and encouraged them to press forward with urgency — the competition only a few weeks away.

The improvement over the last decade has been phenomenal, and if Moore’s Law holds water, we may see a robot beat a human at a mechanical sport sooner than anticipated.

As an athlete, I was surprised by how similar the RoboCup practice resembled a gameplan walk-through or film session. Perhaps this was the most interesting aspect of the practice — the human dimension.

Dr. Peter Stone was, for the hour, Coach Stone. And as a coach on the field, he guided his team with stern direction, vision, and encouragement. He has led robotic sports teams to championships before, and he will likely do so again.

A Post-Practice Interview

After Coach Stone ended practice, we caught up with Doctor Stone for an interview.

Cognitive Times Dr. Stone, what are your specific research areas and what are the goals of the lab that you run?

Peter Stone Our lab is called the Learning Agents Research Group. The unifying long term goal is trying to create fully autonomous agents. It could be robots, it could be software agents that can exist autonomously in the real world for extended periods of time.

That means dealing with lots of uncertainty with perception and with action. That’s the unifying theme.

The sub areas of artificial intelligence that I work on are known as reinforcement learning, which is learning about the effects of actions, and multi-agent systems — how do you get multiple autonomous entities, individuals, and programs to either cooperate or work against each other, or to just coexist such that they are all achieving their own goals and reasoning about each other’s goals.

I also use several test bed domains that involves robotics. So, the robot soccer team involves learning, multi-agent systems, and robotics.

We also have robots you may have seen wandering about the hallways; that’s our building-wide intelligent project. We have robots that can figure out where they are in the building and can interact with people.

The long term goal of that project is to get to the point where people walk into the building and expect to see robots and interact with them to either do things that are entertaining or useful in some way. The robots should exhibit a clear intelligence or knowledge of everything that’s going on in the building.

The long term goal of the robot soccer project is to have a team of human robots that can beat the best soccer team on a real soccer field by the year 2050.

The Next Ronaldo

Cognitive Times And how far away are we from creating that goal, from creating the next Ronaldo?

Peter Stone It turns out to that there are a lot of dimensions to that question. There is the physical hardware aspect of it. A robot that is as physically agile as an athletic person is a long ways off.

There have been some examples of robots that show impressive degrees of agility, but then to combine that with sort of long running autonomy and the ability to make decisions (the software aspect of it), we are still a long ways off.

You saw what is known as the standard platform league in RoboCup, where we all have the same hardware. That’s not by any stretch the state of the art of robots in the sense of agility.

Those are chosen with a particular price point in mind. It’s a nice, stable platform that allows us to push the limit of how fast they can move and also think about the strategic levels. There are enough teams from around the world who can have multiple of these robots so that we can have five versus five competitions.

We also participate in the 3D simulation league which is where there are eleven robots on each team. There, we can even lift the next level up and think at more of the strategic multi-agent level. There are different leagues in RoboCup as well, there are some with wheeled robots that moves much more quickly than these.

We have had people versus robot games against those robots every year since 2007 and it’s getting harder, but people are still much better than the robots.

Cognitive Times Dr. Stone, you have a bookcase full of trophies, a lot of accolades — you are winning a lot. What is the secret sauce, what’s the magic here?

Peter Stone I think the key is that over the years I have really deeply integrated my RoboCup team into PhD level research.

There are some people who do RoboCup on the side, but for many of my students there is actually something related to RoboCup that ends up playing a big role in their PhD thesis. They are motivated to come up with a new algorithm because not only is it going to help our team, but it is also pushing the frontier of research.

We have had a lot of research papers that have resulted from our contributions in RoboCup. I think that’s what makes it so that I can be doing this at a level of very deep involvement by top level graduate students — because it’s very closely tied to their research.

Cognitive Times I have read that your goal is to have complete robust autonomous agent that can learn and interact with other intelligent agents on a wide range of complex and dynamic tasks.

We saw some of that in the soccer lab and we have seen robots roaming the halls here. I know that you also do some work with autonomous vehicles. What can we expect to see, or what is your hope, for the future of robotics?

Peter Stone In terms of robots, we are trying to create the next generation of robots that people can interact with. This building is full of students who are trying to get the robots to be out and about among the students and doing things that students want to do.

There are other domains, I have a student who just finished work on autonomous bidding agents. These are agents that bid in market-based systems, where there are multiple other agents they need to cooperate or compete with in an economy.

There are a wide variety of domains that can serve as test beds for the study of robust autonomous agents. I basically try to give my students a lot of freedom in terms of their applications as long as there’s some tie to the main themes of the lab — reinforcement learning, multi-agent systems, and robotics.

Machine Strategizing

Cognitive Times You have talked about reinforcement learning. What about reasoning? Do you think that reasoning plays a role here? Is that an approach that your students are employing?

Peter Stone Yeah, we have a collaboration with Vladimir Lifschitz, who works in logic-based reasoning, formulization of action languages. We had a paper last year at the main AI conference on integrating common sense reasoning with dialogue systems on a mobile robot.

For example, if a robot heard a person at 9 a.m. speaking to the robot and saying “please bring me toffee,” with common sense reasoning and the right knowledge representation on the line, the robot can reason that at 9 am it’s more likely that the person is asking for coffee rather than toffee.

This sort of integration of reasoning and knowledge into the whole perception system can make a big difference.

Cognitive Times It’s clear that over the span of the RoboCup there have been tremendous improvements in robotics and machine learning. What do you see as the key challenges today that you face, that your students face? Are the challenges more algorithmic or mechanic?

Peter Stone They both go hand in hand. As the mechanical engineering improves, as the computational power improves, it gives us more capabilities and more software challenges to try to rise up and make better use of the hardware and technology.

I focus more on the software side. So we are not making the mechanical engineering improvements in my lab but we are certainly trying to keep abreast of them and always using the best new hardware, because it provides different capabilities and different software challenges.

Cognitive Times We talked about reinforcement learning and reasoning. In the practice arena, when the robot is active in the lab, one can see them thinking. They are turning their heads trying to find the ball, they are getting a sense of where they are on the field.

Could you explain in high-level terms how the robots’ decision-making process works when they are out there playing soccer?

Peter Stone People are very quick to anthropomorphize robots or any kind of object. What you see is not them thinking; you are seeing them acting. You are projecting onto them the concept of thinking and what you would do if you were in their position.

Really what’s going on is that there are several different processes. There is a vision process at first, which is just sensing the world, seeing where is the ball, where is the line, where is the goal, where are the different sorts of recognizable objects — that’s the vision process.

Then on top of that, consuming the output of vision is what we call localization — knowing where the robot is and where the other object, the ball, is on the field.

The robot is keeping a probabilistic representation of where it is at any given time, where it thinks it is most likely to be, but then there are a lot of other places that it might be. And then on top of that, there is more of a behavior-based system.

You can think of it as sort of a finite state machine. If I am at the ball, then I should kick, if I am not at the ball I should walk towards the ball. If I don’t know where the ball is I should be looking for the ball. There is communication involved, the robots are sharing knowledge of where the ball is and what they are doing so that they are all not going to be doing the same thing at the same time.

Underneath that are the action modules, the actual low levels of where the robot sets its joints after it has decided to kick — how do you actually execute that kick, how do walk without falling, how do you get up after you have fallen over. So there is this pipeline of vision, localization, behavior, and action.

Thinking AI

Cognitive Times  How far do you think we are from machines that actually do think, rather than going through an action sequence?

Peter Stone Well in some sense that’s what thinking it is. That’s what you are doing as well. There is a sequence of information processing that’s happening in your brain.

So, it is fair to say that the robot is thinking — taking in information through sensors and deciding what actions to execute. But it’s not doing it the way a person would do it, or the way an animal would do it.

A Competitive Edge

Cognitive Times Now, you are a soccer player, and you are a violinist. There are music competitions that are often much more subjectively scored than a sport that would be defined by a set of rules. What do you define as the ultimate milestone?

Is it getting to the point where a machine can beat a human in a rules-based competition, or would it be the point where a machine can create music that brings you to tears?

Peter Stone Beating a human at something is not a challenge anymore. Machines have been able to beat people at arithmetic for a long time and there are many things that computers are better at than people.

I don’t think the goal is for computers or robots to be better than people at everything. I don’t think that’s ever likely to happen. I think that it’s better to think of machines as a different species, with different capabilities and different weaknesses.

It would be great if there would be a computer that could create a work of art that would be able to bring people to tears. That would be a very nice achievement and milestone.

But for me there are a lot of concrete research challenges, things that programs can’t do yet that we think they ought to do and ought to be able to do. How can we think about the algorithms that are required to make that happen?

One of the exciting things about AI is that it’s a moving target in some sense. Once the field achieves a goal, the world sort of stops thinking of that goal as being an “AI thing.”

There’s always the next thing. Some people define the field of AI as the science of getting the computers to do what they can’t do yet. And there is always something they can’t do yet. That means that what you were working on 10 years ago may not be current anymore.

We are not at risk of running out of things to do, so I don’t think there is really one challenge. I am inspired by the RoboCup goal of trying to obtain that objective measure of being better than a human at soccer, but that won’t be the mark of all intelligence by any stretch.

Cognitive Times So when we talk about that, there is a view around augmented intelligence that, while AI is getting better, there is the idea that human and machine combos can outperform machines by themselves.

Do you see this as a short term situation where some point down the road AI will be able to outperform humans in all capacities, or do you think that human advantage will always outlast machines?

Peter Stone I think there is always going to be things that are unique to humans. Collaborative systems between robots and people and how machines can augment people is something that’s going to be a reality for quite some time.

Cognitive Times Turning to the University of Texas, what are the some other exciting areas that your colleagues are working on, or areas that you see as really advancing to the state of the art in robotics and AI?

Peter Stone AI is a very broad field, we are actually right now growing in a very exciting way in a broad area that we are calling “machine perception.” It includes computer vision and we have one the world’s experts in computer vision, Kristen Grauman.

We just hired a new young faculty member in computer vision. Another aspect of perception is natural language understanding and Ray Mooney is one the world’s experts in natural language processing.

There are also people who have been around in the department for quite some time working on AI in very important areas. Our Chair, Dr. Bruce Porter, has worked in the areas of knowledge representation and reasoning, and he has made some really big contributions that are still ongoing, exciting areas of AI.

Some people are sort of pointing towards machine learning as being a thing that can solve everything. I don’t think that’s true.

We need to stay grounded in some of the logical reasoning that Vladimir Lifschitz does, and in the knowledge representation and reasoning that Bruce Porter does.

In robotics, there are faculty who are doing very interesting work, sort of related to reinforcement learning, in human robot collaborative systems. They have created a system where the person and the robot worked collaboratively to assemble IKEA furniture, having the robot understand from demonstrations the steps of a task.

Innovation in Austin

Cognitive Times: That’s a lot happening on one campus. Do you see Austin becoming a hub for AI given the trajectory that we are on now?

Peter Stone Yeah, absolutely.

We’ve got a lot of people with entrepreneurial experience and a lot of really deep technical experience. So yes, I think Austin should aim to be the capital in this area.

The Next Level

Cognitive Times I know you touched on this specifically for your research group, but taking a macro view of the industry, what do you see as the next big challenge for AI research?

Peter Stone There is not just one, there are many. There are challenges in computer vision, there are challenges in natural language understanding, moving toward more dialogue-based systems.

The challenges I am focused on do pertain to reinforcement learning or understanding the effects of actions, being able to learn the effects of actions, and being able to take the sequential decision-making from that towards real world problems.

The picture of AI in the media and in the press has always been that it’s a series of either huge successes and huge failures. Right now we are in an upswing and so AI is getting lots of positive attention.

Inevitably, if history rings true, many of the promises that are being made won’t be delivered upon, and then there will be a big crash of disappointed, unmet expectations. Then there will be a trough. Then people will say, “Oh no, it’s better than that.”

There’s a view that it is all happening in these waves, when really the reality of AI is that, like any other science, it’s been a gradual and steady process over the past 50 to 60 years.

The public’s perception is often that breakthroughs are sudden, but really they are the result of decades of gradual progress. I think that’s the way it has always been, and that’s the way it will continue to be.

Cognitive Times Dr. Stone, thank you very much for having us, we really appreciate your time, and best of luck in the upcoming RoboCup.

This story was written in 2016. Dr. Peter Stone’s team won first place at the the international RoboCup 2017 competition in the 3D simulation league and third place in the RoboCup@Home Domestic Standard Platform League with UT Austin Villa.

The Connected Home of the FuturePrevious ArticleThe Connected Home of the Future Winning a Game Show with AlgorithmsNext ArticleWinning a Game Show with Algorithms