OpenAI Has Trained a Neural Network To Competently Play Minecraft (openai.com) 24
In a blog post today, OpenAI says they've "trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled video dataset of human Minecraft play, while using only a small amount of labeled contractor data." The model can reportedly learn to craft diamond tools, "a task that usually takes proficient humans over 20 minutes (24,000 actions)," they note. From the post: In order to utilize the wealth of unlabeled video data available on the internet, we introduce a novel, yet simple, semi-supervised imitation learning method: Video PreTraining (VPT). We start by gathering a small dataset from contractors where we record not only their video, but also the actions they took, which in our case are keypresses and mouse movements. With this data we train an inverse dynamics model (IDM), which predicts the action being taken at each step in the video. Importantly, the IDM can use past and future information to guess the action at each step. This task is much easier and thus requires far less data than the behavioral cloning task of predicting actions given past video frames only, which requires inferring what the person wants to do and how to accomplish it. We can then use the trained IDM to label a much larger dataset of online videos and learn to act via behavioral cloning.
We chose to validate our method in Minecraft because it (1) is one of the most actively played video games in the world and thus has a wealth of freely available video data and (2) is open-ended with a wide variety of things to do, similar to real-world applications such as computer usage. Unlike prior works in Minecraft that use simplified action spaces aimed at easing exploration, our AI uses the much more generally applicable, though also much more difficult, native human interface: 20Hz framerate with the mouse and keyboard.
Trained on 70,000 hours of IDM-labeled online video, our behavioral cloning model (the âoeVPT foundation modelâ) accomplishes tasks in Minecraft that are nearly impossible to achieve with reinforcement learning from scratch. It learns to chop down trees to collect logs, craft those logs into planks, and then craft those planks into a crafting table; this sequence takes a human proficient in Minecraft approximately 50 seconds or 1,000 consecutive game actions. Additionally, the model performs other complex skills humans often do in the game, such as swimming, hunting animals for food, and eating that food. It also learned the skill of "pillar jumping," a common behavior in Minecraft of elevating yourself by repeatedly jumping and placing a block underneath yourself. For more information, OpenAI has a paper (PDF) about the project.
We chose to validate our method in Minecraft because it (1) is one of the most actively played video games in the world and thus has a wealth of freely available video data and (2) is open-ended with a wide variety of things to do, similar to real-world applications such as computer usage. Unlike prior works in Minecraft that use simplified action spaces aimed at easing exploration, our AI uses the much more generally applicable, though also much more difficult, native human interface: 20Hz framerate with the mouse and keyboard.
Trained on 70,000 hours of IDM-labeled online video, our behavioral cloning model (the âoeVPT foundation modelâ) accomplishes tasks in Minecraft that are nearly impossible to achieve with reinforcement learning from scratch. It learns to chop down trees to collect logs, craft those logs into planks, and then craft those planks into a crafting table; this sequence takes a human proficient in Minecraft approximately 50 seconds or 1,000 consecutive game actions. Additionally, the model performs other complex skills humans often do in the game, such as swimming, hunting animals for food, and eating that food. It also learned the skill of "pillar jumping," a common behavior in Minecraft of elevating yourself by repeatedly jumping and placing a block underneath yourself. For more information, OpenAI has a paper (PDF) about the project.
I knew it! (Score:2)
Re: (Score:2)
I would really like to know what they used as a loss function.
The primary point of Minecraft is to build something cool so you can show it to your friends.
Re: (Score:3)
Actually, the point is to kill the Ender Dragon. It's been that was since about 2011.
Re: (Score:2)
Do you know anyone who stopped playing after they killed the Ender Dragon? That is in no way the point.
Otherwise what would you do with all that experience and the dragon egg?
Re: (Score:2)
Sometimes people do abandon the server after killing the dragon, yes. Especially when playing solo. The point is to start from scratch.
Re: (Score:2)
Now that I've actually read the blog post, the thing here isn't that they trained it to play Minecraft, they trained it to learn from random internet videos.
First, they gave the NN 2k hours of labeled video (so it knew exactly which buttons were pushed at each point). The NN from that learned how to use the controls.
Then in the next step, they gave it 70k hours of unlabeled video from the internet. It was able to learn from those videos, even though they were unlabeled.
Re: (Score:1)
Minecraft doesn't offer explicit goals, but that's beside the point. An AI coul
Re: (Score:2)
An AI could be given goals like "Get to the nether" or "build a house" and its time to accomplish that compared to how long it takes humans.
Those "goals" in the context of AI are dramatically different that in the context of a human player. It doesn't even really make sense to talk in terms of time when comparing their performance. Hell, it doesn't even make sense to compare their performance. What the model is doing, in this case, is pretty far removed from what we understand a human to be doing.
People have this odd idea that "an AI" is like a cartoon robot. You know, some semi-autonomous thing that can learn and grow over time as it encou
Re: (Score:1)
, an AI can't set it's own goals and evaluate it's own performance and adapt on that basis.
The amount of assumptions being made here is mindboggling. There is no being that has ever existed that can do this without years of training and teaching by family, friends, and society. Some humans never achieve this. Btw, good luck on the goal of 'build a house'. Just building something as simple as a sandwich without relying on anything but the collective sum of human knowledge takes ~6 months and 1500$. https://www.smithsonianmag.com... [smithsonianmag.com]
This feat they have achieved is utterly amazing.
Re: (Score:2)
There is no being that has ever existed that can do this without years of training and teaching by family, friends, and society.
Nonsense. Have you never seen a baby play? They do indeed set their own goals, evaluate their performance, and adapt their behavior accordingly. They even get frustrated when they can't do whatever it is they've set out to do.
This feat they have achieved is utterly amazing.
LOL! No, no it's not. I suggest you read the paper. You're about to be really disappointed.
The real test (Score:3)
The real test is does it know when to stop playing?
Re:The real test (Score:5, Funny)
The real test is does it know when to stop playing?
(if the answer is "no", then it is indistinguishable from a real human and therefore passes the Minecraft Turing Test)
Re: (Score:2)
I thought the right answer was "after Microsoft bought it", which was the case for me since they decided to add a good healthy dose of grind to the anvil mechanics and that got dull.
It digs down (Score:2)
I wonder who taught it to dig down :)
Well then ... (Score:2)
OpenAI Has Trained a Neural Network To Competently Play Minecraft
It's a better AI than I'm not, 'cause I still don't really get Minecraft. I think another Rick said it well [youtube.com]:
Rick Sanchez: So, you're mining stuff to craft with and crafting stuff to mine with?
...
Morty: Uh-huh.
Rick Sanchez: Did your dad write this game?
Morty: You can use that wood to make a chest.
Rick Sanchez: Oh, good. Then I can store all this wood I'll need later for chest-making.
Morty: Okay. You're not going to have fun if you analyze everything.
Re: (Score:2)
OpenAI Has Trained a Neural Network To Competently Play Minecraft
That's more than a slight exaggeration. It's not even what the paper is about.
Re: (Score:2)
OpenAI Has Trained a Neural Network To Competently Play Minecraft
That's more than a slight exaggeration. It's not even what the paper is about.
It is in line with the current no-understanding fanatical AI hype though, where people without even an elementary clue pronounce AI "sentient", "competent", "understanding things", etc., when even a basic understanding of the technology used makes is absolutely clear nothing like that could be the case.
I also trained an AI. (Score:2)
It's based on the principle Don't Repeat Yusuf.
Can it get me a beer as well? (Score:2)
Keep on training your AI to do something usefull...
Nope (Score:2)
What it can do is fumble in the dark by replaying things it has seen before. It has no clue what it does, it cannot plan and "competently" does not even enter the picture.
Stop projecting. This is an automaton, not a sentient being.
Analayzer (Score:1)
Re: (Score:1)
Next: Software tutorial videos (Score:2)