DQN is very efficient when there is a large state space with a large amount of actions, as it can estimate a Q-value for a never-seen-before state by comparing how similar it is to known states. It involves utilizing a deep neural network (CNN) to approximate the Q value for (state, action) pairs instead of a table like classical Q-Learning. However, in a large state space with a large amount of possible actions, this table becomes infeasible to keep and thoroughly ‘fill out’.ĭeep Q-Learning is an alternative approach to Q-Learning that doesn’t face these issues. The agent will simply look at every action that it can take within a state, and execute the action with the largest Q-value.
Q-Learning is a technique to develop such a policy by maintaining a large table with expected rewards (Q values) for every (state, action) pair. In a model with an agent and an environment, the goal of reinforcement learning is to develop a policy such that an agent can find itself in a particular state, query its policy, and perform the corresponding action such that its reward is maximized. As a quick refresher on reinforcement learning, take a look at the graphic below. The DQN, or Deep Q-Learning Network, algorithm is a novel approach to reinforcement learning proposed by the DeepMind team in 2013. The existing research surrounding Settlers of Catan use this agent as a benchmark, but no attempts to develop a new Settlers of Catan AI have been able to be significantly better than the JSettlers agent to date. This program is developed in Java and contains a fully-implemented framework of the board game as well as an entirely rule-based agent with hard-coded decisions and heuristics. The ‘world champion’ standard of AI for this board game is a program called JSettlers, developed by Jeremy Monin over 10 years ago and still in active development today (he gave some advice for this project!). On the other hand, a player must utilize the resources of others: only proposing and accepting trades that are most beneficial to the player. On one hand, a player must create a building policy to prioritize certain structures in certain locations. Throughout the game, players negotiate with each other, proposing trades for various resources and choosing which trades to decline or accept.įrom an Artificial Intelligence perspective, developing an agent for this game is an interesting challenge. Although rolling dice and preparing a building strategy is enjoyable, one of the most fun aspects of the game comes from its social component. It is a four-player game that involves rolling dice, collecting resources, and building a settlement that eventually is successful enough to win the game. Settlers of Catan is a very popular board game created by Klaus Teuber in 1995 and has been played all over the world for almost 25 years. Peter McAughan, Arvind Krishnakumar, James Hahn, Shreeshaa Kulkarni QSettlers: Deep Reinforcement Learning for Settlers of Catan
0 kommentar(er)
