A13: Multi-Robot Mazeworld

Three robots stand in a narrow corridor, in the wrong order, and must rearrange themselves. Each could find its own way to its goal easily if the others weren’t there, but they are, and they can’t pass through one another. In this assignment, you’ll plan their moves together, by searching a graph whose vertices are the locations of all the robots at once, using A* search.

Prerequisites: Heuristic search and A* search, uniform-cost search, priority queues and Python’s heapq module, implicit graphs

Learning Objective: Model a planning problem as a search over an implicit graph of states, and see how much a good heuristic speeds up the search.

Background

Mazeworld

Here is a maze, drawn as text:

......
.##.#.
...#..
.#....
.#.##.
......

Each . is a floor square, where a robot can stand, and each # is a wall. Squares are named by $(x, y)$ coordinates, with $(0, 0)$ at the bottom left: $x$ increases to the east, and $y$ increases to the north. In this maze, the bottom right square is $(5, 0)$.

There are $k$ robots in the maze, named A, B, C, and so on. They take turns, in order: robot A moves, then robot B, then robot C, and then A again. On its turn, a robot can move one square north, south, east, or west, onto a floor square that no other robot is on, or it can wait where it is. So each robot has at most five actions on its turn. A move costs 1 unit of fuel, and waiting costs nothing. Given a starting location and a goal location for each robot, the problem is to find a sequence of turns that brings every robot to its goal, using as little fuel in total as possible.

Taking turns is a simplification. Real robots could all move at the same moment, but then each state would have up to $5^k$ successors, one for each combination of the robots’ actions, and the rules would have to forbid two robots moving onto the same square or swapping squares. Taking turns keeps each state to at most five successors, at some cost:

• Moves that need to happen at once are impossible. If robots fill a loop of squares, such as four robots in a 2 by 2 block, they can’t all step around the loop together, since on each robot’s turn, the square ahead of it is occupied.
• Fuel says nothing about time. Since waiting is free, a plan that uses the least fuel may have robots waiting around for many turns. Your search will break ties between plans that use the same fuel in favor of the one with the fewest turns, as described in Part 3.

The state

A search for a path needs a state: everything that determines what can happen next. Here, that is the location of every robot, and whose turn it is. A state is a tuple (turn, x0, y0, x1, y1, ...), where turn is the index of the robot whose turn it is (0 for A, 1 for B, and so on), followed by each robot’s location. For example, (1, 1, 0, 1, 1, 2, 1) means that it is robot B’s turn, and that A is at $(1, 0)$, B is at $(1, 1)$, and C is at $(2, 1)$.

The states are the vertices of a graph, with an edge for each action, weighted by its cost. The graph is far too big to build, but it doesn’t need to be built: it is an implicit graph, and the search finds a state’s neighbors only when it explores the state. A goal state is any state in which every robot is at its goal, whoever’s turn it is.

It might seem simpler to plan a path for each robot separately, with three separate searches, but separate paths can collide, and in a corridor, a robot may have to step out of the way, or even backward, to let another pass. Searching over the states of all the robots together finds plans like that.

A* search

Uniform-cost search explores the states in order of their path cost from the start, which we’ll call $g$. It spreads out in every direction at once, including directions that lead away from the goal. A* search makes one change: it gives each frontier entry the priority

$f = g + h,$

where $h$, the heuristic, is an estimate of the cost from the state to a goal. A state that’s cheap to reach but far from the goal gets a larger priority, and is explored later, or not at all.

The heuristic has to be chosen carefully. A* still finds a cheapest path, with the same code as uniform-cost search, if the heuristic has two properties:

• It never overestimates: $h$ is at most the true cost of a cheapest path from the state to a goal, and $h$ is 0 at a goal.
• It is consistent: along any edge from a state $u$ to a state $v$ with cost $w$, $h(u) \le w + h(v)$. That is, one action can’t lower the estimate by more than the action costs.

A heuristic that always guesses 0 has both properties, and A* with it is exactly uniform-cost search.

Provided code

Download mazeworld.zip and unzip it. It contains:

• maze.py: the Maze class, which knows the rules of mazeworld. You don’t need to change it.
• single_robot_problem.py: the start of the SingleRobotProblem class, for one robot, which you’ll finish.
• multi_robot_problem.py: the start of the MultiRobotProblem class, for several robots, which you’ll finish.
• astar_search.py: the start of the A* search, which you’ll finish.
• search_solution.py: the SearchSolution class, which holds the result of a search: the path found, its cost, and the number of states explored.
• test_mazeworld.py: tests, which you’ll add to.
• maze1.maz, maze2.maz, and corridor.maz: three mazes.

The Maze class

A maze file is a picture of the maze, followed by a line \robot x y for each robot’s starting location; the first robot listed is A. The Maze class loads a maze file, and provides the states and the rules, but no search:

maze = Maze("maze2.maz")
state = maze.start_state()      # the state with every robot at its start, and A to move
maze.actions(state)             # the actions available to the robot whose turn it is
maze.result(state, action)      # the state after that robot takes the action
maze.locations(state)           # the list of the robots' (x, y) locations
print(maze.picture(state))      # a text picture of the maze, with the robots
maze.animate(path, goals)       # a window that plays a path of states
maze.print_path(path, goals)    # the same path, printed as text

The actions are the strings "north", "east", "south", "west", and "wait". actions lists only the moves onto floor squares with no robot on them, and always includes "wait". result moves the robot and passes the turn to the next robot. For example, in maze2.maz, robots A, B, and C start at $(1, 0)$, $(1, 1)$, and $(2, 1)$:

##.##
#...#
#.#.#
#...#
#BC.#
#A###

It is A’s turn, and A can’t move west or east, into a wall, or north, onto B, so:

state = maze.start_state()
print(state)                       # (0, 1, 0, 1, 1, 2, 1)
print(maze.actions(state))         # ['wait']
print(maze.result(state, "wait"))  # (1, 1, 0, 1, 1, 2, 1): now it is B's turn

Installing PyQt6

animate draws the maze in a window, with the robots sliding from square to square, using PyQt6, a Python library for graphical programs. Install it with pip:

• Mac: in the Terminal, run python3 -m pip install PyQt6.
• Windows: in the Command Prompt, run py -m pip install PyQt6.

The window plays the path, and has buttons to pause, step one turn at a time, and restart, with a slider for the speed. The robot whose turn it is has a dark outline, and each robot’s goal, if you give the goals, is a dashed square in the robot’s color. Close the window to let your program go on.

If PyQt6 won’t install, you can do the whole assignment without it: when PyQt6 is missing, animate calls print_path instead, which prints the maze at the start and after each move, with the robots as capital letters and their goals as lowercase letters. You can also call print_path yourself, or pass text=True to animate.

Part 1: Search problems, and a single robot

A* search doesn’t need to know anything about mazes or robots. All it needs from a search problem is this:

• problem.start_state: the start state
• problem.is_goal(state): whether state is a goal
• problem.get_successors(state): a list of (cost, next_state) pairs, one for each action available in state

and a heuristic, a function that takes a state and returns an estimate of the cost from it to a goal. A state can be anything that can be a key in a dictionary, such as a tuple. If astar_search uses only these, the same code can plan for one robot, for several robots, or for a problem with nothing to do with mazes. You’ll use it for two problems: one robot first, where it’s easy to check the answers by hand, and then several robots.

In single_robot_problem.py, finish the SingleRobotProblem class. Its constructor is given a Maze and the robot’s goal location, an $(x, y)$ pair, and keeps the maze in an instance variable, self.maze. SingleRobotProblem doesn’t extend Maze: a search problem isn’t a kind of maze, but it has one, and asks it about walls. MultiRobotProblem, in Part 3, keeps its maze the same way. The robot starts at the location of the first robot in the maze file. A state is the robot’s location, an $(x, y)$ pair, and there are no turns. Write these methods:

• is_goal(state): whether the robot is at its goal.
• get_successors(state): a (cost, next_state) pair for each move north, south, east, or west onto a floor square, with the maze’s is_floor(x, y). Each move costs 1. A single robot has no reason to wait, so there is no wait action. The dictionary MOVES, from maze.py, maps each action to its change in $x$ and $y$.
• manhattan_heuristic(state): the robot’s Manhattan distance to its goal: the number of squares east or west, plus the number north or south, ignoring walls. A robot at $(1, 0)$ with its goal at $(4, 2)$ is at Manhattan distance $3 + 2 = 5$.

For example:

problem = SingleRobotProblem(Maze("maze1.maz"), (5, 0))
print(problem.start_state)                          # (0, 0)
print(problem.get_successors(problem.start_state))  # [(1, (0, 1)), (1, (1, 0))]

Part 2: A* search

In astar_search.py, finish astar_search(problem, heuristic). Start from the pseudocode in the notes on uniform-cost search, and change it in these ways:

• The neighbors of a state, and the costs of the edges to them, come from problem.get_successors(state), and the goal test is problem.is_goal(state). Nothing else about the problem should appear in astar_search.
• Each frontier entry is a tuple (priority, steps, count, path cost, state, parent state), where the priority is the path cost plus heuristic(state), and steps is the number of actions taken to reach the state: one more than the steps for its parent. count goes up by one with each push, so that no two entries tie, and heapq never has to compare two states.
• Count each state you explore in solution.nodes_visited. When the goal is explored, set solution.path to the list of states from the start to the goal, with the provided backchain function, and solution.cost to its cost, and return the solution. If the frontier runs out first, return the solution with an empty path.

heapq compares tuples entry by entry, so when two entries have the same priority, the one reached in fewer steps comes out first. The priority still comes first, so the search still finds a cheapest path, but among the cheapest paths, it finds one with the fewest steps. For a single robot, every step costs 1, so this makes no difference. It matters for several robots, as Part 3 explains.

Test your search on the single robot first, with the first part of test_mazeworld.py. It runs A* on maze1.maz twice: once with a heuristic that always guesses 0, which makes it uniform-cost search, and once with the Manhattan heuristic. Both should find a path of cost 5; when we ran ours, the first explored 16 states and the second only 6. Check the path by hand against the maze, and fix any problems before going on: a bug is much easier to find with one robot than with three.

Part 3: Several robots

In multi_robot_problem.py, finish the MultiRobotProblem class, which adds the robots’ goals to a maze. The constructor is given a Maze and a tuple (x0, y0, x1, y1, ...) of the robots’ goal locations, and sets self.start_state, the maze’s start state, for you. A state is a tuple (turn, x0, y0, x1, y1, ...), as described above. Write these methods:

• is_goal(state): whether every robot is at its goal. Ignore whose turn it is.
• get_successors(state): a list of (cost, next_state) pairs, one for each action available in state, using the maze’s actions and result. A move costs 1, and a wait costs 0.
• manhattan_heuristic(state): the sum, over all the robots, of each robot’s Manhattan distance to its goal.

For example:

problem = MultiRobotProblem(Maze("maze2.maz"), (1, 4, 1, 3, 1, 2))
print(problem.get_successors(problem.start_state))
# [(0, (1, 1, 0, 1, 1, 2, 1))]

Test these methods on a few more states of your own. Your astar_search should work on this problem without any changes.

Here the tie-breaking by steps matters. Each step is one robot’s turn, waits included. Waiting costs no fuel, so there are often many plans that use the least fuel, and some have robots waiting around for turns on end before they move. Among the plans that use the least fuel, your search finds one with the fewest turns, so the robots don’t wait without a reason.

Now run the rest of test_mazeworld.py. On maze2.maz, with three robots, both searches should find a path of cost 10; when we ran ours, uniform-cost search explored 2998 states and A* with the Manhattan heuristic explored 239. (Your counts may differ a little, depending on the order of your successors.) The window that opens shows the robots taking turns, and stepping aside for one another.

Part 4: Experiments

The robots in corridor.maz start in the order A, B, C, from west to east, and must end in the order C, B, A, on the same three squares:

########
..ABC...
##.#####

The one-square pocket on the bottom row is the only place where a robot can step aside. Run the test, and watch the animation: how do the robots get past one another?

Then make at least three mazes of your own, with one to three robots, from small up to about 20 by 20, and test them. A* with the Manhattan heuristic takes a few seconds on a 20 by 20 maze with three robots, when we ran ours. Make each one interesting, and explain in a comment what it tests. For example, what if the robots must swap places through a single doorway? What if a robot is boxed in by the others? What if no plan exists? For each maze, print the number of states explored by A* with the Manhattan heuristic, and, for the smaller mazes, with the null heuristic too. Uniform-cost search explored more than half a million states on a 10 by 10 maze with three robots, and didn’t finish in five minutes on a 20 by 20 one.

Part 5: Questions

Answer these questions in a comment at the end of test_mazeworld.py:

1. Why does the Manhattan heuristic never overestimate the cost of reaching the goal? Why is it consistent? (Think about how much one action can change it, and what the action costs.)
2. In your experiments, how did the number of states explored compare, with and without the heuristic? On which maze did the heuristic help least, and why?
3. Give an upper bound on the number of states, in terms of $k$, the number of robots, and $n$, for a maze of $n$ by $n$ squares.
4. Suppose there are 10 robots in a 100 by 100 maze with few walls. Would A* with the Manhattan heuristic be practical for every start and goal? Why or why not?
5. In the 8-puzzle, eight numbered tiles sit in a 3 by 3 frame, with one empty square. A tile next to the empty square can slide into it, and the goal is to put the tiles in order. Explain how the 8-puzzle is a special case of mazeworld. Is the Manhattan heuristic a good one for it?

What to do

1. Install PyQt6, if you can, and finish SingleRobotProblem.
2. Finish astar_search, and test it on the single robot.
3. Finish MultiRobotProblem, test its methods on states of your own, and run the provided tests.
4. Run the corridor test, make and test at least three mazes of your own, and compare the searches with and without the heuristic.
5. Answer the questions in Part 5 in a comment at the end of test_mazeworld.py.

Feedback

At the top of your main Python file, add a comment that answers these questions:

• About how long did the assignment take you?
• What was hard?
• What was easy?
• Was anything in the assignment unclear?

We use your answers to improve the assignment.

Grading and requirements

• SingleRobotProblem’s goal test, successors, and heuristic are correct: each move onto a floor square costs 1, and the heuristic is the robot’s Manhattan distance to its goal.
• MultiRobotProblem’s goal test, successors, and heuristic are correct: the successors come from the maze’s actions and result, a move costs 1 and a wait costs 0, the goal test ignores the turn, and the heuristic is the sum of the robots’ Manhattan distances to their goals.
• astar_search uses nothing about the problem but start_state, is_goal, and get_successors, and works unchanged on both problems. It explores each state at most once, uses the path cost plus the heuristic as the priority, breaks ties by the number of steps, finds a cheapest path, and reports the number of states explored, the path, and its cost.
• The provided tests and the corridor test run, and at least three mazes of your own are tested, each with a comment saying what it tests.
• The answers to the questions in Part 5 are correct and explained.

What to submit

Upload single_robot_problem.py, multi_robot_problem.py, astar_search.py, test_mazeworld.py, and your maze files to Canvas.

Challenge problems (no extra points, not required)

• Random mazes. Write a function that generates a random $n$ by $n$ maze, with each square a wall with some probability, and random starting and goal locations for $k$ robots. How large can $n$ and $k$ get before A* takes more than a minute?
• A better heuristic. The Manhattan heuristic ignores walls. Before the search, use breadth-first search from each robot’s goal to find the true distance from every square to that goal, around the walls, and use the sum of these distances as the heuristic. Is it still consistent? How many fewer states does A* explore?

Credits

This assignment is adapted from the “Mazeworld” assignment in Dartmouth’s CS 76, Artificial Intelligence.