A14: Blind Robot

A robot is somewhere in a maze. It has a map of the maze, and a compass, so it can move north, south, east, or west on command. But it has no other sensors: it can’t see, and when it walks into a wall, it doesn’t notice. It doesn’t know where it started. Can it find out where it is? In this assignment, you’ll plan a sequence of moves that leaves the robot sure of where it is, wherever it started, by searching over the sets of squares the robot might be on.

Prerequisites: Sets and Python’s frozenset, heuristic search and A* search, uniform-cost search, priority queues and Python’s heapq module, implicit graphs

Learning Objective: Represent what a robot knows as a set of possibilities, plan with A* search over those sets, and decide whether a heuristic is admissible and consistent.

Background

Mazeworld

Here is a maze, drawn as text:

.#..
....
..#.

Each . is a floor square, where the robot can stand, and each # is a wall. Squares are named by $(x, y)$ coordinates, with $(0, 0)$ at the bottom left: $x$ increases to the east, and $y$ increases to the north. In this maze, the bottom right square is $(3, 0)$ , and the wall in the top row is at $(1, 2)$ .

The robot moves one square at a time, north, south, east, or west, and each move costs 1. If a wall or the edge of the maze is in the way, the robot stays where it is. The robot can’t tell which happened: it can’t tell a wall from the edge of the maze, or even that it bumped into anything.

Belief states

The robot doesn’t know where it is, but it can keep track of where it might be. Its belief state is the set of squares it might be on. At the start, it might be on any floor square, so in the maze above, its belief state is the set of all 10 floor squares:

*#**
****
**#*

Each * marks a square in the belief state. Since the robot knows the map, it can work out what each move does to the belief state, without knowing which square it is really on. Suppose the robot knows it is in the east column, on one of $(3, 0)$ , $(3, 1)$ , or $(3, 2)$ :

.#.*
...*
..#*

What happens if it moves north? If it was on $(3, 0)$ , it is now on $(3, 1)$ . If it was on $(3, 1)$ , it is now on $(3, 2)$ . If it was on $(3, 2)$ , the edge of the maze is in the way, and it stays on $(3, 2)$ . So after the move, its belief state is $\{(3, 1), (3, 2)\}$ :

.#.*
...*
..#.

The robot has no sensors, but it is now more sure of where it is. Walls and edges are what make this possible: two squares that are next to each other, in the direction of the move, end up on the same square when the one in front can’t move. A move never makes the belief state larger, since each square in it goes to one square.

The problem is to find a sequence of moves, a plan, after which the belief state has only one square in it. The robot can follow the plan without knowing where it started, and when it is done, it knows where it is. In the maze above, a plan with six moves does it: north, west, north, south, west, west. After these moves, wherever it started, the robot is on $(0, 1)$ . Some problems ask for more: that the robot end up on a particular goal square, so that the belief state is the set containing only the goal.

The search

A belief state is a set of squares, and a plan is a path through a graph whose vertices are the belief states. The start is the set of all the floor squares, and there is an edge for each move, from a belief state to the belief state after the move, of cost 1. A goal is a belief state with one square in it, or, if there is a goal square, the set containing only that square. A cheapest path from the start to a goal is a shortest plan.

A maze with $n$ floor squares has $2^n$ sets of squares, so the graph is far too big to build. It doesn’t need to be built: it is an implicit graph, and the search finds a belief state’s neighbors only when it explores the belief state. Most sets of squares can’t be reached from the start at all, but many can.

The search needs to keep belief states in a dictionary, so each belief state must be hashable. A Python set isn’t hashable, since it can change, but a frozenset is: it is a set that can’t be changed after it is made. Make a frozenset from a set, or from any collection, with frozenset(...). Two frozensets with the same elements are equal, and have the same hash, whatever order the elements were added in.

A* search

Uniform-cost search explores the states in order of their path cost from the start, which we’ll call $g$ . It spreads out in every direction at once, including directions that lead away from the goal. A* search makes one change: it gives each frontier entry the priority

$f = g + h,$

where $h$ , the heuristic, is an estimate of the cost from the state to a goal. A state that’s cheap to reach but far from the goal gets a larger priority, and is explored later, or not at all.

A* still finds a cheapest path, with the same code as uniform-cost search, if the heuristic has two properties:

It is admissible: it never overestimates. $h$ is at most the true cost of a cheapest path from the state to a goal, and $h$ is 0 at a goal.
It is consistent: along any edge from a state $u$ to a state $v$ with cost $w$ , $h(u) \le w + h(v)$ . That is, one action can’t lower the estimate by more than the action costs.

A heuristic that always guesses 0 has both properties, and A* with it is uniform-cost search. A heuristic that overestimates may lead A* to a path that isn’t the cheapest.

Provided code

Download blindrobot.zip and unzip it. It contains:

maze.py: the Maze class, which loads a maze and draws belief states. You don’t need to change it.
sensorless_problem.py: the start of the SensorlessProblem class, which you’ll finish.
astar_search.py: an A* search, which you may use.
search_solution.py: the SearchSolution class, which holds the result of a search: the path found, its cost, and the number of states explored.
test_sensorless.py: tests, which you’ll add to.
small.maz, maze1.maz, maze2.maz, and split.maz: four mazes.

The `Maze` class

A maze file is a picture of the maze, with . for floor squares and # for walls. The Maze class loads a maze file, and answers questions about it:

maze = Maze("small.maz")
maze.is_floor(3, 0)                       # True: (3, 0) is a floor square
maze.is_floor(1, 2)                       # False: (1, 2) is a wall
maze.is_floor(4, 0)                       # False: (4, 0) is outside the maze
maze.floor_squares()                      # the set of the locations of all 10 floor squares
print(maze.picture({(3, 0), (3, 1)}))     # a text picture, with * on each square of a belief state
maze.print_path(path)                     # each move of a path of belief states, printed as text
maze.animate(path)                        # a window that plays a path of belief states

The dictionary MOVES, in maze.py, maps each action, "north", "east", "south", or "west", to its change in $x$ and $y$ : for example, MOVES["north"] is (0, 1).

Installing PyQt6

animate draws the maze in a window, with a dot on each square of the belief state, and slides the dots with each move. A dot that is blocked bumps against the wall and stays, and dots that end up on the same square merge. It uses PyQt6, a Python library for graphical programs. Install it with pip:

Mac: in the Terminal, run python3 -m pip install PyQt6.
Windows: in the Command Prompt, run py -m pip install PyQt6.

The window plays the path, and has buttons to pause, step one move at a time, and restart, with a slider for the speed. Close the window to let your program go on.

If PyQt6 won’t install, you can do the whole assignment without it: when PyQt6 is missing, animate calls print_path instead, which prints the belief state at the start and after each move. You can also call print_path yourself, or pass text=True to animate.

Part 1: The search problem

A* search doesn’t need to know anything about mazes or robots. All it needs from a search problem is this:

problem.start_state: the start state
problem.is_goal(state): whether state is a goal
problem.get_successors(state): a list of (cost, next_state) pairs, one for each action available in state

and a heuristic, a function that takes a state and returns an estimate of the cost from it to a goal. A state can be anything that can be a key in a dictionary.

In sensorless_problem.py, finish the SensorlessProblem class. Its constructor is given a Maze, and, optionally, a goal location, an $(x, y)$ pair, and keeps the maze in an instance variable, self.maze. It sets self.start_state, the frozenset of all the floor squares, for you. A state is a belief state, a frozenset of $(x, y)$ pairs. Write these methods:

is_goal(state): if there is no goal location, whether the belief state has one square in it; if there is, whether the belief state is the set containing only the goal location.
get_successors(state): a (cost, next_state) pair for each of the four moves, using MOVES and the maze’s is_floor(x, y). Each move costs 1. The next state is the frozenset of the squares that the squares in state go to: one square in the direction of the move, if that square is a floor square, and the same square otherwise. A move that changes nothing is still a successor; the search will skip the state, since it has already been explored.

For example, with MOVES in the order north, east, south, west:

problem = SensorlessProblem(Maze("small.maz"))
for cost, state in problem.get_successors(frozenset({(3, 0), (3, 1), (3, 2)})):
    print(cost, sorted(state))
# 1 [(3, 1), (3, 2)]
# 1 [(3, 0), (3, 1), (3, 2)]
# 1 [(3, 0), (3, 1)]
# 1 [(2, 1), (2, 2), (3, 0)]

Moving east changes nothing, since the edge of the maze is in the way of all three squares. Moving south, the robot on $(3, 0)$ is blocked by the edge of the maze, and ends up where the robot from $(3, 1)$ goes. Moving west, the robot on $(3, 0)$ is blocked by the wall at $(2, 0)$ , and the others move. Run sensorless_problem.py to see the successors of the start state, and check them by hand against the maze.

Part 2: A* search

astar_search.py contains a complete A* search, astar_search(problem, heuristic), which you may use. It follows the pseudocode in the notes on uniform-cost search, with these changes:

The neighbors of a state, and the costs of the edges to them, come from problem.get_successors(state), and the goal test is problem.is_goal(state). Nothing else about the problem appears in astar_search.
Each frontier entry is a tuple (priority, steps, count, path cost, state, parent state), where the priority is the path cost plus heuristic(state), and steps is the number of actions taken to reach the state. When two entries have the same priority, the one reached in fewer steps comes out first. count goes up by one with each push, so that no two entries tie, and heapq never has to compare two states.
It explores each state at most once, and counts the states it explores in solution.nodes_visited. When the goal is explored, it returns the solution, with solution.path, the list of states from the start to the goal, and solution.cost, its cost. If the frontier runs out first, the path is empty.

Read through it, and make sure you understand how it works. You may use an A* search of your own instead, if it works with the same search problems and heuristics, and returns a SearchSolution in the same way. If you use the provided one, cite it: say so in a comment at the top of test_sensorless.py.

To test the search before you have a heuristic, use the null_heuristic in test_sensorless.py, which always guesses 0. On small.maz, it should find a plan with 6 moves; when we ran ours, it explored 120 belief states. (Your count may differ a little, depending on the order of your successors.)

Part 3: Heuristics

A good heuristic for the blind robot must estimate how many moves it takes to shrink a belief state to one square. In SensorlessProblem, write two:

size_heuristic(state): the number of squares in the belief state, minus 1. A plan has to get rid of all but one of them.
bounding_box_heuristic(state): the width of the smallest rectangle that holds every square in the belief state, plus its height, each measured as the difference of the largest and smallest coordinates. For the belief state $\{(0, 1), (2, 2), (3, 1)\}$ , that is $(3 - 0) + (2 - 1) = 4$ . If there is a goal location, the heuristic is instead the larger of that number and the largest Manhattan distance from a square in the belief state to the goal. (The Manhattan distance between two squares is the number of squares east or west between them, plus the number north or south.)

Both are 0 when the belief state has one square in it. Now run test_sensorless.py. It runs A* on small.maz with all three heuristics, and on maze1.maz, a 6 by 6 maze with 28 floor squares, with the size heuristic and the bounding box heuristic:

......
.##.#.
...#..
.#....
.#.##.
......

When we ran ours, the bounding box heuristic found a plan with 14 moves, after exploring 444 belief states. The size heuristic explored only 211, but its plan had 15 moves. Uniform-cost search also finds a plan with 14 moves, but it explores more than 1.5 million belief states, and takes about a minute, so the test leaves it out. Last, the test plans for the robot to end up on $(5, 0)$ , the bottom right square. Watch the animations: how does the robot use the walls to become sure of where it is?

Part 4: Experiments

Run your search on maze2.maz, a 10 by 10 maze with 70 floor squares, with the bounding box heuristic. When we ran ours, it took a few seconds, and explored about 90,000 belief states. Then try split.maz, which has walls dividing it into separate parts:

..#..
..#..
#####
..#..

What does the search report, and why?

Then make at least three mazes of your own, from small up to about 10 by 10, and test them. Make each one interesting, and explain in a comment what it tests. For example, what about a maze with no walls at all? A long corridor? A maze that looks the same after a half turn, so that two squares look alike from everywhere? A maze in which the robot must end up on a square in the middle? For each maze, print the length of the plan and the number of belief states explored by A* with each of the two heuristics, and, for the smaller mazes, with the null heuristic too.

Part 5: Questions

Answer these questions in a comment at the end of test_sensorless.py:

Give a belief state, in small.maz or in a maze of your own, for which the size heuristic overestimates the number of moves needed, and explain why it is an overestimate. What does that tell you about the plans that A* finds with it? Did you see that in your experiments?
Explain why the bounding box heuristic is admissible and consistent, when there is no goal location. (Think about how much one move can change the largest and smallest $x$ coordinates of the squares, and the largest and smallest $y$ coordinates, and what the move costs.)
When there is a goal location, why is the largest Manhattan distance from a square in the belief state to the goal admissible? Why is the larger of two admissible heuristics admissible?
In your experiments, how did the number of belief states explored compare, with each of the three heuristics? On which maze did the bounding box heuristic help least, and why?
Give an upper bound on the number of belief states, for a maze with $n$ floor squares. In practice, A* explored far fewer, even with the null heuristic. Why can most sets of squares never be reached from the start?
Once the robot is sure of where it is, how could it get to a goal square? Would the plan you get this way be a shortest plan to the goal square? Compare your answer with the plans the search found for maze1.maz, with and without the goal location $(5, 0)$ .

What to do

Install PyQt6, if you can, and finish is_goal and get_successors in SensorlessProblem. Check the successors by hand on small.maz.
Read the provided A* search, or use your own, and test it with the null heuristic on small.maz. If you use the provided one, cite it.
Write the two heuristics, and run the provided tests.
Run the searches on maze2.maz and split.maz, and make and test at least three mazes of your own.
Answer the questions in Part 5 in a comment at the end of test_sensorless.py.

Feedback

At the top of your main Python file, add a comment that answers these questions:

About how long did the assignment take you?
What was hard?
What was easy?
Was anything in the assignment unclear?

We use your answers to improve the assignment.

Grading and requirements

is_goal is correct, with and without a goal location.
get_successors is correct: it returns four (cost, next_state) pairs, each of cost 1, and each next state is a frozenset in which every square has moved one square, or stayed, if a wall or the edge of the maze is in the way.
If you use the provided A* search, it is cited in a comment at the top of test_sensorless.py. If you use your own, it uses nothing about the problem but start_state, is_goal, and get_successors, explores each state at most once, uses the path cost plus the heuristic as the priority, finds a cheapest path when the heuristic is consistent, and reports the number of states explored, the path, and its cost.
The size heuristic and the bounding box heuristic are correct, including the goal location case of the bounding box heuristic.
The provided tests run, and maze2.maz, split.maz, and at least three mazes of your own are tested, each with a comment saying what it tests.
The answers to the questions in Part 5 are correct and explained.

What to submit

Upload sensorless_problem.py, test_sensorless.py, astar_search.py (if you used your own), and your maze files to Canvas.

Challenge problems (no extra points, not required)

Plans always exist. Prove that if a maze is finite and its floor squares are all connected to one another, there is always a plan that leaves the robot sure of where it is. (Hint: show that for any two squares, there is a sequence of moves after which a robot that started on either one would be on the same square.)
Planning in polynomial time. The number of belief states can grow exponentially with the number of floor squares, so A* can take a very long time on a large maze. Use the hint above to design a planner whose running time is polynomial in the number of floor squares: repeatedly pick two squares in the belief state, use breadth-first search over pairs of squares to find a sequence of moves that brings them together, and apply the sequence to the whole belief state. Each round removes at least one square. Implement it, and compare the lengths of its plans with the plans A* finds. How large a maze can it handle?
Better heuristics. The bounding box heuristic only looks at the extreme squares. Can you find a larger heuristic that is still admissible? One idea: for each pair of squares in the belief state, the fewest moves needed to bring a robot from each square to the same square is a lower bound, and can be found by breadth-first search over pairs of squares before the main search starts. How many fewer belief states does A* explore with it?

Credits

This assignment is adapted from the “blind robot with Pacman physics” part of the “Mazeworld” assignment in Dartmouth’s CS 76, Artificial Intelligence. Planning without sensors is known as sensorless or conformant planning, and searching over belief states is the foundation of planning when the robot can sense only part of the world.