Graphs

Much of the data we work with records connections between pairs of things, one pair at a time. Two people are parent and child. A road joins two intersections. A web page links to another page. A flight goes from one airport to another. One course must be taken before another. When you get dressed, socks go on before shoes, while socks and a shirt can go on in either order. Each of these facts is local: it says something about just two things.

The interesting questions follow chains of these connections. How is one person related to another, through a line of parents and children, say Professor Balkcom to Abraham Lincoln? With no direct flight from Boston to Tokyo, which route needs the fewest changes of plane? Starting from one web page and following links, which pages can a search engine find? Is there an order for taking all the courses in a major so that every prerequisite comes before the course that needs it, and if so, what is one? (There are usually many; any one will do. The same question asks for an order for putting on your clothes.) No single fact answers any of these questions; they need many connections, followed one after another.

Sometimes we need to know more than just that there’s a connection: some quantity about the connection matters too. For a road map, we might care not only that a road connects two intersections, but also how long the road is. For world currencies, we might want the exchange rate from each one to each other.

Mathematicians devised a model for all of these situations, with many things and relationships between pairs of them: a graph. A graph consists of vertices (singular: vertex) connected by edges. Each edge connects one vertex to another. A vertex may have edges to zero, one, or many other vertices. Think of each vertex as a thing, and each edge as a connection between two things.

Here is a graph with 9 vertices and 11 edges:

Each vertex in this graph is labeled with a letter, though in general vertices can be labeled however we like, or not at all. Vertex A, for example, has two edges: one to vertex C, and one to vertex E.

Graph vocabulary

Some relationships are symmetric: if A is B’s friend, then B is A’s friend too. Others are not: if A follows B on a social network, B need not follow A, and if course A must come before course B, then B certainly doesn’t come before A. A symmetric relationship is modeled with undirected edges, each between two vertices. An asymmetric relationship is modeled with directed edges, each from one vertex to another. A graph with undirected edges is an undirected graph, and a graph with directed edges is a directed graph. An undirected edge between $x$ and $y$ can always be simulated with two directed edges, one from $x$ to $y$ and one from $y$ to $x$. The graph above is undirected.

A number on an edge, such as the length of a road, is called an edge weight, “weight” being a generic term for whatever quantity matters. (Unless you’re a civil engineer building elevated highways, you probably don’t care how much a road actually weighs.) Edges can be weighted or unweighted, in either directed or undirected graphs. Here is a small road map, whose edge weights are driving times, in minutes:

Graphs are sometimes called networks, vertices are sometimes called nodes, and edges are sometimes called links or arcs.

A few more definitions:

• We write the edge from vertex $x$ to vertex $y$ as $(x, y)$. If the edge is undirected, $(y, x)$ is the same edge as $(x, y)$; if it’s directed, it is not.
• In an undirected graph, the vertices $x$ and $y$ of an edge $(x, y)$ are adjacent, or neighbors. In the graph above, D and G are adjacent. In a directed graph, the edge $(x, y)$ leaves $x$ and enters $y$.
• The degree of a vertex in an undirected graph is the number of edges it has. In the graph above, B has degree 5. In a directed graph, the number of edges leaving a vertex is its out-degree, and the number entering is its in-degree.
• A path from $x$ to $y$ is a sequence of vertices, starting at $x$ and ending at $y$, each joined to the next by an edge. The length of the path is its number of edges. In the graph above, one path from D to A is D, I, C, A, of length 3; another is D, G, B, E, A, of length 4. There is always a path of length 0 from a vertex to itself.
• A cycle is a path of at least one edge from a vertex back to itself, with no edge used twice. In the graph above, A, E, B, I, C, A is a cycle.
• An undirected graph is connected if there is a path between every pair of vertices. A graph that isn’t connected falls into separate pieces, called its connected components: within a component, every pair of vertices has a path between them, and there is no path between vertices in different components. This graph of nine people and their friendships has three connected components:

A tree is a special kind of graph: a connected, undirected graph with no cycles. In a tree, any two vertices are joined by exactly one path. A directed graph with no cycles is called a directed acyclic graph, or DAG. The courses and prerequisites of a college form a DAG, since no course can, even indirectly, be a prerequisite of itself. So do the articles of clothing you put on in the morning, with an edge from each article to any that must go on after it:

Representing graphs

There are a few ways to represent a graph in a program, each better for some purposes than others. It’s convenient to write $n$ for the number of vertices of a graph, and $m$ for the number of edges. Here is a graph with $n = 5$ vertices and $m = 7$ edges, in each of the three common representations. Think of the vertices as people, Ann, Bob, Cal, Dee, and Eve, and the edges as friendships:

Edge lists

The simplest representation is a list of the $m$ edges, called an edge list. Each edge is a pair of vertices, such as a Python tuple ("Bob", "Dee"), with a third item added for a weight, if there is one:

edges = [("Ann", "Bob"), ("Ann", "Eve"), ("Bob", "Cal"), ("Bob", "Dee"),
         ("Bob", "Eve"), ("Cal", "Dee"), ("Dee", "Eve")]

An edge list is simple, and adding an edge takes constant time. But almost everything else is slow. To find whether the graph has a particular edge, or which vertices are adjacent to a particular vertex, we have to search through the whole list, taking $\Theta(m)$ time.

Adjacency matrices

An adjacency matrix is an $n \times n$ table of 0s and 1s, with a row and a column for each vertex. The entry in a vertex $x$‘s row and a vertex $y$‘s column is 1 if and only if the graph has the edge $(x, y)$. In Python, the matrix can be a list of $n$ lists, each holding $n$ numbers, with the vertices in the same order as in a list of their names:

names = ["Ann", "Bob", "Cal", "Dee", "Eve"]
matrix = [[0, 1, 0, 0, 1],    # Ann
          [1, 0, 1, 1, 1],    # Bob
          [0, 1, 0, 1, 0],    # Cal
          [0, 1, 1, 0, 1],    # Dee
          [1, 1, 0, 1, 0]]    # Eve

Each row lists one vertex’s edges. Bob’s row, [1, 0, 1, 1, 1], has 1s in the columns for Ann, Cal, Dee, and Eve, so Bob has edges to all four of them, as in the drawing of the graph above. The 0 in Bob’s own column says that Bob has no edge to himself. Cal’s row, [0, 1, 0, 1, 0], says that Cal has edges only to Bob and Dee. The graph is undirected, so the edge between Bob and Cal appears twice: in Bob’s row and Cal’s column, and in Cal’s row and Bob’s column.

Numbering rows and columns. In mathematics, the rows and columns of a matrix are numbered from 1: row 1 is the first row, and $A_{ij}$ is the entry in row $i$ and column $j$, for $i$ and $j$ from 1 to $n$. Python numbers the items of a list from 0, so row $i$ is matrix[i - 1], and $A_{ij}$ is matrix[i - 1][j - 1]. To keep the two conventions apart, these notes give vertices names, not numbers, and number rows and columns only in mathematical formulas, always from 1. In code, a vertex’s index, its position in the list of names, gives its row and column: Bob’s index is names.index("Bob"), which is 1, so Bob’s row, the second row, is matrix[1].

To represent weights, put each edge’s weight in its entry, and reserve a special value, such as None, for an absent edge. For an undirected graph, the matrix is symmetric: the entry in row $i$, column $j$ equals the entry in row $j$, column $i$. For a directed graph, it need not be. Since $(i, j)$ is the edge from $i$ to $j$, row $i$ of a directed graph’s matrix records the edges that leave $i$, and column $j$ records the edges that enter $j$.

With an adjacency matrix, we can find whether an edge is present in constant time, by looking up one entry: Bob and Dee are adjacent, since the entry in Bob’s row and Dee’s column, matrix[1][3], is 1. The matrix has two disadvantages. First, it takes $\Theta(n^2)$ space, even if the graph is sparse, with relatively few edges. A road map with a million intersections, each with about four roads, has a few million edges, but its adjacency matrix would have a trillion entries, almost all 0. Second, finding the vertices adjacent to a vertex means looking at all $n$ entries in its row, taking $\Theta(n)$ time, even if it has only a few neighbors.

Adjacency lists

An adjacency list representation stores, for each vertex, a collection of the vertices adjacent to it. With the vertices’ indices from the list names, it can be a list of $n$ lists of indices:

adjacent = [[1, 4], [0, 2, 3, 4], [1, 3], [1, 2, 4], [0, 1, 3]]

We can get to a vertex’s adjacency list in constant time, by indexing: Bob’s list, adjacent[1], is [0, 2, 3, 4], the indices of Ann, Cal, Dee, and Eve. Looping over the neighbors of a vertex takes time proportional to its degree, which is as fast as it could be. To find whether the edge $(x, y)$ is present, we look for $y$ in $x$‘s list, which takes $O(d)$ time, where $d$ is the degree of $x$. The degree could be as large as $n - 1$, if $x$ is adjacent to every other vertex, or as small as 0.

How much space do adjacency lists take? There are $n$ lists, and although each could hold as many as $n - 1$ vertices, all together they hold $2m$ items for an undirected graph: each edge $(x, y)$ appears exactly twice, once in $x$‘s list and once in $y$‘s. For a directed graph, they hold $m$ items, one per edge. Either way, the space is $\Theta(n + m)$: $n$ for the lists themselves, and $m$ for their contents. For a sparse graph, that’s far less than an adjacency matrix.

Adjacency lists in Python: a dictionary of lists

In Python, vertices don’t need to be numbered. A dictionary can map each vertex’s name, such as "Bob", to a list of its neighbors’ names:

class Graph:
    def __init__(self):
        self.adjacent = {}   # name -> list of neighbors' names

    def add_vertex(self, v):
        if v not in self.adjacent:
            self.adjacent[v] = []

    def add_edge(self, u, v):
        """Add an undirected edge between u and v."""
        self.add_vertex(u)
        self.add_vertex(v)
        self.adjacent[u].append(v)
        self.adjacent[v].append(u)

    def has_edge(self, u, v):
        return u in self.adjacent and v in self.adjacent[u]

    def neighbors(self, v):
        return self.adjacent[v]

    def vertices(self):
        return self.adjacent.keys()

Looking up a vertex’s list in the dictionary takes constant time on average, and adding a vertex or an edge takes constant time on average too. has_edge searches u’s list for v, which takes time proportional to the degree of u. (Storing each vertex’s neighbors in a Python set instead of a list would make has_edge take constant time on average; for searching a graph, which only loops over neighbors, a list is all we need.) For a directed graph, add_edge would append only v to u’s list. To store weights, the value for each vertex can be a dictionary instead of a list, mapping each neighbor to the weight of the edge to it.

Often each vertex carries data of its own: a person’s email address, a city’s population, the text of a web page. Then the value for each name can be an object that holds the vertex’s data along with its list of neighbors’ names:

class Vertex:
    def __init__(self, data):
        self.data = data          # anything: text, a number, another object
        self.neighbors = []       # neighbors' names

vertices = {}                     # name -> Vertex
vertices["home"] = Vertex("Welcome to the store!")
vertices["news"] = Vertex("New this week: apple pie.")
vertices["home"].neighbors.append("news")    # a directed edge from home to news

The dictionary finds a vertex from its name in constant time on average, and the vertex holds everything else. It is still an adjacency-list representation: each vertex’s list is now stored inside the vertex, alongside its data.

Here is the lettered graph from the start of these notes:

g = Graph()
for u, v in ["AC", "AE", "CI", "HI", "BI", "DI", "BE", "BD", "BF", "BG", "DG"]:
    g.add_edge(u, v)

(Each two-letter string unpacks into its two letters.) Now g.has_edge("D", "G") is True, and g.neighbors("B") is the list ["I", "E", "D", "F", "G"], in the order in which B’s edges were added.

Comparing the representations

	Edge list	Adjacency matrix	Adjacency lists
Space	$\Theta(m)$	$\Theta(n^2)$	$\Theta(n + m)$
Is $(x, y)$ an edge?	$O(m)$	$O(1)$	$O(\text{degree of } x)$
Loop over the neighbors of $x$	$O(m)$	$\Theta(n)$	$\Theta(\text{degree of } x)$
Add an edge	$O(1)$	$O(1)$	$O(1)$ on average
Add a vertex	$O(1)$	$\Theta(n)$ amortized	$O(1)$ on average

Adding a vertex to an adjacency matrix stored as a list of lists means appending a 0 to each of the $n$ rows and then appending a new row of $n + 1$ zeros. Each append takes constant amortized time, so adding a vertex takes $\Theta(n)$ amortized time. A matrix stored in an array of fixed size, such as a NumPy array, would instead have to be copied into a new $(n + 1) \times (n + 1)$ array, which takes $\Theta(n^2)$ time.

For most graphs that come up in practice, which are sparse, adjacency lists are the usual choice. An adjacency matrix makes sense for a dense graph, in which most pairs of vertices are adjacent, where it takes little more space than adjacency lists, and its constant-time edge lookup is as fast as it gets.

Implicit graphs

Some graphs are never stored at all. A maze drawn on a grid is a graph: each open square is a vertex, with an edge to each open square beside it. A program exploring the maze needs only a function that, given a square, returns its neighbors:

def grid_neighbors(maze, square):
    """The open squares up, down, left, and right of square, in a maze given as a list of strings."""
    row, col = square
    result = []
    for r, c in [(row - 1, col), (row + 1, col), (row, col - 1), (row, col + 1)]:
        if 0 <= r < len(maze) and 0 <= c < len(maze[r]) and maze[r][c] != "#":
            result.append((r, c))
    return result

More generally, many problems can be modeled as a graph of states, with an edge from one state to another if a single step leads from one to the other. In a sliding-tile puzzle, each arrangement of the tiles is a vertex, with an edge to each arrangement reached by sliding one tile. In a game, each position is a vertex, with edges for the legal moves. Such graphs can be enormous, far too large to build, but they don’t need to be built: a search can generate each state’s neighbors as it reaches the state. Graphs like these, defined by a rule for finding neighbors rather than stored in a data structure, are called implicit graphs. As long as a search needs only a way to find each vertex’s neighbors, the same search works on both kinds of graphs.

Source material for these notes

The introduction, the vocabulary of graphs, and the descriptions of edge lists, adjacency matrices, and adjacency lists are adapted from the CS 1 notes by Balkcom et al., chapter 18, “Graphs”. The examples of symmetric and asymmetric relationships and the costs of the representations draw on the CS 10 lecture notes developed by Drysdale, Bailey-Kellogg, and Pierson, “Relationships,” and the discussion of implicit graphs on “State space.”

Claude (Anthropic) helped adapt these sources, and drafted the Python code for adjacency lists, the figures, and the practice problems, which Balkcom then edited. The same material is covered in Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, in the chapter on elementary graph algorithms, and in Sedgewick and Wayne, Algorithms, section 4.1.

Further reading

• graphlib: Python’s standard library module for ordering the vertices of a directed acyclic graph
• NetworkX: a widely used Python library for building and analyzing graphs

Practice problems

Try these on your own before checking the solutions.

Problem 1: Vocabulary

In the lettered graph at the start of these notes, what is the degree of I? List all the neighbors of B. Find a path of length 2 from A to I, and a cycle of length 3. Is the graph connected?

(Solution)

Problem 2: Representations

A directed graph has vertices W, X, Y, and Z, and edges $(W, X)$, $(W, Y)$, $(Y, X)$, $(Z, W)$, and $(Z, Y)$. Write its edge list, its adjacency matrix, with the vertices in the order W, X, Y, Z, and its adjacency lists. What are the in-degree and out-degree of Y? Is the adjacency matrix symmetric?

(Solution)

Problem 3: Choosing a representation

For each of these, would you choose an adjacency matrix or adjacency lists, and why? (a) A social network with a billion people, each with a few hundred friends. (b) The distances between every pair of 50 cities, stored so that any distance can be looked up quickly. (c) A graph whose vertices are added and removed often.

(Solution)

Problem 4: Degrees and edges

Explain why, in any undirected graph, the sum of the degrees of all the vertices is exactly $2m$. Use this to show that every undirected graph has an even number of vertices of odd degree.

(Solution)

Problem 5: Directed edges

Modify the Graph class to represent a directed graph, and add a method in_degree(v) that returns the number of edges entering v. How long does your in_degree take? How could you store more information to make it take constant time?

(Solution)