Balanced search trees

A hash table implements a map in constant time on average, but it keeps its keys in no particular order. Many problems need the order. A calendar program asks for every event between 2pm and 3pm. A leaderboard asks for the ten highest scores. A spell checker that finds no match for a word asks for the nearest words alphabetically. A database asks for every customer whose last name falls between “Smith” and “Smyth”. None of these can be answered by a hash table without examining every key.

A search tree stores keys in sorted order and still finds, inserts, and removes a key quickly, as long as the tree stays short and bushy. These notes describe the binary search tree, show how it can grow long and thin and lose its speed, and then describe the 2-3 tree, which keeps itself perfectly balanced by splitting nodes. The red-black tree, the balanced tree inside Java’s TreeMap and C++‘s std::map, is a 2-3 tree written with binary nodes, and the B-tree, the index inside nearly every database, is a 2-3 tree with large nodes.

The ordered map ADT

An ordered map supports the operations of a map, put(key, value), get(key), contains(key), and remove(key), and also operations that depend on the order of the keys:

Operation	Returns
min(), max()	The smallest or largest key
floor(key), ceiling(key)	The largest key at most key, or the smallest key at least key
keys()	Every key, in sorted order
range(lo, hi)	Every key between lo and hi, in sorted order

The keys of an ordered map must be comparable with <. That is the ordered map’s counterpart of a hash table’s requirement that its keys be hashable.

A Python list kept in sorted order is one implementation. Binary search finds a key in $O(\log n)$ time, and min, max, and range are easy. Inserting a key, though, requires shifting every larger key one slot to the right to open a gap, which takes $\Theta(n)$ time in the worst case, and removing a key requires shifting them back. A binary search tree keeps the speed of binary search without the shifting.

Binary search trees

A binary search tree is a binary tree in which each node holds a key and a value, and the keys obey the binary search tree property: for every node, all of the keys in its left subtree are smaller than its key, and all of the keys in its right subtree are larger. These notes assume that the keys are distinct.

The two trees below hold the same seven keys, and both have the binary search tree property. Each key’s value is not shown. The shape of a binary search tree depends on the order in which its keys were inserted, so the same keys can form many different trees.

In Python, a node holds a key, a value, and references to its two children, either of which may be None. The tree itself holds a reference to its root:

class Node:
    def __init__(self, key, value):
        self.key = key
        self.value = value
        self.left = None
        self.right = None

class BSTMap:
    def __init__(self):
        self.root = None

Searching a binary search tree

The binary search tree property tells a search which way to go at every node. Compare the key being searched for with the node’s key. If they are equal, the search is over. If the key is smaller, it can only be in the left subtree; if it is larger, it can only be in the right subtree. Each comparison discards one whole subtree, as each step of binary search discards half of a sorted list.

    def get(self, key):
        node = self.root
        while node is not None:
            if key == node.key:
                return node.value
            elif key < node.key:
                node = node.left
            else:
                node = node.right
        raise KeyError(key)

The figure below shows the path of a search for C in the first tree: C is less than D, so go left; C is greater than B, so go right; and there is C. If the key were not in the tree, the search would fall off the bottom of the tree, at the None where the key would have been.

Inserting into a binary search tree

To insert a key, search for it. If the search finds the key, replace its value. Otherwise the search falls off the tree at some None, and that is where the key belongs: a later search for the key will follow the same path and find it there. A recursive helper returns the root of the subtree after the insertion, which lets each parent reattach its child:

    def put(self, key, value):
        self.root = self._put(self.root, key, value)

    def _put(self, node, key, value):
        if node is None:
            return Node(key, value)
        if key == node.key:
            node.value = value
        elif key < node.key:
            node.left = self._put(node.left, key, value)
        else:
            node.right = self._put(node.right, key, value)
        return node

Every new key is added as a leaf. The nodes that are already in the tree never move.

Ordered operations on a binary search tree

The smallest key in a binary search tree is at the end of the path that always goes left, since every step to the right leads to larger keys. The largest key is at the end of the path that always goes right.

Visiting the left subtree, then the node, then the right subtree lists every key in sorted order. This visiting order is called an in-order traversal. A range query is an in-order traversal that skips any subtree that cannot hold a key in the range, and floor and ceiling follow a single path down the tree, as a search does.

Removing a key also follows one path down the tree. It takes more care than insertion when the node to be removed has two children, since the node’s place must be filled by another key that keeps the binary search tree property; the usual choice is the smallest key in the node’s right subtree.

The height of a binary search tree decides its speed

Search, insertion, removal, min, max, floor, and ceiling each follow one path down from the root, so each takes time proportional to the length of that path. In the worst case, that is the height of the tree, the number of edges on the longest path from the root down to a leaf.

A binary tree of height $h$ holds at most $1 + 2 + 4 + \cdots + 2^h = 2^{h+1} - 1$ nodes, so a tree with $n$ keys has height at least $\log_2(n + 1) - 1$. A tree that achieves this, with every level full, is perfectly balanced, and every operation on it takes $O(\log n)$ time. A perfectly balanced tree with a million keys has height 19.

The insertion algorithm does nothing to keep the tree balanced, though. Inserting the keys A through G in alphabetical order makes each new key the right child of the one before, and the tree becomes a vine: a linked list, of height $n - 1$, in which every operation takes $\Theta(n)$ time.

Sorted input is not a rare accident. Keys often arrive in order: timestamps, ID numbers handed out one after another, or the lines of a file that was sorted before it was read. Keys that arrive in random order give a tree whose average depth is about $1.39 \log_2 n$, which is fine, but a data structure whose speed depends on the luck of its input is risky.

Perfect balance costs too much

One fix would be to rebuild the tree after each insertion into a perfectly balanced shape. That costs too much. The tree on the left below is as balanced as six keys allow. Inserting 10 and rebalancing perfectly gives the tree on the right, in which no parent-child pair from the old tree survives. Every node moved, and a sequence of insertions can force a rebuild like this one over and over, at a cost of $\Theta(n)$ per insertion.

Balanced search trees escape by relaxing one of two words in “perfectly balanced binary tree.” AVL trees and red-black trees give up perfectly: they allow some imbalance, but bound it, so that the height stays $O(\log n)$. 2-3 trees and B-trees give up binary: they let a node hold more than one key, and use that flexibility to keep every leaf at the same depth.

2-3 trees: perfect balance with nodes of two sizes

A 2-3 tree is a search tree with two kinds of nodes:

• A 2-node holds one key and has two children, as in a binary search tree.
• A 3-node holds two keys, a and b, with a < b, and has three children: the left subtree holds the keys less than a, the middle subtree the keys between a and b, and the right subtree the keys greater than b.

(A leaf has no children, but it is still called a 2-node or a 3-node by the number of keys it holds.) The rule that makes a 2-3 tree balanced is: every leaf is at the same depth.

Searching a 2-3 tree is a search in a binary search tree with an extra comparison at each 3-node: compare the key with both of the node’s keys, and follow the left, middle, or right child.

Inserting into a 2-3 tree by splitting nodes

Insertion begins as in a binary search tree, by searching down to the leaf where the key belongs, and then adds the key to that leaf:

• If the leaf is a 2-node, it becomes a 3-node, and the insertion is done.
• If the leaf is a 3-node, it now holds three keys, which is one too many. Split it: the middle key moves up into the parent, and the smallest and largest keys become two separate 2-nodes, the children on either side of the middle key in the parent.

Moving a key into the parent may give the parent three keys. It splits in turn, passing its middle key up to its own parent, and the splits continue up the tree until they reach a node with room. If the root splits, its middle key becomes a new root, with one key, and the tree grows one level taller.

Insertion never adds a level anywhere except at the top. Every leaf stays at the same depth, since a split creates new nodes beside the old one, at the same level, and a new root moves every leaf one level deeper at once.

Inserting the keys 1 through 7 in order shows the contrast with a binary search tree. 1 and 2 go into one leaf. 3 makes the leaf overflow, and it splits:

4 joins the leaf holding 3. 5 overflows that leaf, which splits and sends 4 up to the root, which becomes the 3-node 2 4. 6 joins the leaf holding 5. Inserting 7 causes two splits, one at the leaf and one at the root, and the tree grows a level:

The result is a perfectly balanced tree of height 2, the same keys that make a binary search tree into a vine of height 6.

Why a 2-3 tree has logarithmic height

Every leaf of a 2-3 tree is at the same depth $h$, and every node above the leaves has at least two children. The tree therefore has at least as many nodes as a perfectly balanced binary tree of height $h$, $2^{h+1} - 1$, each with at least one key, so

$n \geq 2^{h+1} - 1, \qquad h \leq \log_2(n + 1) - 1.$

A 2-3 tree with a million keys has height at most 18, however the keys were inserted, so a search makes at most 19 visits to nodes, and at most two comparisons at each. Insertion follows one path down and splits at most one node per level on the way back up, so it takes $O(\log n)$ time as well.

The price of this balance is in the code. Nodes of two sizes, and the splits between them, make an implementation of a 2-3 tree long and full of special cases. The red-black tree is a way to get the same balance from binary nodes.

Removal from a 2-3 tree reverses insertion. When removing a key leaves a node empty, the node borrows a key from a neighboring sibling or merges with it, and a merge can pass the problem up to the parent; removal also takes $O(\log n)$ time.

Red-black trees: 2-3 trees written with binary nodes

In 1978, Leonidas Guibas and Robert Sedgewick observed that a 2-3 tree can be stored as a binary search tree. A 2-node is already a binary node. A 3-node holding a and b is written as two binary nodes: b, with a as its left child. The link from b down to a is colored red, to mark that the two nodes are halves of one 3-node; every other link is black. This version, in which red links always lean left, is called a left-leaning red-black tree.

Ignoring the colors, a red-black tree is an ordinary binary search tree, so searching and the ordered operations work without change. Insertion and removal carry out the splits, borrows, and merges of the 2-3 tree as small rearrangements of binary nodes, called rotations, and changes of link colors. A path through the red-black tree is at most about twice as long as the corresponding path through the 2-3 tree, so its height is $O(\log n)$. Java’s TreeMap and C++‘s std::map are red-black trees. Theirs encode 2-3-4 trees, in which a node may hold up to three keys, but the idea is the same.

Other balanced search trees, including AVL trees, treaps, and splay trees, keep the height $O(\log n)$ with different rules, and they also rebalance with rotations.

B-trees: 2-3 trees for disks and databases

A B-tree generalizes the 2-3 tree to nodes holding many keys. In a B-tree of order $d$, each node has between $\lceil d/2 \rceil$ and $d$ children; a 2-3 tree is a B-tree of order 3. Insertion splits a full node into two half-full nodes and passes the middle key up, and removal borrows and merges, as in a 2-3 tree, and every leaf is at the same depth.

B-trees are designed for data stored on a disk or SSD, which is read a block at a time. Reading a block takes far longer than comparing keys in memory, and a block holds thousands of bytes, while a key takes a few. A binary search tree stored on disk would read a whole block to use one key from it at each level. A B-tree whose nodes are the size of a block uses all of it. With $d = 1000$, a B-tree with a trillion ($10^{12}$) keys has height about 4, compared with 39 or more for a binary tree with the same keys, and the top levels fit in memory, so a search reads one or two blocks from the disk.

The index of nearly every relational database, including PostgreSQL, MySQL, and SQLite, is a variant called a B+ tree, and so are the directories of many file systems, including NTFS, APFS, and btrfs.

Where balanced search trees are used

A balanced search tree is the standard implementation of an ordered map or set, and whenever a program needs a map and also needs its keys in order, a balanced search tree is the usual choice:

• Range queries and nearest keys: the events in a time window, the words with a given prefix (all words between "pre" and "prf"), or the next scheduled event after the current time.
• Order statistics: if each node also stores the size of its subtree, the tree finds the $k$th smallest key, or the rank of a key, in $O(\log n)$ time, for medians, percentiles, and leaderboards.
• A priority queue that allows any element to be removed or changed in $O(\log n)$ time, which a binary heap does not. The scheduler in the Linux kernel has long kept the runnable tasks in a red-black tree ordered by how much processor time each has received, so that it can find the task that has received the least, at the leftmost node, and also remove any task that stops running.
• Guaranteed worst-case time. Java’s HashMap turns any bucket that grows past eight keys into a red-black tree, so that keys chosen by an adversary to collide cost $O(\log n)$ per operation, not $\Theta(n)$.
• Intervals and geometry: interval trees, which answer “which intervals contain this point?”, and the sweep-line algorithms of computational geometry are built on balanced search trees.
• Databases and file systems, through B-trees.
• Immutable maps in functional languages such as Haskell, Clojure, and Scala. An insertion into a balanced tree copies only the $O(\log n)$ nodes on one path, so the old version of the map survives alongside the new one at little cost.

Python’s standard library has no balanced search tree. A Python program that needs an ordered collection usually keeps a sorted list and uses the bisect module for binary search, accepting the $\Theta(n)$ cost of insertion, or uses the third-party sortedcontainers package, which keeps a list of short sorted lists to make insertion fast in practice.

Comparing hash tables and balanced search trees

	Hash table	Balanced search tree
get, put, remove	$O(1)$ on average, $\Theta(n)$ in the worst case	$O(\log n)$ in the worst case
min, max, floor, ceiling	$\Theta(n)$	$O(\log n)$
All keys in sorted order	$\Theta(n \log n)$, by sorting	$\Theta(n)$, by in-order traversal
Requirement on keys	Hashable, with __eq__ and __hash__ agreeing	Comparable with <

When the order of the keys does not matter, a hash table is usually faster. When it does, or when the worst case must be bounded, a balanced search tree is the right structure.

Source material for these notes

The binary search tree, its search, insertion, and minimum, the analysis of balanced trees and vines, the example showing that perfect rebalancing moves every node, the division of balanced trees into those that give up “perfect” and those that give up “binary,” the encoding of multiway nodes as red-black binary nodes, the borrow and merge cases of removal, and the description of B-trees and their use on disk are adapted from the CS 10 lecture notes developed by Drysdale, Bailey-Kellogg, and Pierson, “Hierarchies 2” and “Balance,” for the earlier, Java-based version of CS 10. Those notes use 2-3-4 trees, whose nodes may hold up to three keys; these notes use 2-3 trees instead, which have fewer cases. The left-leaning encoding of a 2-3 tree follows Sedgewick and Wayne, Algorithms, 4th edition, section 3.3. Red-black trees were introduced by Guibas and Sedgewick, “A dichromatic framework for balanced trees” (1978), 2-3 trees by John Hopcroft in 1970, and B-trees by Rudolf Bayer and Edward McCreight in 1972.

Claude (Anthropic) helped adapt these sources to Python, and drafted the ordered map ADT, the BSTMap class, the 2-3 tree insertion examples, the section on where balanced search trees are used, the comparison with hash tables, the figures, and the practice problems, which Balkcom then edited. The same material is covered in Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, chapters 12, 13, and 18, and in Goodrich, Tamassia, and Goldwasser, Data Structures and Algorithms in Python, chapter 11.

Further reading

• bisect: Array bisection algorithm, for keeping a Python list in sorted order
• Sorted Containers: sorted lists, dictionaries, and sets for Python

Practice problems

Try these on your own before checking the solutions.

Problem 1: Build a binary search tree

Draw the binary search tree that results from inserting the keys 50, 30, 70, 20, 40, 60, 80, 35, and 45, in that order, into an empty tree. What is its height? List the keys in the order that an in-order traversal visits them.

(Solution)

Problem 2: Insertion orders

Give an order in which to insert the keys 1 through 7 so that the binary search tree is perfectly balanced. How many of the $7! = 5040$ insertion orders give a vine of height 6?

(Solution)

Problem 3: Build a 2-3 tree

Draw the 2-3 tree that results from inserting the keys 10, 20, 30, 40, 50, 60, 70, 80, and 90, in that order, into an empty tree, showing each split.

(Solution)

Problem 4: Heights of 2-3 trees

What are the smallest and largest numbers of keys that a 2-3 tree of height 2 can hold? What are the smallest and largest possible heights of a 2-3 tree with 80 keys?

(Solution)

Problem 5: Choosing a map

For each task, would you use a hash table or a balanced search tree? Explain briefly.

1. Counting how often each word appears in a book.
2. Finding all of the flights departing between 9am and noon.
3. Keeping the set of squares visited during a breadth-first search of a maze.
4. Reporting the median of a stream of numbers after each new number arrives.

(Solution)