Is There a 'Fair' Value for the Letter Tiles in Scrabble®?
Henry DanaherCOSC 134 - Machine Learning and Statistical Data Analysis
Spring 2012
Many people are familiar with the board game Scrabble, or have at least heard of it. But if you have played it a few times you are probably aware that certain tiles are generally more 'desirable' to draw than others. For instance, the X tile is worth twice as much as the V tile, but the X tile is usually easier to play. As a result, the X tile is a 'lucky' event for the player that drew it and is probably correlated with higher scores. However, it is probably not known exactly by how much the tiles are over- or under-valued. [1]
I plan to use machine learning methods to determine a set of tile values for the letters in Scrabble, such that the letter value accurately reflects its 'value' in obtaining a win. Currently the values are mostly determined by the frequency of the letters in prose, not by their observed utility in the game itself. This leaves some players feeling that modern Scrabble has become more and more dominated by the play of unusual words that exploit imbalances in the scoring system and that perhaps it is time to update the tile values.
Despite the fact that Scrabble is just a game, it is an ideal setting for the exploration of machine learning methods for several reasons. First, the randomness and incomplete information present create a high branching-factor (in other words, a high number of moves to analyse each turn) that makes it futile to approach Scrabble with brute-force or purely analytic methods. [2] Similarly, since the space of possible solutions to the tile-valuation problem is 26 dimensional (27 dimensional if one allows the 'blank' tiles to have a value), the number of possible solutions to the tile-valuation problem I have posed is extremely large. Next, each of the tiles value is ultimately determined by skilled play, and Scrabble is not by any means a 'solved' game, despite the fact that computers can compete or surpass the very best human opponents. [3] On top of all this, there is the somewhat arbitrary large collection of words which comprise the legal plays in Scrabble. These facts make it difficult to study Scrabble without simulating large amounts of skilled play.
Fortunately, there is a program called Quackle that is not only capable of rapidly simulating well-played games of Scrabble, but whose source code is also freely available and modifiable for purposes of my experiment. Games simulated by Quackle will comprise the data set on which my project is based. The Quackle program has already been successfully exploited for Scrabble research. [4]
As was stated above, it could be the case that drawing (or not drawing) a certain letter during a game of Scrabble is correlated with either winning or losing that game. In other words, one could rationally say before the game "I hope that I don't draw the Q during this game" or "I hope that I draw the X during this game." Adjusting the value of a tile to a higher value should make drawing it during the game more likely to result in a won game, and adjusting the value of a tile to a lower value should make drawing it during the game more likely to result in a loss.
This leads to the natural question: To what value would I need to set the letter Q, so that before the game begins, I neither favor nor disfavor the idea of drawing it at some unspecified point during the game? Call such a value a neutral value of the letter Q. It is clear that, due to the interactions between certain groups of letters, this number may depend on the values of the other tiles, so call it the neutral value of the letter Q with respect to these other values.
This leads to a further question: Is there a set of values for all of the letter tiles (and the blank tile) such that the above neutrality criterion applies to each of the tiles? In other words, does there exist a set of letter values such that all 27 numbers are neutral values? Of course, there will still exist favorable combinations of letters, and favorable orders in which to draw them. But the question of what a specific letter is 'worth' on average is interesting in itself.
To be precise, if we index each of the letters in the english alphabet with i, and then define λi to be the number of points we should be willing to bid to draw an instance of a letter corresponding to index i before the game, and then denote the collection of these λi by Λ, then the error equation we would like to minimize is simply:
E(Λ) = ∑φi|λi|,We seek Λ for which E(Λ) is minimal.
The most salient feature of the tile-evaluation problem is that we have a strong prior. In other words, decades of experience of Scrabble play in general and, since the 1980's, of strong computer play in particular, give us a good general idea of what tiles ought to be worth. Therefore, it makes sense to implement a Bayesian paradigm in order to exploit this substantive domain-specific knowledge. The method of Simulated Annealing also suggests itself as an interesting possibility, because of its suitability to finding solutions to problems with high numbers of parameters, and its special forte of nondeterministically searching large solution spaces.
I have the source code (in the C/C++ progamming language) of a program called Quackle that can quickly simulate skilled games of Scrabble. I need to develop a method for passing letter values as input to this simulator, then obtaining desired statistics of the games that Quackle simulates using these letter values. Specifically, at a minimum I would like to know the letter frequencies of tiles drawn by the winning player of each game, since this is the information I will use to adjust the letter values iteratively. My hope is that this will require no explicit programming in C/C++. At most I will probably have to create a few mex-files and include some system commands in my MATLAB code that will modify C header files between iterations of the learning algorithm. All programs I create entirely on my own will be written in MATLAB; these should include all the functions related to the machine learning aspect of the program.
Persons familiar with the way Quackle operates will be quick to point out that Quackle employs certain internal lookup tables involving 'leave' (tiles remaining on one's rack after a move) values and win percentages which are based on simulations using current tile values. It is possible that these cannot easily be adjusted during the learning process. Therefore their use will always represent a source of bias in the the tile valuation. Ultimately, to find truly exact values would require the compilation of new such tables based on simulations with new tile values. But in my opinion, meaningful results should still be possible without changing these parts of Quackle's operation.
Since the letters occur at different frequencies, one would expect a larger average difference in the number of letter tiles drawn by the winner and loser for letters that occur at a greater frequency in the game. In addition, the number of tiles drawn by the winner and loser of the game is usually different. Both of these make it reasonable to scale the adjustment made at each iteration of the learning algorithm to the relative frequency of letter tiles present in the set.
Due to the rich capabilities of the Quackle program, it is possible to modify several aspects of the experiment, thereby obtaining a more nuanced picture of what ultimately determines the value of letter tiles. In particular there is the possibility of performing learning on different dictionaries (OSPD 5 as well as SOWPODS). I am also considering running experiments where certain letters are held at fixed values, in order to observe interactions and potential linear relationships between certain letters.
4/17 - Simulate Quackle games with alternate tile values and observe results
4/24 - MATLAB-Quackle Interface completed
5/1 - Preliminary Simulations
5/8 - Milestone Submission