Dating photos by decade

Dan Allen
CS 134 project milestone
dsallen@cs.dartmouth.edu

Introduction

Image recognition and classification are fields that have been advancing rapidly. Machine learning based techniques now make it possible for computers to "learn" what an object looks like, and decide whether an object belongs to a given class or not. Unlike much of previous work on this problem, I intend to focus not on single objects or classes of objects present in photos, but on other qualities which may hold information about the photo. For example, a type of film which was popular at one time but later became obsolete may leave clues that suggest when the photo was taken. In other words, I propose to see if a computer can "learn" what a decade looks like. One problem I expect to encounter while working on this project is a tendency for the lines between the decades to be considerably blurry - after all, photos are taken continuously and not all in one discrete batch each decade. For example, a photograph taken in 1949 might look more similar to one from 1951 than to one from 1940. My goal is to be able to take an arbitrary photo which I know to be from a specific decade, and have it classified as belonging to that decade. This is easy for most humans, usually a photo is not hard to date within a number of years even by the untrained eye. However, it is often more difficult for a human to accurately describe the date of a photo to the year, even a photo which he or she is familiar with. I want to see how difficult this problem is for machine learning.

Methods

In light of the likely problems outlined above, I have chosen to use a method based on Support Vector Machines and linear regression. The goal is to come up with a regression model that minimizes the regression error corresponding to how accurate the prediction is. Feature vectors are extracted from the photos. This has involved some degree of additional research, as it is a quite specific set of features which I want to extract. The goal is to overlook the subject of the photo and focus exclusively on visual cues which are present in the film on which the photo was taken and may hold clues as to the photo's age. To do this I am using color histograms. This feture vector ignores properties such as spatial relation, rotation and translation and presents only color-based information about the photo.(1) This is important for my project because of the wide variety of subjects depicted in the photos. Once the feature vectors are extracted, they will be mapped to a high-dimensional space (about 1000) This method was shown to give good linear seperability between training data in high dimensions.(1) In addition to the methods described in (1), I am implementing a linear regression model as outlined in (6) To write the code for this project, I am using a combination of Python including the mlpy library(2) the Python Imaging Library (3), and MATLAB. After reviewing the literature(5), I will also be implementing k-Nearest Neighbor (k-NN) based classification to supplement my findings.

Dataset

The dataset has been collected from the Google LIFE Image archive(4), containing millions of photographs spanning roughly 250 years and made available by decade. I have created training and test data sets for four decades: the 1940's, 1950's, 1960's, and 1970's. By the time I do my final testing, I intend to have two more decades of training sets, with about 100 images in each training set, and at least 100 test set images. These numbers may fluctuate somewhat before the project is complete. I have decided to narrow the scope of the photo set down to only color photos, as a large enough set of data is available, and I am not prepared to investigate the differences between color and monochrome photography.

Results

Results so far are not conclusive. Preliminary testing has been conducted on small-sized data sets and has not yielded results which seem to indicate that the methods employed have successfully learned the features and characteristics of a photograph based on its age. Much additional testing will be done along with modifications to the algorithm which will hopefully give the desired results.

Remaining Timeline

5-12 -- 5-19: More testing, implement second method (k-NN)
5-20 -- 5-31: Compare results, prepare final presentation and write-up
6-1: Final Presentation

References

1) Support vector machines for histogram-based image classification. Chapelle O, Haffner P, Vapnik VN. IEEE Trans Neural Netw. 1999;10(5):1055-64.

2) MLPY homepage
3) Python Imaging Library.

4) Google LIFE image archive

5) In Defense of Nearest-Neighbor Based Image Classification. Boiman, O.; Shechtman, E.; Irani, M. IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008.
6) Support Vector Machines for Classification and Regression.