Goals
I plan on enriching previous research on image analogies in order to learn generalizable filters. The goal of image analogies is to, given one or more training pairs of "non-filtered" and "filtered" images, learn a filter that transforms a each non-filtered image A to each filtered image A' and apply that same filter to given image B in order to get a new filtered image B'. When thinking about the problem conceptually, we imagine separate stages for learning and application of these filters, which is what I hope to achieve, but in Herzmann et al.'s implementation this is not the case.
The original image analogy research provides powerful results for specific, hand-tailored examples. However, the methods implemented by Herzmann et al. are nonparametric in nature, and thus limited in application, because large training sets are costlier at application time and the algorithm has trouble extrapolating its training data to apply to unfamiliar regions of pixels. I hope to use machine learning in order for the algorithm to actually function in separate learning and application phases, which will ideally create more generalizable filters that will behave better with small training sets in terms of estimating a filter, while managing larger training sets better at run-time by not having to search through the training set at render-time.
Method
This is essentially a regression problem. The core of my work will consist of innovating upon past research by experimenting with various regression techniques, such as local and least-squared regression, in order to learn local filters (learned from local regions of an image, as contrasted with uniform filters), by treating each pixel as a vector composed of information from its neighboring pixels. I also hope to explore further areas of regression not covered in this course, such as support vector regression.
Herzmann's image analogy research will offer significant guidance for specific problems within my project as well, such as gauging coherence within the filtered product image.
Data
An advantage to image analogies is that they can function on very small training sets, consisting of only one image pair. The internet has made finding unfiltered images incredibly easy, and my proficiency with photoshop should make the creation of filtered images equally simple. For these reasons, I expect data collection to be a negligible step.
Timeline
By the milestone I hope to have implemented the basic algorithm using linear and least-squares regression. The rest of the project will likely be largely spent on exploring more advanced forms of regression and solving the problem of coherence.
Sources
I have primarily focused on the original Image Analogies paper (Hertzmann, Jacobs, Oliver, Curless, Salesin) .
There also exist a number of examples that communicate the process and applications of image analysis on their project website