Investigating Measures for Pairwise Document Similarity

Dartmouth Technical Report PCS-TR99-357

	Jeffrey D. Isaacs
	Javed A. Aslam

Date: June 1999

URL (compressed postscript): <http://www.cs.dartmouth.edu/reports/TR99-357.ps.Z> (32KB)

URL (PDF): <http://www.cs.dartmouth.edu/reports/TR99-357.pdf> (32KB)

Abstract:
 The need for a more effective similarity measure is growing as a
 result of the astonishing amount of information being placed
 online. Most existing similarity measures are defined by empirically
 derived formulas and cannot easily be extended to new applications. We
 present a pairwise document similarity measure based on Information
 Theory, and present corpus dependent and independent applications of
 this measure. When ranked with existing similarity measures over TREC
 FBIS data, our corpus dependent information theoretic similarity
 measure ranked first. 

Note:
 Undergraduate Honors Thesis.  Advisor: Jay Aslam.