Dartmouth College Computer Science
Technical Report series
TR search TR listserv
|By author:||A B C D E F G H I J K L M N O P Q R S T U V W X Y Z|
|By number:||2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, 1998, 1997, 1996, 1995, 1994, 1993, 1992, 1991, 1990, 1989, 1988, 1987, 1986|
Traditional approaches to rootkit detection assume the execution of
code at a privilege level below that of the operating system kernel,
with the use of virtual machine technologies to enable the detection
system itself to be immune from the virus or rootkit code. In this
thesis, we approach the problem of rootkit detection from the
standpoint of tracing and instrumentation techniques, which work from
within the kernel and also modify the kernel's run-time state to
detect aberrant control flows. We wish to investigate the role of
emerging tracing frameworks (Kprobes, DTrace etc.) in enforcing
operating system security without the reliance on a full-blown virtual
machine just for the purposes of such policing. We first build a novel
rootkit prototype that uses pattern-searching techniques to hijack
hooks embedded in dynamically allocated memory, which we present as a
showcase of emerging attack techniques. We then build an intrusion
detection system-- autoscopy, atop kprobes, that detects anomalous
control flow patterns typically exhibited by rootkits within a running
kernel. Furthermore, to validate our approach, we show that we were
able to successfully detect 15 existing Linux rootkits. We also
conduct performance analyses, which show the overhead of our system to
range from 2% to 5% on a wide range of standard benchmarks. Thus by
leveraging tracing frameworks within operating systems, we show that
it is possible to introduce real-world security in devices where
performance and resource constraints are tantamount to security
M.S. Thesis. Advisor: Sean W. Smith.
Bibliographic citation for this report: [plain text] [BIB] [BibTeX] [Refer]
Or copy and paste:
Ashwin Ramaswamy, "Autoscopy: Detecting Pattern-Searching Rootkits via Control Flow Tracing." Dartmouth Computer Science Technical Report TR2009-644, May 2009.
Notify me about new tech reports.
Search the technical reports.
To receive paper copy of a report, by mail, send your address and the TR number to reports AT cs.dartmouth.edu
Copyright notice: The documents contained in this server are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.
Technical reports collection maintained by David Kotz.