Research Statement

Carrie Gates

A major challenge facing the computer and communications security community is determining algorithms for detecting security events using only network data. A security event is defined here not as an attempted attack, but as a successful attack (resulting in a compromised host, for example). One of the requirements for detecting security events using network data is the ability to properly evaluate and compare similar detectors. For example, current practice does not include stating the conditions under which one particular detector will perform better than another. As a result the best approaches to solving particular network security problems are not known. My research therefore focuses on the development and evaluation of network security approaches.


Network Security


My general interest in network security is in determining what can be detected given primarily connection information. I work with limited data, such as flow data, because I am particularly interested in very large networks ( e.g., international or government organizations), which often cannot collect individual packet header information due to the volume of traffic passing through them. I am working on two projects in this area: scan detection and the identification of compromised hosts.


Scans have been identified as preceding approximately 50\% of attacks[6], and have also been used for detecting particular types of worms[7]. Current approaches to scan detection require packet-level information, bidirectional data or knowledge of the network. One of the better detection methods is the Threshold Random Walk (TRW)[1], which requires either the ability to see replies to connection requests or knowledge of what hosts are on a network. In contrast, we have designed a system, which is currently in operational use at a large organization, based solely on unidirectional flow information. One of the challenges that this approach still faces is the ability to detect stealthy scans, such as scans that are very slow or that are targeted only against existing hosts. I am interested in detecting such scans as an extension of my PhD dissertation, which focused on detecting distributed scans.


In addition to detecting scans, which can be used to determine potential adversary targets, it is also important to identify compromised hosts. I have developed an initial prototype of a system that detects anomalous network activity on a per host basis by comparing current activity to a baseline. This approach shows promise, but requires further investigation. Open issues include determining the best methods for baselining and the ideal time scale, how the baseline could or should be updated over time, and how far from the baseline is considered an acceptable deviation. In addition, studies will need to be performed to determine the actual true and false positive rates achieved by this approach, along with user studies to determine if this rate is acceptable in practice.


Evaluation Methodologies


There are two approaches generally used to evaluate new network security techniques, the first of which uses a data set specifically developed for testing and comparing anomaly-based intrusion detection systems[2]. However, this data set suffers from several flaws[3][5] and is now over five years old. The second approach uses network traces gathered from live networks to test detectors. However, this approach suffers from a lack of ground truth --- we do not know where all of the attacks are. Also, results obtained in one network environment do not


necessarily reflect the results that would be obtained given a different environment[4].


For my dissertation I developed a new evaluation methodology based on logistic regression modeling of the true positive rate of a detector. Ground truth was obtained by performing the attacks using the DETER network (http://www.isi.edu/deter)[8], which were combined with traffic gathered from live networks. Key characteristics that were suspected to contribute to the true and false positive rates were identified and used to design a representative data set for training a logistic regression model of the detector's true positive rate. The model was tested using a second data set constructed using a similar process. Additionally, a third data set was constructed where the background noise was gathered from networks that had not been used in the training or testing process. This evaluation methodology was also used on a second detector, demonstrating how the logistic regression equations can be used to compare two detectors in terms of the conditions under which each performs best.


Future research on this methodology includes determining if this approach can be more generally applied. It needs to be tested against multiple detectors of different types and complexities in order to determine if it can be generalized. A more generic approach or set of guidelines will follow from these tests, so that others can apply this methodology to their own detectors. The end goal is to develop a methodology that, when applied to a single detector, provides a model that indicates the variables that are important to the performance of the detector. Further, the model will indicate what the expected true and false positive rates would be for this detector when applied in new environments.


References


  1. Jaeyeon Jung, Vern Paxson, Arthur W. Berger, and Hari Balakrishnan. Fast portscan detection using sequential hypothesis testing. In Proceedings of the 2004 IEEE Symposium on Security and Privacy, pages 211-225, Oakland, California, USA, 2004. IEEE Computer Society. May 9-12, 2004.
  2. Richard P. Lippmann, David J. Fried, Isaac Graf, Joshua W. Haines, Kristopher R. Kendall, David McClung, Dan Weber, Seth E. Webster, Dan Wyschogrod, Robert K. Cunningham, and Marc A. Zissman. Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation. In DARPA Information Survivability Conference and Exposition, volume 2, pages 12-26, 2000.
  3. Matthew V. Mahoney and Philip K. Chan. An analysis of the 1999 DARPA/Lincoln Laboratory evaluation data for network anomaly detection. In Proceedings of the Sixth International Symposium on Recent Advances in Intrusion Detection, pages 220-237, Pittsburgh, PA, USA, 2003. September 8-10, 2003.
  4. Roy A. Maxion and Kymie M.C. Tan. Benchmarking anomaly-based detection systems. In Proceedings of 2000 International Conference on Dependable Systems and Networks, pages 623-630, 2000. June 25-28, 2000.
  5. John McHugh. Testing intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory. ACM Transactions on Information and System Security, 3(4):262-294, 2000.
  6. Susmit Panjwani, Stephanie Tan, Keith M. Jarrin, and Michel Cukier. An experimental evaluation to determine if port scans are precursors to an attack. In Proceedings of the 2005 International Conference on Dependable Systems and Networks, pages 602-611, Yokohama, Japan, 2005. June 28-July 1, 2005.
  7. Nicholas Weaver, Stuart Staniford, and Vern Paxson. Very fast containment of scanning worms. In Proceedings of the 13th USENIX Security Symposium, pages 29-44, San Diego, CA, 2004. August 9-13, 2004.
  8. Brian White, Jay Lepreau, Leigh Stoller, Robert Ricci, Shashi Guruprasad, Mac Newbold, Mike Hibler, Chad Barb, and Abhijeet Joglekar. An integrated experimental environment for distributed systems and networks. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation, pages 255-270, Boston, MA, USA, Dec 2002. USENIX Association.