Monday, January 16, 2006

Network Traffic Analysis

Sometime back I read the book, "Intrusion Signatures and Analysis" (by Mark Cooper, Stephen Northcutt, Matt Fearnow, Karen Frederick). I was very much impressed by the approach authors has devised with the crucial loopholes in typical analysis techniques. They divided the work into following parts:
  1. Probability the source address was spoofed.
  2. Description of attack
  3. Attack Mechanism
  4. Correlations
  5. Evidence of the active Targeting
  6. Severity
  7. Defence Recommendations
Their are enough examples given with most crucial threats analysis. A must read for network security guys.
Sometime then, I was thinking that one can even use IDS engine over these traffic dump to get the alert data and then work over alerts BUT with complete ruleset of snort alert logfile is so huge, that situation is equally combursome as if someone analyzing the traffic dump. [with additional chances of missing new attacks if any since snort is an pattern matching engine]. Some say, why not use correlation engines to decrease the amount of work tobe done on the alert data. But what about the attacks/intrusions which are new (not caught by snort, ofcourse they are very few but new vulnerabilities always keep coming :-().
Ofcourse there are anomaly detection tools like Lancope etc. (even snort has preprocessor plugin namely spade for that) but some issues exists with these tools also.

Recently, I read this article "Structured Traffic Analysis" in (IN)SECURE magazine by Richard Bejtlich (october 2005 issue). The article is simply superb describing 13 step procedure to analyze traffic dump using a lot of "simple" opensource tools including tcpdstat, argus etc. These steps mostly includes generating traffic statistics from various perspectives including traffic protocol distribution, total number of packets, session analysis, IP informations etc. And in the last the snort was used for further analysis of alerts (optional step).
But what was really nice about it was that, it is an approach of "unsupervised anomaly detection techniques" with simple tools in simple steps. May be u can add more tools like tcptrace etc. to get more information but at the abstract level, this was kind of "offline (passive) traffic analysis" to detect anomalous traffic in dump capture. Anomaly detection techniques deploy machine learning/data mining approaches on traffic dump getting stats for the "feature set" from the data. Commonly used "features" are:
  1. no. of distinct sessions created in a time window.
  2. protocol distribution
  3. no. of ack/syn packets
etc. There's a large list of feature set which can be found in research papers. Author here has analyzed few of them very simply, with easily available, known tools (most of them).



Blogger Richard Bejtlich said...


I'm glad you liked my Structured Traffic Analysis article. I use that methodology any time I need to investigate a network trace of decent size.

6:39 AM  
Blogger Crazy Dan said...

Just thought I would stop by and say "Hello Nakul." It's been a hectic
but very worthwhile last few days for me. In searching for more intrusion detection methods related info on the Internet, I came across your site. I appreciate your content and I really appreciate your this post! It's been a great help in collecting more info on intrusion detection methods. Thanks again and have a great day!

10:14 AM  

Post a Comment

<< Home