Blue GNU got Jim Sansing to talk about the fairly new RenaissanceCore IDS project, and what they have been able to accomplish. It's certainly a project you might want to consider, especially if security is your gig.

WHEN AND WHY WAS RENAISSANCECORE STARTED?

We will never claim to be "God's gift to software development" (we are merely above average ;-). What we bring to the table is that we have used tools for network security monitoring, day in and day out, for several years in an enterprise environment. Therefore, we have a good understanding of what works in network Intrusion Detection Systems (IDS) and what doesn't. The biggest problem we, and teams we have cooperated with, face is false postives.

The best tool we have used allows the playback of entire sessions. The problem is, the amount of data to be handled limits what it can monitor. In 2001, we began developing inhouse middleware to collect the output of several different IDSes, and added other enhancements to capture more data. This provided us with a more comprehensive picture and we discovered that quite a bit was being missed.

Around 2002, there was an increase in both the number of different exploits and the number of times we saw the same exploit. Also, the sophistication of exploits increased. The response on the detection side was to increase the number of rules and to add some functionality, such as protocol anomaly detection. Although the existing products were catching many more exploits, we saw that they were still missing a lot of them as well.

Because the system we assembled was complicated and expensive, we realized that the best solution was to build a new sensor application that could analyze complete sessions. This would allow us to perform more complete and sophisticated automated analysis which would make the human analysts more productive. But when we proposed this to management, we were told that there was no money for its development.

So we started the project on our own in early 2003 (and yes, management is aware of our work and it has all been done on our own time and equipment). Our goal is to provide a network IDS that analyzes complete TCP sessions in high volume networks. In addition the IDS will analyze other traffic, such as UDP, in the context of a network's usage patterns. As an integral part of the system, it will provide network administrators with an overview of the traffic on their networks, to audit the effectiveness of their security measures and help them recognize questionable activity early.

WHERE DOES THE PROJECT STAND AT THIS POINT?

The project consists of an IDS application, a database (we use PostgreSQL), and a user interface (UI). The output of the IDS is sent to what we call the database daemon (DBD) to be inserted into the database. The IDS is written in C and the DBD and UI are written in Java. It is designed so that all components can be run on separate hosts using encrypted sessions, or the same host, or something in between.

We publicly announced the project recently and our first release version is 0.9. Last week I put 0.9.1 on SourceForge, which fixes a number of problems that our QA department (that would be me) didn't catch. The system works on small networks. My test machine is an 833 MHz Pentium with 256 Meg of memory running on my home network. A local college has graciously agreed to allow us to run a pilot in their network, so we will soon be learning how well it scales.

HOW MANY ACTIVE DEVELOPERS CURRENTLY WORK ON RENAISSANCECORE?

There are two of us, currently. I have 20 years of experience in computer networks, including 4 years of maintenance and development on a TCP/IP stack for IBM mainframes (written in assembler, oh yeah!). Paul has been a DBA for over 10 years, has written web based applications with Java servlets and PHP, and has been trained in computer forensics. We have talked to a number of people locally and some have made good suggestions, but unfortunately haven't had time to actively participate.

HOW DOES RENAISSANCECORE COMPARE (IN YOUR VIEW) TO OTHER PROJECTS?

We believe that we are making positive contributions to the state of network intrusion detection. For example, the sample ruleset we provide monitors access to education websites by looking for '.edu' in the GET request. But it also looks for 'About Harvard' and 'About Stanford' in the response from the web server. The captured data from each session detected by these rules can be displayed side by side in a window, where the matched strings (we call them Triggers) are highlighted in red. This demonstrates our capability for request and reply monitoring.

But we definitely realize we are building on the work of others. For instance, one of our features is a simplistic version of ntop. I believe that the information obtained from ntop is valuable, but it must be run on a host that has access to all network traffic. In many networks, that would be just the IDS sensor, and it may not be possible to run ntop on that host, We are already tracking statistics for each session, so we can provide at least basic information about port usage and selected host activity.

Having worked in enterprise environments, our design includes features that are useful there. A user may be an administrator or any of three analyst types, each with increasing levels of access. Sites, points of contact, and hosts must be recorded. Detected events can be saved in incident reports for trends analysis, and event definitions include a severity rating which can be changed in the incident report. There are a few reports defined and we will be adding the capability to export to problem tracking systems.

The rules are stored in the database, defined in the user interface, and exported to the sensors in XML files. There are three levels of definitions which need to be displayed together, and the UI makes this possible. We have tried to simplify management of rules by defining categories which are displayed in a tree with collapsible branches.

CAN YOU GIVE ME ANY IDEA OF HOW BIG AND/OR ACTIVE THE RENAISSANCECORE USER COMMUNITY IS?

We just recently made our first public announcement of the project. The first available version is 0.9. So far there have been about 100 downloads, but the upcoming pilot will be the only known deployment. We are in that Catch-22 where 'No community' = 'No credibility' = 'No community' ... If the pilot goes well, we should gain some credibility and that will hopefully attract interest.

WHAT NEEDS TO BE ACCOMPLISHED BEFORE THE NEXT RELEASE OF RENAISSANCECORE?

Our biggest concern right now is scalability. We have done what we can to test the system under load, but are our resources are limited. We will be releasing incremental versions of 0.9.x until we feel that it can be deployed in production environments. We will add a few minor features, but most updates will be bug fixes, formatting improvements, etc.

After that, there is rules development. For the pilot, I created a rule called 'Non-Web Traffic Using Port 80', which may produce some interesting results. We have a few dozen rules for specific exploits, but that is mostly for testing the system using Metasploit, which guarantees the exploit exists. So the pilot will also be an opportunity to experiment with rules.

Then there are the usual issues with packaging, documentation, etc. Because the system is a client/server application with a database and needs to be initialized, I created a Debian package with initialization scripts. This took about six weeks total, mainly because I have been running RPM systems (since 1996) and had to learn the Debian way of doing things. We are starting to work on a Fedora Core package, and I recently learned of the ESP Package Manager from the folks who do CUPS, and if that works for us, we will produce packages for several distros.

I have written quite a bit of doc along the way, and recently spent a day collecting it in an ODF document. But most of it is reference material that assumes you already understand how the system works and just need specific information, and much of that is available from the UI help menu. It will take some time to explain how to use the system effectively.

I am proud of the fact that every one of the error messages in the IDS is documented. I embed the message, description, and action as a comment and use doxygen to format them in several HTML pages. Unfortunately, this is only allowed in 3.x versions, so I expect to write an awk script to handle it, and will export them in ODF format.

WHAT ARE THE BIGGEST OBSTACLES THE RENAISSANCECORE TEAM FACES IN DEVELOPMENT?

We are probably no different from any other development group, even large ones, in that there never seem to be enough hours in the day. But our lack of network and hardware resources limit our testing and porting. We have run the system on Suse, Gentoo, and Debian in 32-bit and/or 64-bit mode.

IS THERE ANYTHING ELSE ABOUT RENAISSANCECORE YOU THINK OUR AUDIENCE SHOULD KNOW?

I designed the IDS application to be flexible by building it on top of an analysis engine library. This library is available for projects that need to do automated analysis of multiple, large streams of data. We have licensed all of our code under GPLv3, and the library is currently only available to be linked statically. If any FOSS projects are interested, I will be happy to work with them.

Also, I would like to recognize the many FOSS projects that make ours even possible. The obvious ones are the Linux kernel, GNU software (especially gcc), and PostgreSQL. But the following have been especially important:

- libpcap, libxml, openssl: Library developers just don't get enough credit for carrying the rest of us on their shoulders
- vim: Colorized syntax highlighting, line folding, and huge undo buffers, I could go on and on
- anjuta: Initializing the configure and Makefile alone was worth it, but it has a lot of other very helpful features
- eclipse: A well designed IDE that got our UI working quickly (can you tell I'm not the Java guy?)
- doxygen: A lot of people talk about documentation, Dmitri has done something about it
- Debian: Building a package to meet their standards has made our project much better

And last but not least, I would like to give 3 cheers to SourceForge. The very fact of their existence is great, but from a developer's point of view, they are miracle workers. About a year ago they had a severe hardware failure and CVS was down for a few days. They kept everyone updated on the situation and as far as I know, they did not lose any data saved up to the day of the outage. I have been through this in a shop with three projects and we lost several days worth of work. I can't even imagine handling over 100,000 projects. Add to that the opportunity for small projects to provide downloads, screenshots, and do a little advertising and you have a fantastic example of the best of FOSS.