The Inner Circle

As many of you know I work at one of the smaller universities in the UK, we have about 12000 students. On our network we have 6500 student residences and over 15000 ports throughout the campus for staff and students. On average we pull 400mbs and push 150mbs to our JANET regional network (which we also run ;).

Now and again various department, such as Physics, like to stress our links by swapping mass quantities of research data, last week they helped us average 980mbs incomming for a good four hours (we have 1gbps). I think it’s important to stress that we are an academic university, we encourage research, we don’t place many restrictions on what staff computers can do on our network. At the end of the day we provide a high speed network to be used.

However based upon experience I think many people are under the impression we are clueless about what happens on our network, or are incapable of looking. Which, I’m afraid is wishful thinking. Now that doesn’t mean we are consiously aware of every single action that takes place on the network, anyone who knows how much data 400mbs is will tell you it’s impossible to comprehend, but that doesn’t mean we don’t have a record. So what do we have…

Proxy/Server Logs
This is by far the most obvious one, every client PC records logs, so by rule you should assume our servers do. Which they do. All web traffic is proxied through one of four http proxies, these record your IP address, the date/time and what webpage you visited. These are kept for a significant amount of time, and we are often asked deal with http based abuse (usually to Wikipedia). Our services that require log on credentials always record who logged in from where and when.

Network Identification
Many people think that if they plug a laptop into a random network port get an IP address that they have never had before and isn’t assigned to them that we don’t know what it is. This isn’t true, everything that can be plugged into our network has a number that is unique (or it’s ment to be). We can simply look up this number and see what other IP addresses it’s been, and then look through the above mentioned server logs for usernames.

Network Authentication
This one is up and coming, we intend to implement a system that allows us who can plug into our network and what level of access they get when they do. I won’t go into how this works because we haven’t decided yet, suffice to say every device will be marked against a person and they will be responsible for it.

NetFlow
Here’s the big one for us, we fully know that the internet is not just webpages. There’s things like eDonkey and BitTorrent for doing peer2peer, there’s IRC, ICQ or Jabber for talking. You can watch movies in QuickTime or RealPlayer that come in live. You can make phone calls with Skype, play games on an Xbox or PC. We can’t proxy these things, so how do we know that they have happened.

Every ISP worth it’s salt, be they Pipex, BT, Zen, Bulldog, Tiscali, JANET, the university all use the same method (and more) to ensure they have records. All large scale routers, we’re talking the ones worth £50k+ not at £40 Belkin or Buffalo (although OSS firmware often has it for them) are capable of recording Network Flows.

Network Flows are records of communications on the internet, they don’t include the content of the communication, they just confirm that it took place. They record your IP address, the IP address of the server, the port numbers involved, the date/time, the number of packets, the number of bytes and a few other more technical pieces of information.

Yeasturday we recorded 1.5Gb of NetFlows which equates to about 30 million records, which our system thinks is 150Gb of data and that’s it not running at 100%.

Mirror Ports
Under the RIPA and the computing AUP the university has rights to ensure that it’s network is being used for purposes that do not break it’s AUP and diagnose network problems. Because of this we have ports coming off of our equipment that provides a complete copy of all data flowing through it. These are plugged into servers that can be used to search through the data and identify different flows in real time. We can also uses these mirror ports for running intrusion detection systems.

What else?
Having written this I try to think whatelse we do to ensure that we have some form of a record of what users do, I’m sure there is something, but I can’t think at the moment. It is exceptionally important to point out that access to all these different pieces of data is controlled heavilly to ensure that it can not be misused, and is not available to the majority of staff at the university.

For example mirror ports and network flows are only available to networking team leaders and the security team, server logs are usually only available to the admin team of that server, network identification records are available to all members of the technical groups as they do not contain sensitive or identifying information. Everyone with access to this type of data takes it very seriously as misuse is a serious incident and potentially illegal.

As I personally administrate the mirror servers and network flow server I can say that I take every care to ensure that the server is kept secure and access is limited to those that have a geniune need.

I hope this has opened a few peoples minds, but not frightened or worried them about big brother. It’s also worth writting that any views stated are my own and not thoose of my employer.

D.