• C

Packet Capturing/ Memory Mapping Efficiency Question

Ok this maybe should have been a 2 parter but they are directly related.

I am using snort to grab packet off the wire, now Ive also developed my own packet grabbing program.
I also have a couple more programs I developed that need the raw packet. Basically I was wondering if there is additonal overhead with having mulitple programs grab packets using libpcap. Or if I am better of using my packet grabbing program grab the packet and pass it on to each of the addtional programs.Also is there a better (faster) way of grabbing packets off the wire other than writing my own driver? (though I would be interested in that as well :) )

I was already looking into this now here is my 2nd part. I am assuming memory mapping would be the fastest way to make the packet in its raw form available to each of the 3 programs. Every example I keep reading talks about using a file  where the changes take place in memory but are then put in the file with munmap. Now all I want to do is basically have the packet in its raw for put into memory I dont see any need for a file at all then I would have each of the 3 programs grab that region of memory. Is this the best way to go about it and in which context would I use memory mapping?

I guess Ill just post up the point and spread them out for right answers or if someone knows all this I can drop them in one spot.
Who is Participating?

Improve company productivity with a Business Account.Sign Up

moduloConnect With a Mentor Commented:
PAQed with points refunded (500)

Community Support Moderator
Kent OlsenData Warehouse Architect / DBACommented:

Hi joele24,

I use memory mapping a lot.  For many of the applications that I build it solves a lot of problems.

In your case I'm not sure that this is the best.  First of all, if you're planning to share a memory region, then all of the processes that are to share the region must be built (written and compiled) to explicitly do that.  If you've got the source to all 3 modules and want to share memory between them, you might be better served building them into a multi-threaded application where they all share the same memory anyway.

I'm betting that you don't have the source to all of these modules.  If not, you might be able to pipe the data between the modules, but that too requires special considerations.  The module that you're trying to latch onto must transmit its data via pipes AND run in such a way that you can redirect the output.

Hmmm.....  It's all possible, but the devil's in the details.  What's your access to source for all of these modules?


joele24Author Commented:
I have access to all the source. I wrote all the programs myself except Snort and thats an open source IDS. I had thought about threading. My only problem is that Snort source is under the GPL license so if I use that source in my code then I have to GPL my code as well and I do not wish to do that.

So that why I was considering mmap the raw packet. I figures since Ive already grabbed it what would be the fastest way to get it to Snort.

I actually was playing around with mmap'ing last night and I edited my lilo.conf so that I took some memory away from the system and made it only shared and I was able to do some stuff. The only problem is I dont want the user to have to edit there lilo.conf to make memroy changes so I am thinking of using the other method of mmap with file descriptors. But I dont really understand the performance increase by doing it this way. I read it take some of the kernel interupts out but I dont really see how since you are now dependent on the filesystem.
Get Cisco Certified in IT Security

There’s a high demand for IT security experts and network administrators who can safeguard the data that individuals, corporations, and governments rely on every day. Pursue your B.S. in Network Operations and Security and gain the credentials you need for this high-growth field.

Kent OlsenData Warehouse Architect / DBACommented:

A kernel interrupt is generated every time the program references an address that's not in an active page.  The mmap() process, in overly simplistic terms, instructs the kernel that when an interrupt occurs for a range of addresses, the memory mapping is to occur so that the range of local addresses map to a particular physical address (which is the memory region that you're trying to share).

Since you've got source to all of these, can you try piping the data as a first cut?  It's not the fastest solution, but it should give you an easy starting point to test your applications.

joele24Author Commented:
Actually my applications all run fine Im not testing them. Im just looking for alternatives to speed up the system and packet capturing in general.
wire sniff and pass it on in raw form is the quickest way
joele24Author Commented:
Can yo uelaborate on that. I currently am wire sniffing and getting it passed in a raw form.I was actaully looking for a means of bypassing the kernel so that packets can be directlly mmap'd to user space.
joele24Author Commented:

still looking for an answer on this
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.