Go Premium for a chance to win a PS4. Enter to Win


Packet Capturing/ Memory Mapping Efficiency Question

Posted on 2004-09-29
Medium Priority
Last Modified: 2010-04-15
Ok this maybe should have been a 2 parter but they are directly related.

I am using snort to grab packet off the wire, now Ive also developed my own packet grabbing program.
I also have a couple more programs I developed that need the raw packet. Basically I was wondering if there is additonal overhead with having mulitple programs grab packets using libpcap. Or if I am better of using my packet grabbing program grab the packet and pass it on to each of the addtional programs.Also is there a better (faster) way of grabbing packets off the wire other than writing my own driver? (though I would be interested in that as well :) )

I was already looking into this now here is my 2nd part. I am assuming memory mapping would be the fastest way to make the packet in its raw form available to each of the 3 programs. Every example I keep reading talks about using a file  where the changes take place in memory but are then put in the file with munmap. Now all I want to do is basically have the packet in its raw for put into memory I dont see any need for a file at all then I would have each of the 3 programs grab that region of memory. Is this the best way to go about it and in which context would I use memory mapping?

I guess Ill just post up the point and spread them out for right answers or if someone knows all this I can drop them in one spot.
Question by:joele24
LVL 46

Expert Comment

by:Kent Olsen
ID: 12185679

Hi joele24,

I use memory mapping a lot.  For many of the applications that I build it solves a lot of problems.

In your case I'm not sure that this is the best.  First of all, if you're planning to share a memory region, then all of the processes that are to share the region must be built (written and compiled) to explicitly do that.  If you've got the source to all 3 modules and want to share memory between them, you might be better served building them into a multi-threaded application where they all share the same memory anyway.

I'm betting that you don't have the source to all of these modules.  If not, you might be able to pipe the data between the modules, but that too requires special considerations.  The module that you're trying to latch onto must transmit its data via pipes AND run in such a way that you can redirect the output.

Hmmm.....  It's all possible, but the devil's in the details.  What's your access to source for all of these modules?



Author Comment

ID: 12191165
I have access to all the source. I wrote all the programs myself except Snort and thats an open source IDS. I had thought about threading. My only problem is that Snort source is under the GPL license so if I use that source in my code then I have to GPL my code as well and I do not wish to do that.

So that why I was considering mmap the raw packet. I figures since Ive already grabbed it what would be the fastest way to get it to Snort.

I actually was playing around with mmap'ing last night and I edited my lilo.conf so that I took some memory away from the system and made it only shared and I was able to do some stuff. The only problem is I dont want the user to have to edit there lilo.conf to make memroy changes so I am thinking of using the other method of mmap with file descriptors. But I dont really understand the performance increase by doing it this way. I read it take some of the kernel interupts out but I dont really see how since you are now dependent on the filesystem.
LVL 46

Expert Comment

by:Kent Olsen
ID: 12191869

A kernel interrupt is generated every time the program references an address that's not in an active page.  The mmap() process, in overly simplistic terms, instructs the kernel that when an interrupt occurs for a range of addresses, the memory mapping is to occur so that the range of local addresses map to a particular physical address (which is the memory region that you're trying to share).

Since you've got source to all of these, can you try piping the data as a first cut?  It's not the fastest solution, but it should give you an easy starting point to test your applications.

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.


Author Comment

ID: 12194116
Actually my applications all run fine Im not testing them. Im just looking for alternatives to speed up the system and packet capturing in general.

Expert Comment

ID: 12300393
wire sniff and pass it on in raw form is the quickest way

Author Comment

ID: 12353314
Can yo uelaborate on that. I currently am wire sniffing and getting it passed in a raw form.I was actaully looking for a means of bypassing the kernel so that packets can be directlly mmap'd to user space.

Author Comment

ID: 12878734

still looking for an answer on this

Accepted Solution

modulo earned 0 total points
ID: 13101296
PAQed with points refunded (500)

Community Support Moderator

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

An Outlet in Cocoa is a persistent reference to a GUI control; it connects a property (a variable) to a control.  For example, it is common to create an Outlet for the text field GUI control and change the text that appears in this field via that Ou…
Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use for-loops in the C programming language.
The goal of this video is to provide viewers with basic examples to understand opening and reading files in the C programming language.
Suggested Courses

886 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question