Solved

Packet Capturing/ Memory Mapping Efficiency Question

Posted on 2004-09-29
10
250 Views
Last Modified: 2010-04-15
Ok this maybe should have been a 2 parter but they are directly related.

I am using snort to grab packet off the wire, now Ive also developed my own packet grabbing program.
I also have a couple more programs I developed that need the raw packet. Basically I was wondering if there is additonal overhead with having mulitple programs grab packets using libpcap. Or if I am better of using my packet grabbing program grab the packet and pass it on to each of the addtional programs.Also is there a better (faster) way of grabbing packets off the wire other than writing my own driver? (though I would be interested in that as well :) )

I was already looking into this now here is my 2nd part. I am assuming memory mapping would be the fastest way to make the packet in its raw form available to each of the 3 programs. Every example I keep reading talks about using a file  where the changes take place in memory but are then put in the file with munmap. Now all I want to do is basically have the packet in its raw for put into memory I dont see any need for a file at all then I would have each of the 3 programs grab that region of memory. Is this the best way to go about it and in which context would I use memory mapping?

I guess Ill just post up the point and spread them out for right answers or if someone knows all this I can drop them in one spot.
0
Comment
Question by:joele24
10 Comments
 
LVL 45

Expert Comment

by:Kdo
ID: 12185679

Hi joele24,

I use memory mapping a lot.  For many of the applications that I build it solves a lot of problems.

In your case I'm not sure that this is the best.  First of all, if you're planning to share a memory region, then all of the processes that are to share the region must be built (written and compiled) to explicitly do that.  If you've got the source to all 3 modules and want to share memory between them, you might be better served building them into a multi-threaded application where they all share the same memory anyway.

I'm betting that you don't have the source to all of these modules.  If not, you might be able to pipe the data between the modules, but that too requires special considerations.  The module that you're trying to latch onto must transmit its data via pipes AND run in such a way that you can redirect the output.


Hmmm.....  It's all possible, but the devil's in the details.  What's your access to source for all of these modules?

Kent

0
 

Author Comment

by:joele24
ID: 12191165
I have access to all the source. I wrote all the programs myself except Snort and thats an open source IDS. I had thought about threading. My only problem is that Snort source is under the GPL license so if I use that source in my code then I have to GPL my code as well and I do not wish to do that.

So that why I was considering mmap the raw packet. I figures since Ive already grabbed it what would be the fastest way to get it to Snort.

I actually was playing around with mmap'ing last night and I edited my lilo.conf so that I took some memory away from the system and made it only shared and I was able to do some stuff. The only problem is I dont want the user to have to edit there lilo.conf to make memroy changes so I am thinking of using the other method of mmap with file descriptors. But I dont really understand the performance increase by doing it this way. I read it take some of the kernel interupts out but I dont really see how since you are now dependent on the filesystem.
0
 
LVL 45

Expert Comment

by:Kdo
ID: 12191869

A kernel interrupt is generated every time the program references an address that's not in an active page.  The mmap() process, in overly simplistic terms, instructs the kernel that when an interrupt occurs for a range of addresses, the memory mapping is to occur so that the range of local addresses map to a particular physical address (which is the memory region that you're trying to share).


Since you've got source to all of these, can you try piping the data as a first cut?  It's not the fastest solution, but it should give you an easy starting point to test your applications.


Kent
0
 

Author Comment

by:joele24
ID: 12194116
Actually my applications all run fine Im not testing them. Im just looking for alternatives to speed up the system and packet capturing in general.
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 3

Expert Comment

by:happythedog
ID: 12300393
wire sniff and pass it on in raw form is the quickest way
0
 

Author Comment

by:joele24
ID: 12353314
Can yo uelaborate on that. I currently am wire sniffing and getting it passed in a raw form.I was actaully looking for a means of bypassing the kernel so that packets can be directlly mmap'd to user space.
0
 

Author Comment

by:joele24
ID: 12878734
**bump**

still looking for an answer on this
0
 

Accepted Solution

by:
modulo earned 0 total points
ID: 13101296
PAQed with points refunded (500)

modulo
Community Support Moderator
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

An Outlet in Cocoa is a persistent reference to a GUI control; it connects a property (a variable) to a control.  For example, it is common to create an Outlet for the text field GUI control and change the text that appears in this field via that Ou…
Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
The goal of this video is to provide viewers with basic examples to understand and use pointers in the C programming language.
The goal of this video is to provide viewers with basic examples to understand opening and writing to files in the C programming language.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now