Solved

netcat nc -l reads data in socket too fast - slow it down

Posted on 2014-09-09
18
507 Views
Last Modified: 2016-07-15
I run a client program which outputs to a socket whose destination is same PC, port 4321. On the same VMware, I run a simulated server using netcat:
nc -lk 127.0.0.1 4321 > /dev/null

Open in new window

so that the program can connect to the nc listening command and write stuff to it. This all works fine. The internal output buffers in the program are nearly 0 since nc reads much faster than the program can produce data. The program spits out the average size of its internal output buffer, and it is almost 0 (which would be great in real life). But I would like to see some simulated congestion so that the program spits out a larger amount of buffering.

I would like to know if there is an easy way to slow down nc's high speed read capability to simulate congestion.

Using a debugger is not in the cards for this problem. Just looking for some standard way to throttle the nc read. If not nc directly, is there another set of commands to do this? (If there is no standard solution, then I will have to write a simple server program to simulate periodic delays.)

I did come across this:
   http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
but not sure how it might be useful with the nc command. Plus, this link talks about "all packets going out of the local Ethernet", and in this case, the socket writes are not going out on the Ethernet.
0
Comment
Question by:phoffric
  • 10
  • 4
  • 3
18 Comments
 
LVL 32

Author Comment

by:phoffric
ID: 40313674
I looked at the nc man page a little more carefully, and saw that -i interval might do the job.
So I added a -i 3 to the nc command and I saw the backlog.
0
 
LVL 32

Author Comment

by:phoffric
ID: 40313681
I had accepted my first post using the -i 3 option; but it is possible (and I didn't have time to verify this) that the backlog was caused during the brief time that I stopped and restarted nc.

Also, I thought that maybe there was a finer granularity than just plain seconds. So, I am reopening the question to see whether there are any better "simple" command line solutions.
0
 
LVL 35

Accepted Solution

by:
mccarl earned 350 total points
ID: 40313704
Have you tried using Pipe Viewer (it also has a rate limiting option)...

http://www.ivarch.com/programs/pv.shtml

You could potentially use it like...
nc -lk 127.0.0.1 4321 | pv -L 69 > /dev/null

Open in new window

Obviously, change 69 (the rate in bytes per second) to something that gives you the effect that you are after.
0
 
LVL 32

Author Comment

by:phoffric
ID: 40313730
That might do the trick. pv is not installed, so I will have to see if IT will add it to their closed repository behind our firewall.
Thanks.
0
 
LVL 35

Expert Comment

by:mccarl
ID: 40313738
I will have to see if IT will add it to their closed repository behind our firewall
Good luck with that, if your IT is anything like our IT  !
0
 
LVL 32

Author Comment

by:phoffric
ID: 40313740
lol

Hmm, the VMware is installed on Ubuntu 12.04, and I was able to install pv on it. Maybe there's a way I can direct the VM Centos output to the host and use pv.
0
 
LVL 32

Author Comment

by:phoffric
ID: 40322411
IT asked me what pv was. I got access to our local repo but yum install pv produced nothing. I see that I can get the pv source code from http://www.ivarch.com/programs/pv.shtml.

I did find a work-around using:
nc -lk -I 7 127.0.0.1 4321 > /dev/null

Open in new window

I mentioned that I was able to install pv on the Ubuntu 12.04 host. The program spitting out the data to nc is on the Centos 6.3 VM. Maybe there's a way I can direct the VM Centos output to the host and use pv.

I have a question about your line:
nc -lk 127.0.0.1 4321 | pv -L 69 > /dev/null

Open in new window

Since the output of nc will be restricted to 69 bytes/second, there is still nothing limiting the input read of nc initially, right? I would think that nc might still be able to read its input rapidly until its buffer(s) filled, and then it would coalesce on the 69 bytes/second rate. Do I have that right?
0
 
LVL 35

Assisted Solution

by:mccarl
mccarl earned 350 total points
ID: 40322427
Yeah, I was going to say that actually (in my last post) that the usefulness my suggestion would depend on how netcats buffering works. And yes, if netcat has big enough internal buffering, it would still read at full speed to fill those buffers, and then the buffers get drained at X bytes/second, until there is enough room for netcat to fill them again. If they are small enough, then this probably does close enough to what you want, but if they are large, then yeah, it's probably not what you are after.
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 
LVL 1

Assisted Solution

by:Serafin Rusu
Serafin Rusu earned 150 total points
ID: 40323968
try to use trickle utility.

https://wiki.archlinux.org/index.php/Trickle


usage should be something like:

trickle -d200 -u50 nc -lk 127.0.0.1 4321

change d and u settings.
0
 
LVL 32

Author Comment

by:phoffric
ID: 40324197
I did a "yum install trickle", and got "No package trickle available". I am on a closed network with repositories that the IT department choose to grace us with. (And apparently the Ubuntu repository gives us more packages on this same network.)
0
 
LVL 1

Expert Comment

by:Serafin Rusu
ID: 40324690
you need to install epel repository to install trickle.

rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
yum install trickle
0
 
LVL 32

Author Comment

by:phoffric
ID: 40324718
My Centos 6.3 VM is in a closed environment, so the only repositories that I have access to are the ones the Centos IT folks give us. Due to the firewall, I have no access to the www. But thanks.

I was hoping that just as there is nc, there might be a very basic Linux command that is in the most basic repository so that even I would have access to it.
0
 
LVL 1

Expert Comment

by:Serafin Rusu
ID: 40324723
you can upload tricket rpm via ssh protocol. using scp utility ;)
0
 
LVL 32

Author Comment

by:phoffric
ID: 40324725
My network does not go out into the www.
0
 
LVL 1

Expert Comment

by:Serafin Rusu
ID: 40324734
well I think you need to hire a specialist to manage that server.

http://www.commandlinefu.com/commands/view/4642/upload-file-to-remote-server-using-scp
0
 
LVL 32

Author Comment

by:phoffric
ID: 40326714
They are specialists who are up to their ears in work, both admin and projects. If enough people request an item, then a manager may authorize them to pay attention. The other possibility is that they have imposed tight rules as to what is permitted to be brought into through their firewall. There appear to be no exceptions to anyone getting in or out with the exception of couple of admin hats. Not sure, but they may even burn DVDs and scan it before adding it to our network.

Now why I have more luck on Ubuntu 12.04 instead of Centos 6.3 is beyond me. I thought the same group handles both. Possibly, for example, pv is included in a more standard repository in Ubuntu 12.04 than Centos 6.3.

I can ask. At this point, I think I need to learn Python and write my own routine with a widget to dynamically provide flow control. With nc -lk -i, I think the program starts over when the connection is lost.
0
 
LVL 32

Author Closing Comment

by:phoffric
ID: 41713785
Although I left the company, based on the comments, I believe I would have made good headway with your posts. Thanks.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
Configuring network clients can be a chore, especially if there are a large number of them or a lot of itinerant users.  DHCP dynamically manages this process, much to the relief of users and administrators alike!
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now