Solved

netcat nc -l reads data in socket too fast - slow it down

Posted on 2014-09-09
18
919 Views
Last Modified: 2016-07-15
I run a client program which outputs to a socket whose destination is same PC, port 4321. On the same VMware, I run a simulated server using netcat:
nc -lk 127.0.0.1 4321 > /dev/null

Open in new window

so that the program can connect to the nc listening command and write stuff to it. This all works fine. The internal output buffers in the program are nearly 0 since nc reads much faster than the program can produce data. The program spits out the average size of its internal output buffer, and it is almost 0 (which would be great in real life). But I would like to see some simulated congestion so that the program spits out a larger amount of buffering.

I would like to know if there is an easy way to slow down nc's high speed read capability to simulate congestion.

Using a debugger is not in the cards for this problem. Just looking for some standard way to throttle the nc read. If not nc directly, is there another set of commands to do this? (If there is no standard solution, then I will have to write a simple server program to simulate periodic delays.)

I did come across this:
   http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
but not sure how it might be useful with the nc command. Plus, this link talks about "all packets going out of the local Ethernet", and in this case, the socket writes are not going out on the Ethernet.
0
Comment
Question by:phoffric
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 10
  • 4
  • 3
18 Comments
 
LVL 32

Author Comment

by:phoffric
ID: 40313674
I looked at the nc man page a little more carefully, and saw that -i interval might do the job.
So I added a -i 3 to the nc command and I saw the backlog.
0
 
LVL 32

Author Comment

by:phoffric
ID: 40313681
I had accepted my first post using the -i 3 option; but it is possible (and I didn't have time to verify this) that the backlog was caused during the brief time that I stopped and restarted nc.

Also, I thought that maybe there was a finer granularity than just plain seconds. So, I am reopening the question to see whether there are any better "simple" command line solutions.
0
 
LVL 35

Accepted Solution

by:
mccarl earned 350 total points
ID: 40313704
Have you tried using Pipe Viewer (it also has a rate limiting option)...

http://www.ivarch.com/programs/pv.shtml

You could potentially use it like...
nc -lk 127.0.0.1 4321 | pv -L 69 > /dev/null

Open in new window

Obviously, change 69 (the rate in bytes per second) to something that gives you the effect that you are after.
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 32

Author Comment

by:phoffric
ID: 40313730
That might do the trick. pv is not installed, so I will have to see if IT will add it to their closed repository behind our firewall.
Thanks.
0
 
LVL 35

Expert Comment

by:mccarl
ID: 40313738
I will have to see if IT will add it to their closed repository behind our firewall
Good luck with that, if your IT is anything like our IT  !
0
 
LVL 32

Author Comment

by:phoffric
ID: 40313740
lol

Hmm, the VMware is installed on Ubuntu 12.04, and I was able to install pv on it. Maybe there's a way I can direct the VM Centos output to the host and use pv.
0
 
LVL 32

Author Comment

by:phoffric
ID: 40322411
IT asked me what pv was. I got access to our local repo but yum install pv produced nothing. I see that I can get the pv source code from http://www.ivarch.com/programs/pv.shtml.

I did find a work-around using:
nc -lk -I 7 127.0.0.1 4321 > /dev/null

Open in new window

I mentioned that I was able to install pv on the Ubuntu 12.04 host. The program spitting out the data to nc is on the Centos 6.3 VM. Maybe there's a way I can direct the VM Centos output to the host and use pv.

I have a question about your line:
nc -lk 127.0.0.1 4321 | pv -L 69 > /dev/null

Open in new window

Since the output of nc will be restricted to 69 bytes/second, there is still nothing limiting the input read of nc initially, right? I would think that nc might still be able to read its input rapidly until its buffer(s) filled, and then it would coalesce on the 69 bytes/second rate. Do I have that right?
0
 
LVL 35

Assisted Solution

by:mccarl
mccarl earned 350 total points
ID: 40322427
Yeah, I was going to say that actually (in my last post) that the usefulness my suggestion would depend on how netcats buffering works. And yes, if netcat has big enough internal buffering, it would still read at full speed to fill those buffers, and then the buffers get drained at X bytes/second, until there is enough room for netcat to fill them again. If they are small enough, then this probably does close enough to what you want, but if they are large, then yeah, it's probably not what you are after.
0
 
LVL 1

Assisted Solution

by:Serafin Rusu
Serafin Rusu earned 150 total points
ID: 40323968
try to use trickle utility.

https://wiki.archlinux.org/index.php/Trickle


usage should be something like:

trickle -d200 -u50 nc -lk 127.0.0.1 4321

change d and u settings.
0
 
LVL 32

Author Comment

by:phoffric
ID: 40324197
I did a "yum install trickle", and got "No package trickle available". I am on a closed network with repositories that the IT department choose to grace us with. (And apparently the Ubuntu repository gives us more packages on this same network.)
0
 
LVL 1

Expert Comment

by:Serafin Rusu
ID: 40324690
you need to install epel repository to install trickle.

rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
yum install trickle
0
 
LVL 32

Author Comment

by:phoffric
ID: 40324718
My Centos 6.3 VM is in a closed environment, so the only repositories that I have access to are the ones the Centos IT folks give us. Due to the firewall, I have no access to the www. But thanks.

I was hoping that just as there is nc, there might be a very basic Linux command that is in the most basic repository so that even I would have access to it.
0
 
LVL 1

Expert Comment

by:Serafin Rusu
ID: 40324723
you can upload tricket rpm via ssh protocol. using scp utility ;)
0
 
LVL 32

Author Comment

by:phoffric
ID: 40324725
My network does not go out into the www.
0
 
LVL 1

Expert Comment

by:Serafin Rusu
ID: 40324734
well I think you need to hire a specialist to manage that server.

http://www.commandlinefu.com/commands/view/4642/upload-file-to-remote-server-using-scp
0
 
LVL 32

Author Comment

by:phoffric
ID: 40326714
They are specialists who are up to their ears in work, both admin and projects. If enough people request an item, then a manager may authorize them to pay attention. The other possibility is that they have imposed tight rules as to what is permitted to be brought into through their firewall. There appear to be no exceptions to anyone getting in or out with the exception of couple of admin hats. Not sure, but they may even burn DVDs and scan it before adding it to our network.

Now why I have more luck on Ubuntu 12.04 instead of Centos 6.3 is beyond me. I thought the same group handles both. Possibly, for example, pv is included in a more standard repository in Ubuntu 12.04 than Centos 6.3.

I can ask. At this point, I think I need to learn Python and write my own routine with a widget to dynamically provide flow control. With nc -lk -i, I think the program starts over when the connection is lost.
0
 
LVL 32

Author Closing Comment

by:phoffric
ID: 41713785
Although I left the company, based on the comments, I believe I would have made good headway with your posts. Thanks.
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Creating an OSPF network that automatically (dynamically) reroutes network traffic over other connections to prevent network downtime.
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question