Why does my faulty wired-broadband work with a VPN but not otherwise?

redmondb
redmondb used Ask the Experts™
on
Hi.

My home broadband is, as I understand it, fibre to the local junction box (less than a mile away) and then very old copper to my house. When I first signed up a couple of years ago, I was consistently getting around 40Mbs down and 10Mbps up.

During this period ( the "golden age") , the line would occasionally "fail" but, as I'll explain,  with minimal impact to me. I say "fail" because Ping and TraceRT worked, but more importantly, using a VPN (Witopia), I could happily "punch through" the problem and reliably get my full 40Mbps. The ISP engineer would eventually arrive out, test the line, agree that it could support 40Mbs, agree that there was no problem in my house, go away, do "someting" and my connection would go back to providing 40mbs without the VPN. Happy days, indeed,

Sadly, back in May. at 23:36 one night, my line went down and when it came back *three* minutes later, my max speed had gone from 40Mbps to 22Mps (with upload going from 10Mbps to 3.3Mbps). Aargh! Unfortunately, I had neither the time nor the energy to get into an extended row with my ISP and "settled" for 30Mbps down, 10Mbps up (which I verified independently).

And now my latest situation - as of a week or so ago, performance has collapsed. Some times my line supports 30Mbs/10Mbps, but for extended periods, I'm getting some data, but usually not enough to even load a web page. Unless, of course, I use my VPN. But not all of its setups work - I get a perfect connection using OpenVPN+TCP. UDP simply can't hack it.

So, my big question, please, is ...
Why is my TCP VPN working? I'm guessing that it's something to do with TCP error-correction, but I don't understand how that's so much more powerful than that of a a non-VPN TCP session.
(Subsidiary question, if it's not apparent from the main answer, what's the root problem?)

Background:
ISP - Sky Ireland.
PC - Laptop running Windows 8.1 (but I've got a consistent result from an Android phone).
Modem/Router - Sky QHub ER110 (and, previously, SR102).
If it helps, I am the only person in the house and I do not now, nor have ever, torrented or file-shared, My WiFi is WPA2-Personal and my password is 63 random characters.
Line monitoring - Every 10 minutes, a utility checks my line speed - JD Auto Speed Tester (https://www.gmwsoftware.co.uk/).
Phone - while my broadband is misbehaving, the phone line and independent alarm system (both sharing the same copper as my broadband) is working fine. (The phone line shares a "master box" with the broadband whereas the alarm is wired-in upstream of that.)

Many thanks,
Brian.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
Do you have a separate ISP Modem and Router?  If so, connect another computer directly to the modem and measure speed.  If this returns to normal, perhaps the router needs to be reset, or TCP/IP on your computer needs to be reset.

Open cmd.exe with Run as Administrator
Then: netsh int ip reset c:\resetlog.txt
Then: ipconfig /flushdns
Then: restart the computer
Thanks for the timely response, John. PC and modem/router have each been power recycled. I have also tried wired connection. Results didn't change.

Some additional information...
Pre the latest situation, my typical figures were - "Attenuation Down" 23.4dB, "Noise Margin Down" 9.4dB, "Noise Margin Up" 9.9dB. Currently they are 23.3/16.4/14.14. (While, I presume, the Noise figures are dramatically better, they have made no difference to my non-VPN  failure.)

Regards,
Brian.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
You need to see if you can isolate the problem to your computer or to the modem. If the modem and router are combined, consider doing a reset back to factory specifications to eliminate the VPN part of it (if it exists in the router).
Success in ‘20 With a Profitable Pricing Strategy

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

John,

Android's giving similar results. The VPN is on the PC and the phone.

FWIW, the evidence seems clear to me that the problem is outside my house. If you can persuade me otherwise, great, I can do something about it! Otherwise, I just want to understand the magic of the VPN!

Regards,
Brian.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
The one (I think) piece of equipment between your devices and the outside world is the modem/router OR the ISP itself. So talk once more to the ISP about resetting your modem and perhaps replacing the connection from your home and the main ISP equipment
John,

Thank you, but my main concern here is to identify what's actually happening. Can you you explain any situation in which the VPN works, but a plain connection doesn't?

Thanks,
Brian.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
If your devices work the same, the issue seems to be device independent and therefore:  (a) in your modem (reset it to factory specifications) or (b) your ISP. Your VPN connection may terminate in a different ISP.
John,

i didn't mention it above, but I did reset the modem to factory defaults.

Yes, I have little doubt that the problem is with my ISP... I just want to understand what's happening! Do you have any idea as to what kind of a problem could result in the situation that I have described?

Thanks,
Brian.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
I am not sure what VPN you are using, but it would appear that it is the kind of VPN that ends in another endpoint. That is the only reason I can come up with. That or your ISP handles VPN differently than bland connections.
Top Expert 2013

Commented:
One thought;  most VPN's do not allow "split tunneling".  This means connections to local devices are blocked while the VPN is connected.  Is there a possibility that your PC is "chatting" with some local device?  Such as sharing data or streaming media, etc..  This could be consuming bandwidth.
Folks,

My apologies, I thought that I made it clear that I'm using a commercial, external VPN (www.Witopia.com). I do not use split-tunnelling, so, while the VPN is running, all of my internet traffic is going out through the encrypted tunnel to the appropriate Witopia server.

There is not an issue with other traffic swamping the link - I can see via the router that there is trivial to no traffic when the VPN is not running and only my normal traffic when it is running.

Thanks,
Brian.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
I can see via the router that there is trivial to no traffic when the VPN is not running and only my normal traffic when it is running.

Yes, that is what I assumed. I use IPsec Split Tunnel VPN and the tunnels are part of my VPN router (and the modem is the separate ISP box).

I think you need to engage ISP support based on everything you have said.
Top Expert 2013

Commented:
>>"so, while the VPN is running, all of my internet traffic is going out through the encrypted tunnel to the appropriate Witopia server".  Exactly, thus the local network is disconnected.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
Yes, but when the VPN is disconnected, his local bland connection is not working. So there is something different going on.
I have spent tens of hours on the phone with the ISP and got nothing... except a halving of my effective speed.

Sadly their personnel are just not up to it - depending on the number of days in the month, they either blame the quality of the wires in the ground or else claim that their remote testing shows it as a problem in my house (despite every visit by their provider's engineers giving the house a clean bill of health).

I have enough technical knowledge to recognise my ISP's lack of it, but not enough to explain what's happening, hence this question.

Can anyone give any possible explanation for what I'm seeing? The only thing that I could think of was that they were having a problem with their DNS servers, but trying Google (8.8.8.8 and 8.8.4.4) and then Norton (199.85.126.10 and 199.85.127.10) had no effect

Thanks,
Brian.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
I use my own ISP DNS servers but Google DNS works fine also.

they either blame the quality of the wires in the ground or else claim that their remote testing shows it as a problem in my house

Does that not belong to the ISP?  They have replaced my cable from street to house (their cable and now some years (10?) back). The cable from the box outside to my ISP modem is theirs but it is in plain view and not an issue. They replaced the modem a year ago with a faster modem. All this is theirs. They can test it properly from their office.

Can you get a new modem from them, bridge it, and supply your own router? I use a top grade router.
The ISP doesn't own the lines, and the company that does gets paid by the ISP for every call-out so I'm not holding my breath for an upgrade!

I now have a collection of five perfectly functioning Sky modem/routers as sending me a new one was their initial response to a number of earlier incidents. I also bent the ISP's T&Cs by trying my own, but to no avail.

Even if the lines are little better than damp cotton wool and the local junction box is infested by rabid aardvarks, how is a TCP VPN getting through when everything else is crippled?!
Qlemo"Batchelor", Developer and EE Topic Advisor
Top Expert 2015

Commented:
Looking at the physical bit encoding, SSL traffic is not different enough from SSL VPN to explain anything. The VPN has the option to compress traffic, but that usually leads to added delays, and doesn't improve transfer rates. To me, it sounds like an artificial limitation applied to your traffic. Sadly, there is nothing to allow to prove that (for you). It is something outside of your house.

Having said that: there is still one test you can perform. Use
  mturoute -t
against one arbitrary target, with and without VPN. This will show if there are MTU issues somewhere, and you might get much better results if reducing the MTU on your PC (not available for mobile, though).
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
Unless there is some information we do not have, the issue would very much appear to be your modem and beyond and apparently not your devices.
Top Expert 2013

Commented:
>>"Yes, but when the VPN is disconnected, his local bland connection is not working. So there is something different going on. "
This is my point, when the VPN is enabled all local traffic to other PC's, TV's, and Internet is blocked, thus VPN speed is OK.  Could there be a problem with local traffic reducing bandwidth.

I can also see it being a problem with router or modem, but the idea that the wires are the problem, I don't believe it.  Wiring is the #1 cause of poor performance, but VPN or no VPN it is all TCP/IP packets, I cannot see why there would be a difference.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
Could there be a problem with local traffic reducing bandwidth.   <-- He says no, which I took at face value.

I can also see it being a problem with router or modem  <-- Yes, although in this case I think the ISP service at their end is also an issue.
John and Rob.
Apologies, and I'm not sure how this confusion arose, but I'm having no problems within my LAN - the problem is purely with web traffic.

Qlemo
Yes, I like your thinking! I've run MTURoute (see below), but, sadly, the line is currently going though one of it's sporadic periods of sanity so I'm not sure how useful these results are. As soon as the problems recurs I'll try again and post back. Fair warning - it worked fine for five days at the start of the week, so please don't think I've forgotten you if you see no activity for a few days.

Your suggestions triggered a distant memory, which I should have considered before. Have you any thoughts on NetAlyzr (netalyzr.icsi.berkeley.edu)? It attempts to analyze your web connection. I ran it with and without the VPN (the line may have been ok then, but note that it has interruptions). I've included a possibly significant part-extract from both runs

(Edit: I removed a comment based on a misreading of the NetAlyzer runs.)

Many thanks to you all for your time.
MTURoute_I_-Redacted-.txt
Netalyzr_I_-Extract-.txt
Qlemo"Batchelor", Developer and EE Topic Advisor
Top Expert 2015

Commented:
It is important that you have the mturoute results of both situations to compare, so you now have part 1 of 2 ;-). The same applies to the NetAlyzr report.
Regarding the latter,  I assume 172.30.1.207 is nothing on your LAN, but the ISPs gear? And those results do not reflect a bad connection, though it looks like so (seeing packet loss, reduced upload rate and a very small MTU)?
Qlemo,

Of course, it's still behaving itself.

172.30.1.207 is yet another mystery in all this - it's in the Private Address Range 172.16.0.0 to 172.31.255.255. It certainly isn't on my LAN. There's no mention of it when the VPN is used, so I guess it must be in the ISP's network. As I understand it, by definition, Private Addresses are unroutable. So how does NetAlyzr know about this address?!
Qlemo"Batchelor", Developer and EE Topic Advisor
Top Expert 2015

Commented:
Unroutable doesn't mean you cannot see that IP somewhere ;-). In an ICMP trace recording the route taken each hop is contained. In an ICMP trace not recording, you see at least see the hop IP the packet has been dismissed (because of the current TTL = max. hop count set). So even when you cannot reach that IP, you see it.
But wouldn't NAT change that address before passing on the packet?

Edit: My LAN address is 192.168.0.3, but surely nobody outside my LAN can know that? (Unless I tell them!)
Qlemo"Batchelor", Developer and EE Topic Advisor
Top Expert 2015

Commented:
IPs in the payload are never subject to NAT - unless special application layer protocol gateways are applied, e.g. for FTP. FTP in active mode includes transfer of the real IP in the payload for establishing the data channel, and that one needs to get translated to the public IP. But as said, that is an exception and used rarely.
Qlemo.

Just for my education, please, when I send a "normal" TCP packet to the internet does my Router not strip my local IP address from the packet and replace it by my public address?
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
In a normal network connection, yes
Correction: So, in this case, what could be storing my ISP's local address in the payload as well as the Source Address?
Qlemo"Batchelor", Developer and EE Topic Advisor
Top Expert 2015

Commented:
Traceroute does, and TCP if the corresponding TCP flag is set (in which case hops can still ignore it; if not, they add their IP to a special area in the TCP headers as another payload).
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
My error above. Looking a Comm View packet, the local address is part of the packet out. I was incorrect above.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
I am not sure, however, what the packets contain has to do with slowness by your ISP.
John,
The MTU seems to be a possible cause. NetAnalyzr flagged the Private Address as the source of a bottleneck.

Qlemo,
I captured a TraceRT session NetMon and drilled into one of the pings generated. My Private Address is only shown in the source address, not the payload.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
I have never seen that in my network setup. Perhaps your VPN setup is causing this.
John,

You've never seen the bottleneck? That's a good sign as you don't have my problems!

Perhaps your VPN setup is causing this.
Quite the contrary - as the NetAlyzr (apologies for the inconsistent spelling) extract shows, the bottleneck occurs when the VPN isn't being used.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
You have (I have) 3 or 4 computers sending out packets. They need the address so as to return the information. This does not cause bottlenecks - me or any client, within the normal realm of traffic flow.
Qlemo"Batchelor", Developer and EE Topic Advisor
Top Expert 2015

Commented:
Can we please stop distracting ourselfs by talking about the private IP? Your own IP will not be shown in a traceroute, that is correct. The ones of hops can, though, or when using ping -r for example. Whatsoever, it is not part of your issue.

The interesting part is that a bottleneck is shown though you do not have issues currently. That's confusing.
Quick update:
Line went down at 3:45 this morning and stayed down until I rebooted the router at 9:00. While it's not unusual to see the line go down and immediately come in the small hours, I can't remember the last time I needed to do something to get it back,

In the 10 hours since the reboot, the line has been pretty much rock solid (out of 65 tests, 61 had a download speed of 17.0 to 17.2 Mbps). I've run NetAlyzr once or twice and am consistently getting the same bottleneck details. (I haven't used the VPN at all today.)
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
I have no idea why you are getting bottlenecks. Normal traffic like this never causes me bottlenecks year in and year out. Can you give these results to your ISP to inquire why they think this might be the case.
John,

There's a technician due in a day or so and I'll certainly mention it to them. I'll also pass it on to the Support desk, but the chance of it being read by anyone who's even heard of  MTU is very remote!

BTW, have you tried NetAlyzr yourself? Sadly the only results I have to go on are my own (both here and in a neighbour's house) and some I did a while ago testing mobile broadband.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
I have seen it used and it was not identifying and issue. The traffic being analyzed was completely normal.

See what the technician says about your results
Ok, the Technician's been and gone. He checked the line and said it was perfect, so he restored it to close to its original speed. His remit pretty much finishes at the junction box, so couldn't help with problems further up the line than that. He'd never come across VPN's successfully dealing with a poor connection, but agreed that it certainly seemed like an ISP issue.

For the last three weeks, the line has been fine early in the week and then failed on Thursday/Friday. All going well, I'm going to hold off until Friday and then, <sigh>, contact the ISP.

Thanks, all!
I was talking to my ISP yesterday. They say that they are going to get their technical people to look into this. I've heard that before, but slightly more encouraging is that it's starting to dawn on them how much all this costing.
For the last few weeks, the lines been OK for, say four or five days and then "temperamental" for a couple more. The ISP has finally acknowledged that it's beyond their technical competence and have escalated it to the infrastructure owner. Of course, they're off until the New Year, so, at best, it'll be a few days before I hear anything.

At various times, MTURoute has given unusual MTU's. I suspect that this is simply a reflection of lost packets on one or more hops fooling MTURoute's binary search.

Just before the holidays, I was in touch with one of the Netalyzr people. He's quite interested in what's going on, so I hope to talk more to him. I would dearly love to know where the private IP address is coming from.

Finally, this question has been open for many weeks. I'm more than happy to stay with it, but if you guys feel that enough is enough, then I'll close it.
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
I am not sure why you are getting these bottlenecks. This does not happen to me or to my clients, so I have not experienced what you have and cannot replicate it.
No, I can't replicate it either - it's either happening or it isn't.
Qlemo"Batchelor", Developer and EE Topic Advisor
Top Expert 2015

Commented:
Any news here?
Moving at a rate of inches. My ISP finally passed this to their wholesaler. They are slowly revisiting everything that the ISP has done so many times (send out an engineer, replace the modem, <sigh>). Except for the fact that my ISP finally seems to believe me, I've seen nothing new.

IIH,  I got into a neighbour's house while they had a similar problem (connection up, but unusable) and using a VPN fixed things exactly as it did for me.

I'm not going anywhere, but you two have lives. Want me to close this?

Thanks,
Brian.
Qlemo"Batchelor", Developer and EE Topic Advisor
Top Expert 2015

Commented:
I would like to know the outcome.
Me too! Don't worry, whatever happens, I'll update here.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial