Internet bandwidth restricted or capped on Sonicwall NSA 3500 wan interface.

Hi nhaydock,

Make sure the clients are not running on 100BASE-T NICs, Cables, Switches. In order to get higher rates than 100 Mbps all aforementioned must be 1000BASE-T/1GigE compatible.

Also the NSA 3500 is approaching EOL (End of Life, 5/19/2018). From what I can recall the NSA 3500 can only achieve 240 Mbps UTM Throughput. You should really upgrade to NSA 3600, which provides 500 Mbps of Full Stack DPI throughput or the NSA 4600 yielding 800 Mbps. UTM throughput is really all that you should focus on because if you configure the SonicWALL properly from a security standpoint then UTM throughput is the only real number that will affect you.

I also plugged in my laptop directly to the Bell connection and was getting about 180 up and down

Seems like a problem with your ISP providers...a direct connect test should yield very close to, if not in some cases over, the allotted amount (200 Mbps).

Let me know if you have any questions!

Hi, thanks for your comment. I guess I forgot to mention a few things:
1. We have had the 200mbps Bell connection for many years and were getting up to 200 up and down speeds consistently before we got the extra feeds and the bonder.
2. Once the bonder was in place (fairly recent) the speeds fell below 100. However direct out and from the bonder itself we get much higher speeds so it seems to point to the sonicwall.

Good to know re the 240 throughput - upgrading the sonicwalls later on in the year or early next so will have to remember this limitation.

Thanks!

Here are the actual specs for the 3500 Series along with some of the other SonicWALL devices :

Thanks for your comments.

The interface was set on 1Gbps Full Duplex. I think this was on advisement from the guys that setup the bonder. I changed to automatically detect and got the same speed results. It also doesnt explain the bell x2 connection readings.

I am pretty sure I have run through the MTU exercise before but will give it another try along with your other suggestions although we have a small envirtonment (50 users) and the speed was fine before the bonder was put in place.....weird

Thanks

Using AutoDetect is not a very good idea. A lot of times the connection will resolve to half duplex and that will cause issues with TCP windowing. After testing I would Definitely recommend that you hard code it back to 1g /Full Duplex.

On a side note, I'm curious why the bonder/Aggregator was implemented? The NSA3500 can easily accept multiple WAN connections, and LB (Load Balance) them as well with Active/Active Failover?

So if i take the bell internet connection and plug it directly into my laptop (configure it with the bell public IP) i get speeds of close to 200 both ways. If i take that same cable and plug directly into x2 on the sonicwall (bypass the bonder completely) and create a route policy to send anything from my desktop through the x2 interface then I get speeds of around 95.

To me this seems to be a sonicwall issue and not a bonder issue although it is weird why we have only had these issues since the bonder has been put in play. But taking the bonder out of the equation still seems to result in the same behaviour.

We went with the bonder on a bit of a whim to try - it also has SD wan capabilities which we may implement in the near future.

I will look at the other suggested things to try and get back to you asap.

The sonic wall what was your link negotiating at?

Next question, do you have any kind a QOS

What you were describing, sounds almost identical to what I would expect duplex mismatch

Using voice, So grammar May be a little bad

My other guess would be that configuration changes may have taken place on the SonicWALL at the time and to accommodate the bonder, which may be affecting the throughput.

Sounds good... I'll wait to hear back from you!

Ok some more investigation notes:

CPU doesnt look high when looking at dashboard - multi core monitor, dont have sonicanalyser

1. Stop the Packet Monitor under Dashboard if it is running - confirmed
2. Set the logging level to Informational under Log > Categories - confirmed
3. While in logging, set the Name Resolution to none. - confirmed
4. Verify there are no unnecessary NAT Policies under Network > NAT Policies - will check a little closer but this seems to be ok
5. Verify there are no unnecessary firewall rules under Firewall > Access Rules - will check a little closer but this seems to be ok

Changed mtu on x2 (bell testing) to 1468 - no difference in speeds

For good measure turned off all security services temporarily to test on both lan and wan zones. Also security service setting at performance optimized already.

No QoS or BWM

this is an interesting one....sonicwall still hasnt replied to my case...

Thanks

So after all still that no change with the speed test?

No even after disabling security services still got the same readings....

OK so I'd look at the network layer solely then. Look at the Status page of the SonicWALL and look at the Peak Connections... are they close to or over the Max Connections?

Also, is next to CPU what is the %?

Peak connections are at 3215 out of max 65536. CPU at 4-6.5%

I am back to the bonder being the problem now. I think my desktop client switch is playing up as when I plugged in directly and then made it go to the X2 interface (bell 200 bypassed bonder) I was getting good speeds (180+) but if I point it to the default X1 again (bonded connection) I go back to very inconsistent, jumpy tests (70-130) but never over 180.

I am now pushing the bonder suppliers to take another look but it is now starting to point to their equipment.

Thanks

Ok plugged back the Bell into X2 and client is now going out over that interface (minus my desktop switch but still plugged into edge switch) and get consisten 180-190 down and 120+ up speeds. Changed to max security recommended and got the same results. Have got the bonder guy coming in to troubleshoot and/or replace the hardware.

Thanks for your assistance -  pretty sure I know where the problem lies now....

Terrific...glad I could help!

Awesome - thanks guys!

Your welcome!

Just wanted to follow up with this in case anyone experiences anything similar:

We had the bonder people in to check the speeds. There was a little bit of configuration to be done and it looked like our Shaw feed was very inconsistent. We put this into fail-over mode only and just bonded the two fiber connections. We got very consistent +300 speeds from the bonder itself out. Plugging this into the firewall speeds dropped considerably again. Which was weird as plugging in the Bell fiber directly into the X2 wan connection did not affect speeds by much at all.

We disabled the security services on the WAN and LAN zones again and these shot right back up. Turned them back on one at a time and found that either the Gateway AV or Anti-spyware services drop the speed tests by at least half - brutal!

Because directly connecting the Bell didn't have this drop I am now thinking that this may be something to do with MTU. I believe MTU on the direct Bell connection was 1450. The bonded connection MTU is 1250. Could this make a difference?

Anyway below is my notes to sonicwall....I will add the response once I hear back:
As a test I turned off all WAN and LAN security services (apart from CFS) here are my speedtest results:

279 238
256 194
273 213

Turning on the service one by one...

WAN App control (is this service even necessary - what does it do?)

275 220
266 205
237 186

WAN above + IPS (results in a drop in bandwidth)

242 199
206 198
207 196

WAN above + Gateway AV (results in a HUGE drop taking away half of bandwidth)

113 111
113 109
107 109

Reverted back to APP, IPS but then added Anti-Spyware (results in similar HUGE drop taking away half of bandwidth)

109 202
112 234
109 116

Turning on all WAN security services get similar results as when turning on Gateway AV or Anti-Spyware

117 110
111 110
116 111

So by the looks of it turning on security services kills almost 2 3rds of bandwidth (279-117) I am not sure if this is normal but its brutal if it is. I see the NSA 3500 should have a througput of 240 with all services turned on. I am not getting anywhere close to that.

Are services affected by MTU? I had an old Bell connection which had a high MTU (about 1450 if i recall) and it did not have the same performance hit as this bonded connection with MTU of 1250.

Should all security services be run on both LAN and WAN zones (are we doubling up checking on traffic)?

We disabled the security services on the WAN and LAN zones again and these shot right back up. Turned them back on one at a time and found that either the Gateway AV or Anti-spyware services drop the speed tests by at least half - brutal!

Something is wrong here. As mentioned above your lowest limit for bandwidth is 240 Mbps so clearly this should not affect your traffic individually...aggregated obviously since your yield would be possibly higher depending on how the Aggregator is configured. You had already tested this with different results so I believe there is inconsistency in the testing process. When you are testing...nothing else should be running on the network...you have to isolate it to get scientific about this. Even if the ports are isolated the aggregate throughput and resource consumption will still take place for the other traffic and WAN segment operations.

Are services affected by MTU? I had an old Bell connection which had a high MTU (about 1450 if i recall) and it did not have the same performance hit as this bonded connection with MTU of 1250. I believe MTU on the direct Bell connection was 1450. The bonded connection MTU is 1250. Could this make a difference?

Yes, MTU matters, re-read my article again: https://www.experts-exchange.com/articles/12615/Unstable-Slow-Performing-Networks-or-VPNs-just-go-grocery-shopping.html. In fact your configuration makes it a bit interesting because each WAN Interface should have a custom configured MTU within the Aggregator then from the Aggregator to your X1 within the SonicWALL you should again thoroughly test the MTU, which is going to most likely bounce all over the place since each WAN MTU is going to be different but regardless you should use the most common variable and configure the SonicWALL's X1 MTU accordingly.

Again, I'd recommend removing the Aggregator because the SonicWALL can handle all of that. It only adds more points of failure and complexity to your network like what you are experiencing now.

So by the looks of it turning on security services kills almost 2 3rds of bandwidth (279-117) I am not sure if this is normal but its brutal if it is. I see the NSA 3500 should have a througput of 240 with all services turned on. I am not getting anywhere close to that.

This is an big issue and doesn't add up to the specs nor does it from what I've seen in the field. Make sure the SonicOS is on the latest release!

Should all security services be run on both LAN and WAN zones (are we doubling up checking on traffic)?

Security Services should be running on all Zones ingress, egress &inter-connected, e.g. WAN>LAN, LAN>WAN & LAN>WLAN (or other internal Zones) and vice versa. Security Services are what actually protect the network. We manage over 600 SonicWALLs and every Zone has all Security Services running on them without issue. In fact we are running CAPTURE ATP and DPI-SSL on top of that with no issues.