webapp performance issues in DMZ

we are facing issue in one of our web application it works fine locally .Response Time on internal network  is 4ms but when we try to access it from out side  web application open but took long time to load the files. The application accessed by all the customers to view and download load their licenses.

OS :WIndows sever 2012
Database: Oracle
Memory 32GB

All the resources are Normal  except Network latency

Network Latency reaches to 280ms when access from outside .The server is in DMZ .
kastro AbbasiIT consultantAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Gary PattersonVP Technology / Senior Consultant Commented:
Well with latency 70x slower outside than inside, any application that sends a lot of small packets is going to be slow.  How are you loading files?

You'll need to talk to your internet service provider, and look at any network devices between the internet and your web server to determine if it is possible to reduce latency.

Please provide more information about application and the network configuration.  Also, are you able to isolate what operations, in particular, are slow when accessed from outside the network?
nociSoftware EngineerCommented:
Maybe you need a caching DNS server on the DMZ, so lookups are kept for a while.  (A reverse lookup is often done).
Also be aware that lower network speeds (bandwidth) induce bigger timeouts.

Other issues might be asymetric connections or one way of a connections drowning out.  (preventing small ACK packet to move in a timely fashion. Also check for packetloss,  and fragmented packets.   Fragmented packets can be prevented by lowering the MTU on the server, or by enabling PMTU discovery).
kastro AbbasiIT consultantAuthor Commented:
[Edited by Gary:  information posted to Report Comment by mistake by poster]
Application is used to authenticate number of users and upload lot of files which are placed on local server.

From remote User the request comes to the core Switch then to the fortinet fire wall and then to the application server and application server send the request to Oracle DB .
As per my investigation till firwall there is no latency it send and recive the packet immediately but from app to the DB it take 30 Seconds there is too much TCP trafic approximate 200MB between apps and DB.
[End edit]

Moreover when we assign an IP of our Production VLAN the server performed well but when we assign an ip of DMZ VLAN its performance goes down.
this webapp server is in DMZ
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

Gary PattersonVP Technology / Senior Consultant Commented:
Is the same amount of data being sent, and it is just a different in network speed, or is there actually a different amount of data being sent?  

It is not uncommon for local test environments to be faster than production environments.  It also is not uncommon for test database to be smaller than production databases.

It is hard to provide much help without doing some first-hand performance troubleshooting.
nociSoftware EngineerCommented:
If your database access is through the firewall that will cause unneeded delays. 4  hops in stead of one, probably overhead by the firewall in filtering packets.  And for further analysis more info is needed.  Which is i think beyond EE to answer. Maybe start a gig for it, i don't do gigs btw.
Gary PattersonVP Technology / Senior Consultant Commented:
As per my investigation till firwall there is no latency it send and recive the packet immediately but from app to the DB it take 30 Seconds there is too much TCP trafic approximate 200MB between apps and DB.

You say here performance problem is between app server and DB?  So does this describe your setup?

Fast:  Browser - (LAN) - Test AppServer - (LAN) - Test DB Server
Slow: Browser - (Internet) - Firewall(DMZ) - Prod App Server - Firewall(LAN) - Prod DB Server

To many differences to make reasonable guess at problem.

Problem could with internet connection, Firewall(DMZ) configuration, DMZ network, Prod AppServer setup, Firewall(LAN) configuration, Prod DB Server configuration, differences in test/prod database size, etc.

You need to narrow it down more.  What app server are you using?  Have you have your Oracle DBA compare performance of the test and production databases?  Have them log the queries that your application is executing and look at the execution plans.  Maybe you are missing indexes on the production database server, or maybe the prod database is a lot bigger than the test database.

Then work backwards from there to the app server.  I don't know what you're using, so I can't offer performance guidance, but most app servers I've worked with have performance monitoring tools.

If you think it is the network, Wireshark traces from the "fast" and "slow" networks might be usefule.  Capture between browser and appserver first, then appserver and DB.  Happy to take a look at captures if you collect them and post them someplace I can get to them.

- Gary

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Operating Systems

From novice to tech pro — start learning today.