How to find out with say wire shark - all the ip addresses a site tries to go to

I need to allow specific sites on our firewall to go say for instance weather underground.
I want to be able to go to the site and generate a list of sites/ips that the site uses so i can then provide it to the network admin to allow those sites so the site will render correctly.
What is the best thing to use for this that is free?
Thanks!
bbimisAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

btanExec ConsultantCommented:
looks like more of checker to surface all external URL from the website per se..maybe more of link analyser and checker to surface those and for FW to allow accordingly as a whitelist. As whole, I see it more of the tool to
- validate recursively all pages starting with the parent URL;
- check all external links pointing outside the parent URL (stop from further check into external third-party web pages).

LinkChecker - http://wummel.github.io/linkchecker/
Internal/External Link Analyzer - http://tools.seochat.com/tools/page-link-analyzer-seo/
 Link Checker (alt) - http://validator.w3.org/checklink
0
deroodeSystems AdministratorCommented:
You could use Wireshark, but it will generate a lot of information which can be difficult to extract relevant info from.
For problems like this i have used HTTrack: https://www.httrack.com It will download an offline version of the website, with separate folders for every domain that your primary website gets assets (pictures, scripts) from.

I have added a picture of a military music website (bands.army.mil) which shows that this site gets scripts and pictures from googleapis.com, tweetmeme.com, addthis.com etc.
httrack.PNG
0
btanExec ConsultantCommented:
in fact, you need a crawler or spidering tool instead of network sniffer per se to be more targeted to identify those links. Furthermore, if the link are SSL based the sniffer is unlikely to surface totally the URL.

Prior to these spidering/crawling, do make sure you set the appropriate rules to allow your machine IP or the user agent - otherwise any security "middleman" native module/services or appliance will block it.. May also be good to check the sources of all the pages on the http:/https: and href more for a due diligence
0
Bill BachPresident and Btrieve GuruCommented:
If this is really a one-time thing, and if you really only want IP addresses (which can change over time due to load balancing services) instead of URL's (which may be static), then you can indeed just use Wireshark.  Install Wireshark onto your workstation and stop all other non-essential services and tasks.  (You don't want Windows Update or an AV update to start, tainting the results.)

Using Wireshark, start a basic capture by selecting your network card in the top window, then entering "port 80" in the Capture Filter box.  Now, load the web site, and immediately stop the capture when it is complete.

Select Statistics/Conversations, and click on the TCP tab.  You will see a list (in the Address B column) of all IP addresses to which your machine has connected during that window.  Send this list to your firewall people, and they can open it up.

Note that this is not a PERFECT solution, but it will get you there.  A more specific solution might entail checking URL's instead of IP addresses (as the latter can change over time), and any other renegade service that tries to make a connection at the same time might cause you to open up too much.  However, running the test several times might be a way to filter out any "extra" connections.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
bbimisAuthor Commented:
This worked and it was just for a one time thing. Thanks!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Networking

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.