How do I create a script to crawl my site and pull data via .csv?
Hello Experts,
My company recently entered the business of redesigning client websites. During the discovery phase of these site redesigns we perform a site content audit. In the audit we gather - site structure information, page names, urls, downloadable documents etc. We hand click through every link and note all of these items in an Excel document. This may sound very primitive to you, which I am sure that it is, and I would like to figure out how to automate this process.
What can I do to move more quickly through these sites while maintaining documentation of what is being found within them?
Note:
I have never written a script before, and would not know where to start, but my Development team might. A tool may be more helpful to me at this time, but I am certainly not above asking my team for a bit of assistance with this.
Thank you!
My company recently entered the business of redesigning client websites. During the discovery phase of these site redesigns we perform a site content audit. In the audit we gather - site structure information, page names, urls, downloadable documents etc. We hand click through every link and note all of these items in an Excel document. This may sound very primitive to you, which I am sure that it is, and I would like to figure out how to automate this process.
What can I do to move more quickly through these sites while maintaining documentation of what is being found within them?
Note:
I have never written a script before, and would not know where to start, but my Development team might. A tool may be more helpful to me at this time, but I am certainly not above asking my team for a bit of assistance with this.
Thank you!
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
You would have to look at the detailed requirements. I believe one of those offers a custom solution, so they for sure would support mac. I would think for the kind of coding to do what needs to be done it would not be real hardware specific anyway. A crawler is basically dealing with code on a web page delivered by a server so the client OS should be transparent.
Cd&
Cd&
ASKER
Have you heard of WebLight? http://download.cnet.com/WebLight/3000-10248_4-25438.html?tag=lia;rcol Would you recommend it?
I'm not familiar with the product, but I don't spend much time in Mac world. It has a free 30 day trial, so it costs nothing but some of your time to check it out. I don't see anything in the product description that causes concern. Because I'm pc-centric rather being in the Mac zone I don't know that I am qualified to recommend Mac products, but from a general high level view i would not recommend against it.
Cd&
Cd&
ASKER
Sure, well you have been helpful COBOLdinosaur. Thank you!
ASKER
Yes I was thinking it was not best to start with a script at my level. Would these solutions work for a Mac? I apologize for neglecting to mention that is what I am using.