royalgoldleaf
asked on
Pls explain how to grab text from a website
My background:
Total new to programming language
Linux web hosting
I will hire someone to achieve this task.
Dear experts
I plan to grabs some texts and images from a classified website.
I need to know how things work so that I can explain to programmers what I need to achieve.
The programmer will need to create an automated tool.
The main page of the website are updated every few minutes with new listing.
The tool will need to click on each of this listing, and grab Topic, Description, Image etc from the listing and put into a database... The tool will also need to check the main page for un-grab listing hourly.
My questions
a, Is this call data parsing?
b, what programming language are commonly used to create the tool? Can php be use?
c, once the programmer create the codes, I will put the files in my linux web server and run it myself . Is this alright?
d, the tool will grab the data and put in MYSQL?
Did I left out anything that worth mentioning?
Your details explanation is much appreciated.
thx
Total new to programming language
Linux web hosting
I will hire someone to achieve this task.
Dear experts
I plan to grabs some texts and images from a classified website.
I need to know how things work so that I can explain to programmers what I need to achieve.
The programmer will need to create an automated tool.
The main page of the website are updated every few minutes with new listing.
The tool will need to click on each of this listing, and grab Topic, Description, Image etc from the listing and put into a database... The tool will also need to check the main page for un-grab listing hourly.
My questions
a, Is this call data parsing?
b, what programming language are commonly used to create the tool? Can php be use?
c, once the programmer create the codes, I will put the files in my linux web server and run it myself . Is this alright?
d, the tool will grab the data and put in MYSQL?
Did I left out anything that worth mentioning?
Your details explanation is much appreciated.
thx
have a look at snoopy
it's a really nice class that does it for you..
snoopy.sourceforge.com
it's a really nice class that does it for you..
snoopy.sourceforge.com
sorry I didn't fully read the threath ;)
it can all be done in php...
using snoopy you'd grab the full page and then strip out the stuff you like and put it in the database..
any php programmer should be able to do this..
it can all be done in php...
using snoopy you'd grab the full page and then strip out the stuff you like and put it in the database..
any php programmer should be able to do this..
ASKER
Any pre-written scripts that i can customize to do the job?
Cant find in snoopy..
Cant find in snoopy..
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Thanks for the points!
Thanks,
Umesh
Thanks,
Umesh
PHP -> use curl to grab text from remote web site, extract the required data from the copied text & put this into MySQL
Bash/sed/awk - > Using lynx copy the entire text into a text file, extract the required data and format according to MySQL compatible (csv or tsv) and just load this into MySQL (using heredoc, non-interactive mode).
Perl-> can be done...