Solved

Pls explain how to grab text from a website

Posted on 2008-10-17
7
355 Views
Last Modified: 2013-12-13
My background:
Total new to programming language
Linux web hosting
I will hire someone to achieve this task.

Dear experts

I plan to grabs some texts and images from a classified website.
I need to know how things work so that I can explain to programmers what I need to achieve.
The programmer will need to create an automated tool.
The main page of the website are updated every few minutes with new listing.
The tool will need to click on each of this listing, and grab Topic, Description, Image etc from the listing and put into a database... The tool will also need to check the main page for un-grab listing hourly.
My questions
a, Is this call data parsing?
b, what programming language are commonly used to create the tool? Can php be use?
c, once the programmer create the codes, I will put the files in my linux web server and run it myself . Is this alright?
d, the tool will grab the data and put in MYSQL?

Did I left out anything that worth mentioning?
Your details explanation is much appreciated.
thx


0
Comment
Question by:royalgoldleaf
  • 3
  • 3
7 Comments
 
LVL 26

Expert Comment

by:ushastry
ID: 22745737
I think php,perl or any shell script will do this stuff..

PHP -> use curl to grab text from remote web site, extract the required data from the copied text & put this into MySQL
 
Bash/sed/awk - > Using lynx copy the entire text    into a text file, extract the required data and format according to MySQL compatible (csv or tsv) and just load this into MySQL (using heredoc, non-interactive mode).

Perl-> can be done...
0
 
LVL 5

Expert Comment

by:RightNL
ID: 22747518
have a look at snoopy
it's a really nice class that does it for you..
snoopy.sourceforge.com  
0
 
LVL 5

Expert Comment

by:RightNL
ID: 22747531
sorry I didn't fully read the threath ;)
it can all be done in php...
using snoopy you'd grab the full page and then strip out the stuff you like and put it in the database..
any php programmer should be able to do this..
 
0
Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

 

Author Comment

by:royalgoldleaf
ID: 22749160
Any pre-written scripts that i can customize to do the job?
Cant find in snoopy..
0
 
LVL 26

Accepted Solution

by:
ushastry earned 250 total points
ID: 22755803
0
 
LVL 5

Assisted Solution

by:RightNL
RightNL earned 250 total points
ID: 22755843
snoopy can be found : http://sourceforge.net/projects/snoopy/
it's a highly customable php class. which you can just point the url to...
 
0
 
LVL 26

Expert Comment

by:ushastry
ID: 22887433
Thanks for the points!

Thanks,
Umesh
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

SASS allows you to treat your CSS code in a more OOP way. Let's have a look on how you can structure your code in order for it to be easily maintained and reused.
These days socially coordinated efforts have turned into a critical requirement for enterprises.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

896 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now