?
Solved

extract data from web

Posted on 2011-03-18
8
Medium Priority
?
183 Views
Last Modified: 2013-11-19
Hi,

collecting data from websites manually is very hard and time consuming, I'm looking for free application to extract data from web and put it in database or file. for example: I need to get the university information (faculties, department, members, contact information .. etc )

I found different application, but it is not free and hard to learn and customized. can you please guide me to find easy, free and powerful application that do the example mentioned above in a short time.

thanks
0
Comment
Question by:nmokhayesh
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
8 Comments
 
LVL 75

Expert Comment

by:Michel Plungjan
ID: 35170400
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 35173465
Do you want to search the contents of web sites, or do you want to copy the web sites?
0
 

Author Comment

by:nmokhayesh
ID: 35173574
I need to copy selected contents
for instance
list of all professors names and their contacts (email , tel, website, research interests)
list of all departments and contact information
list of schools and programs discription in each one

thank
 
0
WordPress Tutorial 3: Plugins, Themes, and Widgets

The three most common changes you will make to your website involve the look (themes), the functionality (plugins), and modular elements (widgets).

In this article we will briefly define each again, and give you directions on how to install them.

 
LVL 111

Expert Comment

by:Ray Paseur
ID: 35173641
I have used httrack and it worked fairly well to make a copy of the web site onto my hard drive.  Selection of the contents was still the major issue.  Although the local web site was faster than using the internet, you would still have to manually or programmatically isolate the information you wanted to keep.

It might be possible to get a copy of Wrensoft Zoom Indexer and use that to spider the site.  Caveat: I have never tried that on a site that I did not control.

One other possibility might be to contact the site owners and ask if they can isolate this information for you.  Educational institutions are often willing to help with requests like this.
0
 

Author Comment

by:nmokhayesh
ID: 35216825
OK I need to extract selected data from some university web pages to XML or excel sheet file using web scraping software

can you please tell me which free web scraping application can do this job in easy way. I search it but i got a lot of application but I do not know which one is useful/efficient

Thanks
Naif
0
 
LVL 111

Accepted Solution

by:
Ray Paseur earned 1500 total points
ID: 35220096
There is no "easy way" because there is no clear vision of what you want to extract.  Each university web site is likely to be a bespoke application, so each such scraping and extraction algorithm will require custom programming.

That is why I recommended that you contact the site owners and ask if they can isolate this information for you.
0
 

Author Closing Comment

by:nmokhayesh
ID: 36710149
still not solved 100%
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 36710198
No, it will never get solved 100%, full stop.  Here is what you asked for back in March (how many months ago was that?)

...find easy, free and powerful application that do the example mentioned above in a short time.

It would surprise me if you find easy, free and powerful all in the same package.  Those things are like Ohm's law.  Fix any two variables and the third is determined.
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Ready to improve network connectivity? Watch this webinar to learn how SD-WANs and a one-click instant connect tool can boost provisions, deployment, and management of your cloud connection.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article was originally published on Monitis Blog, you can check it here . Today it’s fairly well known that high-performing websites and applications bring in more visitors, higher SEO, and ultimately more sales. By the same token, downtime…
The Windows functions GetTickCount and timeGetTime retrieve the number of milliseconds since the system was started. However, the value is stored in a DWORD, which means that it wraps around to zero every 49.7 days. This article shows how to solve t…
The purpose of this video is to demonstrate how to Import and export files in WordPress. This will be demonstrated using a Windows 8 PC. Go to your WordPress login page. This will look like the following: mywebsite.com/wp-login.php : Click on Too…
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.
Suggested Courses

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question