asked on

How to extract data from resume board.

Hey guys,

My company has purchased access to a resume board with over 10,000 resumes. The resumes are stored on their server in the format of "http://[Domain]/EmpResume.asp?ResumeID=x" where x = a resume ID number (somewhere between 1 and 10000). Once you have logged in once, you can navigate to any resume you wish by just changing "x" in the url.

I need to find a way extracting the data from each resume in to a table or database. The data appears on the web site in a structured table, so the data is always in the same place on each resume.

Since there are so many, I would need this as automated as possible.

Thanks,
Grant

bpmurray

You don't say what machine you're working on, nor how the resumes are returned. Your biggest issue is trying to parse the HTML that's returned. There are a bunch of libraries you could use, but I'd simply use a Web page that has its fields bound to the database. Using a hidden iFrame, you can loop through the resumes (set src="url"), and then pull the information from the various fields to populate the local fields, do a submit, and loop to get the next.

All this can be done using Javascript, along with whatever you use to access your DB on the backend (.Net/PHP/Java/whatever), so you can have a visual feedback of what it's doing.

bpmurray

I should have said that you could of course read the HTML using an HTML library, e.g. http://www.codeproject.com/library/GomzyHTMLReader.asp, and then update the DB directly.

gtporter

ASKER

I'm not looking for anything that complicated because I do not have much expierence in C++ or programming in general. I am looking for a possible unicorn of a program. Something that I can open a sample document (IE http://[Domain]/EmpResume.asp?ResumeID=1) and tell it what fields in the table I wish to save to specific fields in a database or excel document. Is this a program that doesn't exist or am I just googleing the wrong things?

bpmurray

OK, try importing one of the pages into Excel, and see how it looks. If the stuff loads consistently, you can process it from there, pushing the cell contents into the DB using Excel's database connectivity. This is essentally the same model as above, but using Excel as the HTML parser.

If you post a sample page (replace the specific content, of course), I can get a better idea of what might be possible.

BTW, is this on windows? The reason is that many Unixes have standard commands that make this kind of thing much easier.

gtporter

ASKER

I am using Windows XP Pro. I'll try the Excel idea after lunch and post on how it went (I like the concept though).

Sample code for ResumeID 1:

<noscript><meta http-equiv="refresh" content="0;url=/unsupported.asp?URL=/EmpResume.asp?ResumeID=1"></noscript>

<html>
<head>
<title>[--(First Name)--] [--(Last Name)--] - Engineering Resume from EngCen.com</title>
<link rel="stylesheet" type="text/css" href="include/engcen.css">
<script language="javascript">
function ReviseSearch()
{
 if(document.form1.formtype.value=="advance")
 {
 document.form1.action = "AdvResSearch.asp";
 }
 else
 {
 document.form1.action = "ResumeSearch.asp";
 }
 document.form1.submit();
}

function saveresume(rid)
{
document.form1.action="StoreResume.asp?ResumeID="+rid;
document.form1.method="post";
document.form1.submit();

}

</script>
<body>
<div align="center">
<center>

<table border="0" width="90%" border="0" cellpadding="6" bordercolor="#111111" cellspacing="0"><tr>
<td class="text">

<center>
<a href=index.asp>
<img alt="Engineering Central - Engineering jobs resumes chemical manufacturing electrical civil
mechanical ENGINEERS" src="images/logo.jpg" border="0" width="231" height="84"></a>
</center><hr SIZE="1" color="#000000">
EngCen.com - Engineering Resume - You have seen this resume before 

<ul style="font-size: 13px; font-family: verdana, arial">
<li>

<a href="ResumeSearch.asp">Do a new search</a></li>
<li>
<a href="javascript:ReviseSearch();">Revise this search</a></li>
<li>
<a href="javascript:saveresume(1)">Save this resume</a></li>
<li><a href="EmailCandidate.asp?ResumeID=1">Email this candidate</a></li>
<li>
<a href="javascript:window.history.back();">Back to results</a></li>
<li>
<a href="EmpAccountPage.asp">Your Account Page</a></li>
<li><a href="db/reshelp.asp">Resume Database Help</a></li>
</ul>

<div align="center">
<center>

<form name="form1" action="RunResultsList.asp" method="post">
<input type="hidden" name="keywords" value="">
<input type="hidden" name="searchfor" value="">
<input type="hidden" name="country" value="">
<input type="hidden" name="areacode" value="">
<input type="hidden" name="citizenship" value="">
<input type="hidden" name="state" value="">
<input type="hidden" name="city" value="">
<input type="hidden" name="degree" value="">
<input type="hidden" name="certification" value="">
<input type="hidden" name="datemodified" value="">
<input type="hidden" name="odrfld" value="">
<input type="hidden" name="odrby" value="">
<input type="hidden" name="cPage" value="">

<input type="hidden" name="fname" value="">
<input type="hidden" name="lname" value="">
<input type="hidden" name="phone" value="">
<input type="hidden" name="email" value="">
<input type="hidden" name="daterange1" value="">
<input type="hidden" name="daterange2" value="">
<input type="hidden" name="relocation" value="">
<input type="hidden" name="expyears" value="">
<input type="hidden" name="salary" value="">
<input type="hidden" name="contract" value="">
<input type="hidden" name="formtype" value="">

<table border="1" cellpadding="4" cellspacing="0" style="border-width:0; border-collapse: collapse; " bordercolor="#111111" id="table1" width="100%">
<tr>
<td width="50%" colspan="2" align="center" class="text">
Personal Information</td>

<td width="2%" align="center" class="text"> </td>

<td width="50%" colspan="2" align="center" class="text">

Employment Details</td>
</tr>
<tr>
<td width="25%" class="text">
Name:</td>
<td width="25%" class="text">
[--(First Name)--] [--(Last Name)--]</td>
<td width="1%" class="text"> </td>
<td width="24%" class="text">
Type of Degree:</td>
<td width="25%" class="text">
BS Chemical Engineering, MS Environmental Engineering</td>
</tr>
<tr>
<td width="25%" class="text">
Email:</td>
<td width="25%" class="text">


 <a href="[--(Email Address)--]?subject=Resume Seen on Engineering Central (www.engcen.com)">[--(Email Address)--]</a>

</td>
<td width="1%" class="text"> </td>
<td width="24%" class="text">
Years Experience:</td>
<td width="25%" class="text">
0-2 Years</td>
</tr>
<tr>
<td width="25%" class="text">
Phone:</td>
<td width="25%" class="text">

[--(Phone Number)--]
</td>
<td width="1%" class="text"> </td>
<td width="24%" class="text">
Target Salary:</td>
<td width="25%" class="text">
$40,000-$50,000</td>
</tr>
<tr>
<td width="25%" class="text">
City:</td>
<td width="25%" class="text">
[--(City)--] </td>
<td width="1%" class="text"> </td>
<td width="24%" class="text">
Certifications Held:</td>
<td width="25%" class="text">
None</td>
</tr>
<tr>
<td width="25%" class="text">
State:</td>
<td width="25%" class="text">
[--(State)--]</td>
<td width="1%" class="text"> </td>
<td width="24%" class="text">
Relocation:</td>
<td width="25%" class="text">
Anywhere</td>
</tr>
<tr>
<td width="25%" class="text">
Country:</td>
<td width="25%" class="text">
USA</td>
<td width="1%" class="text"> </td>
<td width="24%" class="text">
Contract Work?</td>
<td width="25%" class="text">
Yes</td>
</tr>
<tr>
<td width="25%" class="text">
Citizenship / Visa Status:</td>
<td width="25%" class="text">
US Citizen</td>
<td width="1%" class="text"> </td>
<td width="24%" valign="top" class="text">
Resume Date:</td>
<td width="25%" valign="top" class="text">
May 10, 2006</td>
</tr>
</table>
<div align="center">
<center>
<table cellPadding="4" border="0" bordercolor="#111111" width="100%" id="table2">
<tr>
<td width="100%">
 
Professional Qualifications
<table border="1" cellpadding="4" cellspacing="0" style="border-width:0; border-collapse: collapse; " bordercolor="#111111" width="100%" id="table3">
<tr>
<td width="19%" valign="top" class="text">Short
Description:</td>
<td width="81%" valign="top" class="text">Environmental engineer with Masters Degree. Willing to travel and/or relocate.</td>
</tr>
<tr>
<td width="19%" valign="top" class="text">
Education:</td>
<td width="81%" valign="top" class="text">EDUCATION 
Masters of Science in Civil and Environmental Engineering, Expected: August 2006<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p></o:p> 
<?xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" /><st1:PlaceType w:st="on">University</st1:PlaceType> of <st1:PlaceName w:st="on">[--(City)--]</st1:PlaceName>, <st1:place w:st="on"><st1:City w:st="on">[--(City)--]</st1:City> <st1:State w:st="on">PA</st1:State></st1:place><o:p></o:p> 
GPA: 3.7 
  
Bachelor of Science in Chemical Engineering, May 2002 
Rensselaer Polytechnic Institute, Troy NY 
  
Courses taken include: 
Environmental Processes I, II 
Environmental Microbiology 
Environmental Impact Assessment 
Kinetics and Catalysis 
Transport Phenomena 
Advanced Thermodynamics 
Introduction to Environmental Engineering 
Organic Chemistry 
Separation Processes 
Experimental Chemistry 
Chemical Engineering Laboratory I, II 
Molecular Separations Engineering 
Chemical Process Design 
 </td>
</tr>
<tr>
<td width="19%" valign="top" class="text">
Description:</td>
<td width="81%" valign="top" class="text">RELEVANT EXPERIENCE 
Water Treatment Plant Intern, July 2005 - Oct 2005 
<?xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" /><st1:City w:st="on">[--(City)--]</st1:City> Water and Sewer Authority, <st1:place w:st="on"><st1:City w:st="on">[--(City)--]</st1:City>, <st1:State w:st="on">PA</st1:State></st1:place> 
<st1:place w:st="on"><st1:State w:st="on"></st1:State></st1:place>• Develop matrix spikes for analysis with HAPSITE GC/MS to determine matrix effects in detecting various organics. 
• Provide support in performing wet chemistry for daily analysis of water samples, including pH, turbidity, and alkalinity and hardness titrations. 
• Assist in monthly cleaning and calibration of turbidometers. 
• Experience with daily sample collection throughout water treatment plant.<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p></o:p> 
 Engineering Intern, June 2000 - August 2000 PPG Industries, Inc., Photochromics Division, Monroeville PA • Support position in Process Development group. • Established baseline operating parameters and wrote user manual for RC1. • Researched heats of crystallization and studied thermodynamic effects on yield and purity for final and intermediate dye products using RC1 calorimeter. • Tracked division inventory using MAX ERP software. • Laboratory experience in preparing and running small-scale and scale-up reactions, running HPLC samples to track reaction progress, and preparing product samples for quality control. OTHER EXPERIENCE Research Specialist, Jan - Aug 2001, Summer 1999 University of [--(City)--] Medical Center, [--(City)--] PA • Experience in creating, organizing, and documenting raw data sets, including the cleaning of raw data files. • Developed and maintained integrity of Access databases to track participant information and data and took responsibility for data- related problems. • Created and presented training manual to study’s interviewers. • Provided clerical, administrative, and technical assistance to principal investigators and data coordinator. SKILLS 
•  Advanced knowledge of Windows 95/98/2000/NT/XP, DOS, Macintosh, and Unix operating systems, Microsoft Office Suite (Excel, Word, Access, PowerPoint), Microsoft Visual C++, Matlab, Maple 
•  Experience with HPLC, NMR, IR, GC, and AA analysis, pH meters, refractometers, RC1, PCR 
•  Proficient in spoken and written French 
 ACTIVITIES • Dean's List, 5 semesters • Social and Recruitment Chair, Psi Upsilon Fraternity • Volunteer, East End Food Co-op </td>
</tr>
</table>
</td>
</tr>
</table>

</center>
</div>

</center>
</div>

<ul style="font-size: 13px; font-family: verdana, arial">
<li>

<a href="ResumeSearch.asp">Do a new
search</a></li>
<li><a href="javascript:ReviseSearch();">Revise this search</a></li>
<li><a href="StoreResume.asp?ResumeID=1">Save this resume</a></li>
<li><a href="EmailCandidate.asp?ResumeID=1">Email this candidate</a></li>
<li><a href="javascript:document.form1.submit();">Back to results</a></li>
<li><a href="EmpAccountPage.asp">Your Account Page</a></li>
<li><a href="db/reshelp.asp">Resume Database Help</a></li>
</ul>



</table></center>

<hr size=1>
Copyright © 2006, Engineering Central 
 

</body>
</html>

gtporter

ASKER

... I replaced the personal iformation with [--(....)--] tags. This information was in the source as plian text.

ASKER CERTIFIED SOLUTION

bpmurray

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

gtporter

ASKER

I'll give that a shot. Thanks. I'll post the resluts.

gtporter

ASKER

Well, I got together with out VB.NET programmer and he worked out an app to extract the data... I think...

Thanks, bpmurray, for your input anyway.

Since you were the only one who responded, you get the points.