Website programming - how to populate a field by fetching data from another page?

First off, I am not a programmer or web designer. I am a network admin.

I am trying to help a friend with an ecommerce website. He is reselling items for vendors who send him a .CSV file with products to advertise on his site. One of the fields is producturl which is just the URL pointing to the product on THEIR website. It is easy enough to import the CSV file to create the products on his site, but is there a way to somehow fetch the data in the product description on the vendor's web site to automatically populate the item description on his site? There is a combined total of around 4,000 items so it would take a very long time to manually create the product description and formatting for each and every product. And since it's already been done on their site it would be nice to just copy it somehow. There is a field for 'Body HTML code' in the product import CSV template on his site, and the product description is an HTML table on their site. Is there such code that can copy the HTML table from their site and paste it into the Body field of the CSV file using the producturl? I hope this makes sense.

Any thoughts? Is this even possible?
LVL 5
CoSmismgrAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jason C. LevineDon't talk to me.Commented:
is there a way to somehow fetch the data in the product description on the vendor's web site to automatically populate the item description on his site?

Yes, this is called "scraping" but if you are not familiar with web scripting, you have some learning to do.

Basically you would need to write a script that goes through each of the product links in the CSV and download the HTML (a GET operation).  You then need to use a regular expression to match up the container that has the description, extract it to variable and write that variable back.  Instead of dealing with a CSV, I would assume the contents of the CSV are put into a database on your friend's site so I would do most of my interacting with that.

There are other possibilities.  I would be shocked if the source of the file isn't producing an XML or JSON feed of their items.  If so, you could import that fairly easily.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
CoSmismgrAuthor Commented:
I would be shocked if the source of the file isn't producing an XML or JSON feed of their items.  If so, you could import that fairly easily.

I am going to see if I can find out if they offer this; if it's easier then it would be the way to go ;) Can you give me any more info about this?
Jason C. LevineDon't talk to me.Commented:
Since large e-commerce sites are almost always database-based, one thing they can do is expose their current product list as an XML or JSON feed for Google.  This is usually superior to a CSV export that is manually generated because it stays up to date automatically.  Add a new product? It shows up in the feed.  Remove one? It disappears.

If they do this, it's likely that all the data you would need to replicate the products on your end exists in the feed.  Then it becomes a simpler matter of writing a script to consume the feed at regular intervals and output data used to power your friend's site.
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

CoSmismgrAuthor Commented:
Thanks for the info Jason, I am definitely out of my element lol

I will leave this question open for a few days just to see if others can add to it.
Jason C. LevineDon't talk to me.Commented:
Yeah, sorry. You didn't exactly pick an easy one to jump in with both feet.  From a completely non-technical standpoint, your best bet may be to work with the data source and see if they can add that info to the CSV.
Scott Fell, EE MVEDeveloper & EE ModeratorCommented:
If they are giving you a url that has the html, then there is a good chance your not supposed to be scrapping the data anyway.  I would look at the terms of service for that.  What you can do is load that url in a frame and style the frame so it it looks like it is part of the site.  

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe
<h2>Product title from your csv file</h2>
<iframe width="400" height="215" frameborder="0" 
   src="http://the_url_from_your_csv_file.com/somefolder/link">
</iframe>

Open in new window

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
HTML

From novice to tech pro — start learning today.