We help IT Professionals succeed at work.

Parse HTML Table and store in MySQL

3,239 Views
Last Modified: 2013-11-19
I'm trying to get the daily prices of stocks from a webpage (Its a table) and store them in a MySQL database. Thats my initial goal in a much bigger project.
So i want to be able to fetch the table periodically (every work day preferably) and parse the table for specific fields and then store the table in a MySQL Database. I then want to do some calculations and present them on a web page (This will remain static, using the DB until the next parsing)

These are the tools i have at my disposal
PHP, MySQL and A Linux server that can run Cron Jobs ( +Bash Scripts then )
I might have a little more tools like Ruby installed on the server but i would like to limit it to that.

I have basic knowledge of what i am about to get into (HTML/PHP/MySQL) so my question won't be specific but i would like someone to direct me in a path to start.

No offense, but pointing me to a web page that teaches PHP/MySQL in general with no concrete link (or examples) to what i'm trying to do doesn't help much


PS>> If its easier.. for starters i might want to parse the HTML and store it in a flat file and then fetch numbers from the flat file to the webpage

Thanks
Comment
Watch Question

Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Commented:
While a very relevant question, it is irrelevant if the user is storing and using for their own personal use and not trying to resell or use any other way.   All data that is returned is for free for that user, you do what you wish with it, so, let's just hope and assume this individual isn't attempting to scrape a site and then use that data for non-personal use.

You did make me wonder and a quick check of marketwatch.com site has the following:
"All information provided by ComStock, Inc.("ComStock") and its affiliates (the "ComStock Information") is ownedby or licensed to ComStock and its affiliates and any user is permittedto store, manipulate, analyze, reformat, print and display the ComStockInformation only for such users personal use. In no event shall anyuser publish, retransmit, redistribute or otherwise reproduce anyComStock Information in any format to anyone, and no user shall use anyComStock Information in or in connection with any business orcommercial enterprise, including, without limitation, any securities,investment, accounting, banking, legal or media business or enterprise."

So, unless otherwise I assume this individual is getting the data for their own personal use.

Commented:
You are welcome to delete his question, doesn't matter to me, he simply asked how to store the data in a database and present on a webpage, I have no idea if he plans to then distribute that page to anyone other than himself and it is fairly irrelevant to me, I am not supplying him anything that violates any copyright and he can use what people have recommended without any violation but if you are concerned then just delete the question.  A quick google search for him would have worked to find screen scraping programs ;)


Author

Commented:
Its personal use... I'm trying to somehow parse the data for the relevant stock prices and create some sort of portfolio management thingie. Basically an extremely crippled/bare version of google finance portfolio...

Also, unless their tables include special things like analysis or archived share prices... I don't see why things like the daily prices & volume should be proprietary data. Its also an international (non-US) stock exchange. But i understand.. laws are laws... I'll abide... its actually more for educational use that it would be for actual use because i don't see if being fully functional (to a satisfactory level) anytime soon

Anyways, I've seen a few starting positions.. points all around to the most helpful post with more points to the first.

Thanks

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.