Solved

Web Scraping - I Believe that Is What it is Called

Posted on 2014-02-20
8
245 Views
Last Modified: 2014-05-25
I have a few websites.  They added more ways to accept credit cards (iPad, iPhone, Android, Windows (possibly mobile) and they lowered all the prices.

I have some sites that are going to be PHP and moving the ASP pages to .aspx

Could I create a text file with the prices and have the websites read the data?  Or would it be better to be in databases?

I have images as well - usually float some left, some right - so I would like to stay the same.

For some (i.e. adding new pages), those would need to be added.  But for the pricing for ecommerce  that was 2.09% and now 1.75%.  And added more features to help reduce chargebacks, etc.  

The layouts are a static (header, left nav, right content, footer), (header, left content, right nav, footer), (header, left nav, main content, right nav/info, footer), and then a version for tablets and mobiles.
0
Comment
Question by:coreybryant
  • 4
  • 3
8 Comments
 
LVL 74

Assisted Solution

by:käµfm³d 👽
käµfm³d   👽 earned 75 total points
ID: 39875382
Could I create a text file with the prices and have the websites read the data?
You could.

Or would it be better to be in databases?
Most definitely.
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 39876600
Web "Scraping" generally refers to the process of reading the HTML document from a URL and programmatically teasing out some of the information.  It's the equivalent of using "view source" and reading the markup as well as the visual data, and it's possible to do this (some of the time) with PHP.

I have a few websites.  They added more ways...
What are the URLs of the sites?  Do you control the content or are the sites controlled by others?  What is your level of skill in PHP programming?
0
 
LVL 29

Author Comment

by:coreybryant
ID: 40089462
I know what web scraping it, had to do it a few for some clients. Some of the websites will be .NET and some PHP. Also mobile and tablet version

I thought maybe a simple XML for the websites on the same server /company.  The merchant account prices are going down and we are adding hew products.

I just want to add the prices for the discount rate,  transaction rates, etc.  Right now, they are going down and we are adding iPad, iPod, Windows Phone, Android.

I could put   MySQL for the {PHP sites) and MS Access  / MSSQL on the windows.  The sites are at least ten or more, re-doing them to focus on the type.  LoudestCommerce

The reason I thought XML - it might be faster.  I would like to use only one XML file but I want the site to load quickly
0
 
LVL 108

Accepted Solution

by:
Ray Paseur earned 425 total points
ID: 40089512
XML is just a transport mechanism -- it's used to mark up data with semantic tags so you can send data from one place to another, preserving the meaning.  The XML might be created from information in a data base.  You can hand-code XML if you want to, but I would think that programatically generating it would be a better idea.

XML is declining in popularity when compared to JSON.  JSON is "friendly" with a variety of programming languages and is more efficient in terms of the propellant-to-payload ratio.  The JSON markup is more compact and does not require the "end" tags for each field.  PHP has some ways of converting XML to JSON.
http://php.net/manual/en/function.simplexml-load-string.php
http://php.net/manual/en/function.json-encode.php
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 29

Author Comment

by:coreybryant
ID: 40089570
Thanks - I am not doing this - having someone else to help me code.  I did ASP, but since that is disappearing, I wanted something pretty easy.  I guess my theory was that I make changes to one page and then it gets copied to the others, but of course, that also makes the same info on the same page(s).

I just have a lot to do and I am helping our HOA with a new website - easier for the users to update and change.  

I have one more (new) question about this one
0
 
LVL 29

Author Closing Comment

by:coreybryant
ID: 40089574
Thanks for the ideas, the more little bits that I know, the developer should (have hired some developers that did not complete the job because the did not know how.
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 40089621
Thanks for the points.  I understand there is a "Hire Me" button available here at EE.  If you need a knowledgeable developer, you might find a good resource right here!
0
 
LVL 29

Author Comment

by:coreybryant
ID: 40089743
Thanks, I might look into that also!
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

Suggested Solutions

Whether you’re a college noob or a soon-to-be pro, these tips are sure to help you in your journey to becoming a programming ninja and stand out from the crowd.
This article discusses four methods for overlaying images in a container on a web page
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.

759 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now