We help IT Professionals succeed at work.

Parsing Large XML Files - Web setup/configuration in CF Application

Maxafi
Maxafi asked
on
456 Views
Last Modified: 2013-12-24
I have a system currently set up that performs the following actions;

1 - Downloads an XML data file
2 - Parses the data into a MySQL database

This is done on a daily basis.

Each XML file may have a slightly different structure, hence I therefore need to tell the system which elements map to which fields in the database.

At the present time I am accomplishing this through Navicat - setting up each import individually, then running them as a scheduled task.  However, I would like to be able to set up and manage all of this through a web-based interface.  i.e point to an XML file, set up and save the mapping, then run the process automatically each evening.

One important point of note is that the XML data files can be quite large - some will be 300Mb+ in size.  Whilst Navicat is capable of handling this, I doubt Coldfusion itself can (unless there is a method I am unaware of?).  I'm using a dedicated machine for this task, so additional software to handle it is not problem - I simply want to be able to manage it within my own application, rather than having to open Navicat and configure each one individually.  The end goal is a streamlined management system that can be administered via the web.  The application is running on Coldfusion MX7.

Any pointers on this would be very much appreciated.
Comment
Watch Question

Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.