Maxafi
asked on
Parsing Large XML Files - Web setup/configuration in CF Application
I have a system currently set up that performs the following actions;
1 - Downloads an XML data file
2 - Parses the data into a MySQL database
This is done on a daily basis.
Each XML file may have a slightly different structure, hence I therefore need to tell the system which elements map to which fields in the database.
At the present time I am accomplishing this through Navicat - setting up each import individually, then running them as a scheduled task. However, I would like to be able to set up and manage all of this through a web-based interface. i.e point to an XML file, set up and save the mapping, then run the process automatically each evening.
One important point of note is that the XML data files can be quite large - some will be 300Mb+ in size. Whilst Navicat is capable of handling this, I doubt Coldfusion itself can (unless there is a method I am unaware of?). I'm using a dedicated machine for this task, so additional software to handle it is not problem - I simply want to be able to manage it within my own application, rather than having to open Navicat and configure each one individually. The end goal is a streamlined management system that can be administered via the web. The application is running on Coldfusion MX7.
Any pointers on this would be very much appreciated.
1 - Downloads an XML data file
2 - Parses the data into a MySQL database
This is done on a daily basis.
Each XML file may have a slightly different structure, hence I therefore need to tell the system which elements map to which fields in the database.
At the present time I am accomplishing this through Navicat - setting up each import individually, then running them as a scheduled task. However, I would like to be able to set up and manage all of this through a web-based interface. i.e point to an XML file, set up and save the mapping, then run the process automatically each evening.
One important point of note is that the XML data files can be quite large - some will be 300Mb+ in size. Whilst Navicat is capable of handling this, I doubt Coldfusion itself can (unless there is a method I am unaware of?). I'm using a dedicated machine for this task, so additional software to handle it is not problem - I simply want to be able to manage it within my own application, rather than having to open Navicat and configure each one individually. The end goal is a streamlined management system that can be administered via the web. The application is running on Coldfusion MX7.
Any pointers on this would be very much appreciated.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.