Manage changing Excel formats during ETL - in principle

Hi there,

I'm currently using Talend Open studio Data Integration to managing the importation of several Excel spreadsheets. However, what I've found is that some of these spreadsheets can change slightly: field names change, what was once single sheets are split into multiple sheets, column orders change.

What I want to know is has anyone encountered these problems in ETL design and is it even possible to overcome these issues and if so what technologies were used? Is it even possible in principle to account for changing Excel formats when importing data into a database?


Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

I would use a VBA macro in Excel to validate your sheets before importing them.  A macro-enabled workbook can read/process another workbook, such as your import sheets. The macro can prompt for which file to process, and it can read each column name and sheet name to validate the data. The macro could even rearrange the columns to match your expected format.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Martin LissOlder than dirtCommented:
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Excel

From novice to tech pro — start learning today.