Solved

Data manipulation in Excel

Posted on 2011-03-01
7
406 Views
Last Modified: 2012-08-13
I would like an expert's opinion on something.  So, this does not have a specific answer.

Here is my goal:
I would like to take structured data from an excel sheet and create a financial statement with it.  I would like the user to be able to select a few criteria using a combobox control and then push a button and up pops the financial report.

Here is what I am considering:
- I know one popular path to what I am doing is to use pivottables which uses the efficient pivotcache.  I could programmatically pull data from the pivottable and feed it into the financial statement.
Positives: This is absolutely what most people would recommend and I would have a lot of example code to use.  Also, this may be the fastest since it uses an internal data store native to excel called pivotcaches.
Negatives: I have used pivottables before with vba and found it to be clumsy.  It was difficult to trap errors when I used it.  I also had to debug and view the pivot tables and found it added an extra layer of programming.  I know this is probably the best method to make the most efficient use of memory, but is this really the most stable method?

- I could also convert the structured data into xml and pull data from the xml table created directly into my report.
Positives:  the data verification in excel for xml is strong
Negatives: There is not too much example code in xml.  

- I could simply pull the structured data into a sheet and use the standard "collection" in vba to persist the data in active memory.  I could filter the data programmatically and then dump it into a report format.
Positives: I have greater control in this method
Negatives: This punishes system resources the most

More Background:
The structured data will have around 5,000 rows with 200 unique columns.
The resulting financial statement will consolidate this into around 30 rows and 12 columns.
The user will be a intermediate user who has no desire to manipulate the data with pivottables or otherwise.  S/he will want to simply press a button and have the financial statement appear.  However, wait times such as 3 or 4 minutes are acceptable.

My Status:
I am between a power user and an expert.  I can use vba, classes, and interfaces.

So, specifically, if you were in my shoes, what approach would you take to my project?

Thanks in advance for any comments.

0
Comment
Question by:Opie1313
  • 4
  • 2
7 Comments
 
LVL 45

Expert Comment

by:patrickab
ID: 35011581
Opie1313,

>So, specifically, if you were in my shoes, what approach would you take to my project?

I would establish an Excel database with all the data in it that I will need to assemble the required reports/summaries. I would then use SUMPRODUCT() to extract what was needed to create the reports/summaries. That would make it highly flexible and easy to maintain.

Patrick
0
 
LVL 30

Expert Comment

by:SiddharthRout
ID: 35011622
I would go for this


- I could simply pull the structured data into a sheet and use the standard "collection" in vba to persist the data in active memory.  I could filter the data programmatically and then dump it into a report format.
Positives: I have greater control in this method
Negatives: This punishes system resources the most

At the same time I will try and use stuff like

1) Application.Screenupdating = False
2) Avoid use of .Select and .Activate
3) Minimal use of variables etc...

Sid
0
 
LVL 45

Expert Comment

by:patrickab
ID: 35011784
I would make every effort to:

1.  avoid like the plague using VBA. The reason for that is that VBA is ALWAYS slower than native Excel functions.

2. Avoid using the filter function. Instead use SUMPRODUCT() as it extracts only what is needed and sums it on the fly.

3. Avoid using Pivot Tables as they are clunky and inflexible. They have their place but not for what you want to do.

Patrick
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 
LVL 45

Accepted Solution

by:
patrickab earned 500 total points
ID: 35011872
I would also ensure that:

1. There were no hyperlinks to anything

2. All the data and all the reports were in one file. That would maximimse speed and simplify formulae enormously.

3. Avoid using volatile Excel functions - they are:

Volatile functions in Excel: RAND(), NOW(), TODAY(), OFFSET(), CELL(), INDIRECT(), INFO().

Despite claims, these are not volatile: INDEX(), ROWS(), COLUMNS(), AREAS().


Patrick
0
 

Author Closing Comment

by:Opie1313
ID: 35012216
Had some nice comments to help with my solution.
0
 

Author Comment

by:Opie1313
ID: 35012222
Thanks Patrick for the help.  I will stay away from those volatile functions that you mentioned.
0
 
LVL 45

Expert Comment

by:patrickab
ID: 35012237
Opie1313 - Thanks for the grade - Patrick
0

Featured Post

What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Dealing with unintended Excel Active-X resizing quirks (VBA code simulates "self correction") David Miller (dlmille) Intro Not everyone is a fan of Active-X controls in spreadsheets (as opposed to the UserForm approach, the older Form controls …
Introduction While answering a recent question (http:/Q_27311462.html), I created an alternative function to the Excel Concatenate() function that you might find useful.  I tested several solutions and share the results in this article as well as t…
The viewer will learn how to use the =DISCRINV command to create a discrete random variable, use this command to model a set of probabilities and outcomes in a Monte Carlo simulation, and learn how to find the standard deviation of a set of probabil…
This Micro Tutorial will demonstrate in Microsoft Excel how to add style and sexy appeal to horizontal bar charts.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now