Reasonable approach to testing complex software when there's a minor change

I'm involved in a software testing group, it's the basic testing of an existing enterprise system. My question is about testing a fairly complex piece of software, that does this and that, and then inserts data into an Oracle table.

How do you test something like this "to the penny"? In order to have an independent test, you'd kind of have to build the same logic separately. Then if the program changes, you'd have to update your testing module.

Nevertheless, this is a system test and not a unit test. But still, to balance "to the penny", this is more than a causal glance at the numbers, "Hey, looks good!"

In my example, the logic retrieves data from Oracle where [such and such] = A, B, or C, and we changed the logic to add D, E, F. I'm inclined to inspect the code as our verification method. But to write a query to test this, I'm thinking I have to build the actual logic from the program.

Just looking for a reasonable approach to this.
Gadsden ConsultingIT SpecialistAsked:
Who is Participating?
 
Kathy JacobsSocial media geekCommented:
One way to ensure your testing is getting to the exact error? Build a test data set that causes the problem in the old code. Then, run that test data through the new code. This simulates the original issue and verifies that it is fixed. In your case, this would mean having test data that includes criteria D, E, and F. Verify that it fails on the old code, then run the test on the new code. Do the same with A, B, and C to ensure that doesn't fail in some new spectacular way. Once you have done that, then you can expand the data to verify that nothing else falls into the area that was fixed. The approach is to verify both that the code fixes what it is supposed to and doesn't break something unexpectedly.

I know this sounds simplistic, but it is what is going to happen with the fix. As a former tester, I have learned - the best tests are the ones that don't look at what the change does, but rather look at the results of the change. That way, you don't have to reimplement the code. Instead, you reimplement the desired result.

Yes, I do know this approach gets hard the larger and more complex the system gets. But it also gets to the result without risking future problems. Inspecting the code will tell you if the code is right for the exact cases the code was written to handle, but it won't necessarily tell you if the change causes any unexpected situations. Simulating the data involved will ensure that the real life change is handled correctly.

All that said, do the code inspection too. I'm as big of a believer in good inspections as in testing. :)

(Credentials? Yup, I've got them. Tested and broken everything from airplane self check systems to PPT/Word/Excel add-ins.)
1
 
Kyle AbrahamsSenior .Net DeveloperCommented:
I would build a catalog of tests that test each portion of functionality.

EG: If the Machine's function A with inputs 1,2,3 and outputs O1 then write few tests for the input and expected results.

What if you pass in negative numbers, null, string data, etc?

You don't need to rebuild the logic per se but you do need to verify that you're getting expected results along the way.

This way when they add D you can run A,B,C again to verify nothing got affected.
0
 
Gadsden ConsultingIT SpecialistAuthor Commented:
Kyle,

thanks, just looking for a reasonable approach. One takeaway of your comment is that there needs to be a toolkit of testing components.

Your suggestion is more of an integrated test as I understand. A full system test is still another approach.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
Gadsden ConsultingIT SpecialistAuthor Commented:
Kyle, those are great links, thank you!
0
 
Gadsden ConsultingIT SpecialistAuthor Commented:
Kathy,

excellent, thank you.

>>That way, you don't have to reimplement the code. Instead, you reimplement the desired result.
- great perspective

>>the best tests are the ones that don't look at what the change does, but rather look at the results of the change
- great point

>>I'm as big of a believer in good inspections as in testing. :)
- totally agree!
1
 
Gadsden ConsultingIT SpecialistAuthor Commented:
thanks for the discussion!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.