Currently, we have a small team of accountants/bookkeepers whose occasional job it is to carefully review a merchant's credit card statement. The purpose is to find and add up all the charges on that month statement, as many are hidden and hard to decipher. Once the total is found, that amount is divided by total gross credit card sales to find the monthly aggregate transaction rate.
This number can be low, like 2.55%, but can be very high, particularly when sales are low. Monthly fixed charges can cause slow months to cost as mush as 10-12%!
Anyway, currently, experts do this analysis, with an eagle eye developed over decades of doing this. They support over 40 different report types, so they need to make judgement calls, based on their collective experiences. I'd like to harness these skills into a system which can provide instant turn-around for this entire effort.
At first, I would have those same experts be the sole users of this system until they find zero errors for a prolonged period of time and/or for particular report formats.
A key element of this process is pattern matching. For example, in theory, each line item in a merchant's credit card monthly statement matches an item in the Interchange table. When there is an exact match, this would enable a validation that the rate charged for that line item matches the rate allowed by law. This also enables the "expert system" to have a high confidence the match is correct.
Conversely, there are some companies that use "monkey business" when creating their reports, seemingly confusing their merchants intentionally. They change the names of these line items entirely, to obscure them, making it harder (or impossible) for a non-expert to audit that report.
There are fixed published lists which could be consumed every time a new version comes out...
And the system would correlate items from the monthly statement to those line items...
So, I need to get a very high level explanation of the kinds of tools I could use for this expert system. There would be heavy use of pattern matching. Would this be simply regular expressions? Or would this require Python or some other language?
And for the training portion, I presume when a match is accepted by an expert, that association would be logged and potentially be a match just for a single credit card processor. It seems they each have their own "language" when they depart from the conventional names found in the Interchange Table.
It will be a web based technology, which does help guide us to the proper choice of this back-end logic needed for this.
Thanks for whatever help you can suggest.
And about about a Rules Engine? Isn't that something at the core of an expert system?