Solved

Transferring data from one system to another

Posted on 2014-07-22
4
288 Views
Last Modified: 2014-08-22
I have recently been thrown into a situation where I'm dealing with financial data (batch processing environment) that is transferred on a daily basis from a Mainframe COBOL system to (after some intermediate steps including ETL) an Oracle system.  As I am not in the IT department, all I really have direct access to is the dollars the Mainframe systems people say they transferred and the dollars the Oracle systems people say they received.  Those dollars may include multiple batches.  As ridiculous as this may seem, whenever there is a problem (for example, there is a multi-day delay in receiving a particular day's work in Oracle), the way the Oracle people are going about trying to find the data (that came into Oracle late)  is by searching Oracle batch reports for a batch or group of batches whose dollars add up to (or closely add up to) the amount the Mainframe people say they transferred. The batches (batch numbers) I am referring to are on the Oracle side only.  They are not batch numbers that are transferred by the Mainframe and captured by Oracle.  So they are of little if any use in cross-referencing between the two systems.

My question is:    What identifying data should be being transferred by the mainframe to Oracle to enable the Oracle IT department to identify which mainframe data was attempted to be transferred, what data actually got into Oracle and what data didn't get in?   The entire problem is currently the inability to match data that was sent with data that was received (and also what data was attempted to be sent but failed).  Thank you.
0
Comment
Question by:dbfromnewjersey
4 Comments
 
LVL 16

Assisted Solution

by:Wasim Akram Shaik
Wasim Akram Shaik earned 125 total points
Comment Utility
There is a broad question.. Overall you are transferring data between two systems.. but how the transmission takes place?? this does really matter to answer your question better.

Is it a file based mechanism... I don't know much of mainframes.. but oracle has all its data in only tables...!!

so if any thing is not there in table means, it has got stuck somewhere in transmission(failed or missed)

To overcome this type of problem probably you need to get the batch numbers from oracle itself, overall it should be like this

1. Mainframe System will ask for oracle to send out the batch number(oracle sequence may be, for suppose), oracle will generate batch number and will update it status as pending
2. Once you receive batch number from oracle , then use it as an acknowledgement and start sending data
3. Once Batch is Sent, Oracle will automatically see the batch number which has been sent and updates corresponding flag as received in Oracle

The steps which were posted above may seem to over burden the process a little bit, but it will help out in such cases where in you have dilemma/difference between systems.
0
 
LVL 37

Assisted Solution

by:Gerwin Jansen
Gerwin Jansen earned 125 total points
Comment Utility
I would say transaction ID's - for each financial data 'record' to be transferred, a unique ID from the COBOL system should be transferred into the Oracle system, as a (source) reference.
0
 
LVL 29

Accepted Solution

by:
MikeOM_DBA earned 125 total points
Comment Utility
As a minimum, you need to control the batches with following numbers:

- Batch number
- Timestamp
- transaction (row) sequence# and/or identification
- Record (row) count per batch
- Sum total of amounts (DB/CR) per batch

At each step (stage) transferred, received, loaded, etc... you should verify the above numbers and reject batch if there is a difference.

Good luck!
;)
0
 
LVL 48

Assisted Solution

by:PortletPaul
PortletPaul earned 125 total points
Comment Utility
There is also a need to understand the method of file handling (I'd be pretty confident this is file based, most M/F interfaces to non-M/F system work that way).

The initial file is probably placed in a fixed location. Where is that?
(nb: What happens if that location isn't available? there should be procedures for this)

Will the M/F produce a file even if there are no transactions to process?
What is the cycle? daily? is the file produced at the same time of day for each cycle?

What is the naming convention of the files? (this can be very important)

What is the file "format"? (e.g. csv?)

What does the recipient system do with the file? Does it get "moved"? to where?
(nb: What happens if that location isn't available? there should be procedures for this)

If the file gets "moved" is it moved according to status?
(e.g. to a "done" folder? to "reject" folder?)

How long are files retained in these locations?
What are the backup regimes?
Are the files archived?

Operations staff should have all of this documented. Where is that documentation?
(although getting operations staff to share documents can be a rare event)
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Join & Write a Comment

Working with Network Access Control Lists in Oracle 11g (part 1) Part 2: http://www.e-e.com/A_9074.html So, you upgraded to a shiny new 11g database and all of a sudden every program that used UTL_MAIL, UTL_SMTP, UTL_TCP, UTL_HTTP or any oth…
Subquery in Oracle: Sub queries are one of advance queries in oracle. Types of advance queries: •      Sub Queries •      Hierarchical Queries •      Set Operators Sub queries are know as the query called from another query or another subquery. It can …
This video shows setup options and the basic steps and syntax for duplicating (cloning) a database from one instance to another. Examples are given for duplicating to the same machine and to different machines
This video shows how to recover a database from a user managed backup

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now