asked on

Data archiving and replicating database

Hi,

I need advise with regards to archiving old data from our prod database
since db size has increased
All tables in the database do not have a date column but they do relate to the tables
which have date columns
Users might require to access this old database for reporting or data analysis
once in a while, so it needs to exist on a remote server for them to access

Currently this prod database is getting replicated to a reporting server.
So now once the data is archived we still would require to replicate both the archived and
the current prod data to the reporting server on a single db

Please suggest how this can be achieved??

Thanks

Ryan McCauley

For starters, does historic data change at all? If so, how far back? I ask because, if historic data is static, you don't need to worry about ongoing maintenance - once it's been replicated to the reporting server, you no longer need to keep it syncronized and you can delete it from the production server (or move it to the archive server you're referencing). Also, are you actually using SQL Replication, or are you using "Replication" to mean some kind of ongoing ETL process that moves older data to your reporting server?

If you're using SQL Replication, then both copies of the database (production and reporting) need to be exactly the same, data-wise. If they're the same, then why do you need to archive server? It seems like you'd just be maintaining a second copy of the data that's already on your reporting server.

If you're using some kind of ETL process, then you're free to delete data from your source server without worrying about the impact on the reporting server, so you can go forward with the Archive server you're talking about. You can set up a second ETL process (or extend the existing one) to also copy historic data to the archive server, and then delete it from your source server. As long as the records your working with have a date you can attach to them (through a join, it sounds like), then you can trim old data from your production server and move it to your archive.

Since the data has already been moved from production to the reporting server previously, you don't need to worry about syncing up anything between the archive and reporting servers.

SP_2018 .

ASKER

Hi,

The historic data would not change but we need to archive this old data to archive server
as well replicate the old and the current data to the reporting server with some kind of
ETL strategy. Currently we are using transactional repication to replicate the data from
Prod server to the reporting server.
Do you think if this could be achieved using merge replication between the production data and the archived data.

Also not all the tables on the prod server has date column in it , but the ones which do not
have date column are referencing the tables which do have date column, so can
we still archive the data from all tables.

Is their some kind of reference you could provide which explains such a solution.

Thanks

ASKER CERTIFIED SOLUTION

Ryan McCauley

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial