Link to home
Start Free TrialLog in
Avatar of marrowyung
marrowyung

asked on

database synchronization between traditional RDMS and NoSQL

hi,

any  tools/idea you all use to synchronization between traditional RDMS and NoSQL in MS SQL and maria DB and noSQL?
Avatar of David Favor
David Favor
Flag of United States of America image

This almost always requires custom code, as each combination will have complex indexing (SQL side) which must be normalize going to simple indexing (no-SQL side).
Avatar of marrowyung
marrowyung

ASKER

how about ETL and replication server like oracle goldengate, sybase replication serer ?
Same answer. You're moving from complex SQL indexing to simple non-SQL indexing, hashed lookups.

Each of these types of moves tend to be unique in their design.
so can't do much on this ! no tools for it, right?
No tools for this. You'd write custom tools designed like this.

1) Open connections to database + no-SQL store.

2) Generate your custom SQL SELECT calls.

3) Convert retrieved data into one or more no-SQL writes.

4) Do all your writes.

5) Rinse + repeat for all SQL data tables.

6) Close all your connections.

When I have a client requesting this type of work, I just use PERL to quickly whip up a one-time, throw-away script.
you are PERL programmer ?
I write PERL code pretty much every day, for projects + command line tools I use with my private hosting clients.

I've been using PERL since... 1994, so I'm guessing I qualify as a PERL programmer.

I only work on code for sites I host though + I'm very expensive.

Likely your best bet is to open an EE Gig + someone can assist you with your project.
"I only work on code for sites I host though + I'm very expensive."

ahahha, a lot of people here say the same thing.

other than PERL,  c# can do that too ?
All languages are just languages. So any language will work.

That said, if I had a client suggest c# I'd try to contain my laughter... probably couldn't...

I'd figure 10x to 100x the time for c# instead of PERL.

Maybe longer, as your first step will be getting c# to actually work under Linux... as I'm hoping you're using Linux for your project.

If you're imagining Microsoft as your target OS... whew... add 10x to every minor daily task.

You're going to require a massive budget for your project is you're using non-Linux based solution.

https://LaunchSpeedHosting.com covers the LAMP Stack code versions my team currently uses for projects.

I always use Ubuntu for host + container OS, then run LXD at the machine level, where sites/projects all live in their own LXD containers.

You may find this helpful also, a you can clone a production container into a dev container in a few seconds.

This also allows you to deploy a dev site as a production site in roughly 30 seconds.
"Maybe longer, as your first step will be getting c# to actually work under Linux... as I'm hoping you're using Linux for your project."

excellent  !

but I heard that Tungsten replicator 3.0 can replicate from MySQL / oracle to NoSQL like Hadoop, right? it seems nearly real time, right?
ASKER CERTIFIED SOLUTION
Avatar of David Favor
David Favor
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
"
vhttps://mariadb.com/kb/en/library/full-text-index-overview/ covers details of how this works.
"
oh, by using MariaDB ?
you mean Maria DB can do a lot in this area?


"Based on your questions, likely best if you hire someone to assist you with your project."
you mean Database developer ?
tks.