Combine SQL (Derby) databases

I have several (Derby) databases with the same structure (populated by different machines), and want to combine them into one large database (using Java programming).  The tables like tClient, tTransaction, etc. in each database have records that the other databases don't contain.

What are some fast ways of combining the databases into one large database "in a bulk fashion"?

Once I have a large database B and a little database L comes along later, I would like to write all of L's data into B.

I am comparing this "several-databases separately" with using a client-server version of Derby (several databases, each machine can work independently using its own fast resources; client-server, each machine writes to a centralized derby database)


Any and all tips and tricks would be appreciated, I will split the points between all the useful answers.
amp834Asked:
Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

x
 
CEHJConnect With a Mentor Commented:
I don't there's anything special you can do. Simply have the mini dbs do a batch insert into the maxi db

http://www.exampledepot.com/egs/java.sql/BatchUpdate.html
0
 
amp834Author Commented:
Another way to ask the question:  If I'm populating a database with a million records (say, from a CSV file or any format that would make it fast), how should it be done?
0
 
CEHJCommented:
The way i'd do it is to use exactly the method i posted
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
amp834Author Commented:
If you're populating a table for the first time, for example, you may remove all (or most) of the indexes, populate the table, and then add the indexes.  It's also tips and tricks like this I was looking for, that's why I also posted to the general database topic, and not just the java topic.  

I wanted to hear from people who actually do things like this.  Do you think I should submit the question differently?
0
 
CEHJCommented:
>>remove... indexes, populate the table, and then add the indexes.

You could do that, but bear in mind that if there are duplicates, you will need to write another query to find them, as you won't be *able* to add the indexes
0
 
amp834Author Commented:
I was just using an example (and that's why I added "Most indexes")

I was more interested in...
>I wanted to hear from people who actually do things like this.  Do you think I should submit the question differently?
0
 
CEHJCommented:
>>Do you think I should submit the question differently?

Not really: there are not many options if you want to do it in Java
0
 
objectsCommented:
getting rid of indexes is a good idea
generally simpler to do it with a database import tool or script instead of using java
0
 
amp834Connect With a Mentor Author Commented:
I *could* have each machine write data to a text file (or internally formatted binary file), transfer it to the other machine, and then somehow post the data.  The posting does NOT have to be done in Java (thought it's easier for me if it was in Java).  It could be done with a script.  I'm using Derby db.
0
 
objectsConnect With a Mentor Commented:
have a look at the import/export options provided by Derby
http://db.apache.org/derby/docs/10.8/tools/ctoolsimport16245.html
0
 
amp834Author Commented:
This looks promising.
0
 
amp834Author Commented:
Thanks for your help and comments.  I am finally getting around to closing some of the questions I asked!
0
All Courses

From novice to tech pro — start learning today.