wsyy
asked on
How to use a static instance in multi-threading application
Hi,
I originally had a multi-threading application, each of the threads creating a list of objects and writing the list to database. This method creates a lot of database IO, thereby slowing down the application.
I now want to create a static list, and have each thread add objects to the list and save the list of objects to database whenever the size of list reaches 5000.
How can I pass through the static list to each of the threads effectively?
I currently define the static list in theIncremental.java which utilizes the Java ExecutorSerices framework.
as follow:
public static List<DBObject> toMove;
The way to call and use the toMove variable is:
synchronized (Incremental.toMove) {
for (String code : code2erid.keySet()) {
DBObject q = new BasicDBObject();
q.put("code", code);
DBObject o = NeferCollection.getInstanc
.findOne(q);
o.put("erid", code2erid.get(code));
o.removeField("_id");
Incremental.toMove.add(o);
int size = Incremental.toMove.size();
if (size % 1000 == 0 && size > 0) {
ProdCollection.getInstance
.insert(Incremental.toMove
Incremental.toMove.clear()
System.out.println("Moving
}
}
}
My question is whether my way is effective to pass through a static object into each thread.
I originally had a multi-threading application, each of the threads creating a list of objects and writing the list to database. This method creates a lot of database IO, thereby slowing down the application.
I now want to create a static list, and have each thread add objects to the list and save the list of objects to database whenever the size of list reaches 5000.
How can I pass through the static list to each of the threads effectively?
I currently define the static list in theIncremental.java which utilizes the Java ExecutorSerices framework.
as follow:
public static List<DBObject> toMove;
The way to call and use the toMove variable is:
synchronized (Incremental.toMove) {
for (String code : code2erid.keySet()) {
DBObject q = new BasicDBObject();
q.put("code", code);
DBObject o = NeferCollection.getInstanc
.findOne(q);
o.put("erid", code2erid.get(code));
o.removeField("_id");
Incremental.toMove.add(o);
int size = Incremental.toMove.size();
if (size % 1000 == 0 && size > 0) {
ProdCollection.getInstance
.insert(Incremental.toMove
Incremental.toMove.clear()
System.out.println("Moving
}
}
}
My question is whether my way is effective to pass through a static object into each thread.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Hi rumi,
The code does insert records to MongoDB in a batch of 1000.
The reason why to clear the list is to keep only new records in the toMove list. (If not cleared, all the old records would be inserted multiple times.)
The issue is, as you point out, the size checking is performed every loop, which may explains why this multi-threading code is outperformed by non-multi-threading code.
Surely, that is not what I want :)
The code does insert records to MongoDB in a batch of 1000.
The reason why to clear the list is to keep only new records in the toMove list. (If not cleared, all the old records would be inserted multiple times.)
The issue is, as you point out, the size checking is performed every loop, which may explains why this multi-threading code is outperformed by non-multi-threading code.
Surely, that is not what I want :)
The challenge is to make it access this safely.
I would suggest using:
public static List<DBObject> toMove = new CopyOnWriteArrayList<DBObj
which creates a list which can be accessed simultaneously by multiple threads without throwing exceptions. If you use just a "normal" list like new ArrayList<DBObject>() you'll get exceptions if one thread reads from the list while another thread is modifying it.
Doug