Looking for the best way to divide a List of objects equally among multiple instances of a class that processes those objects, using the Executor framework, and a fixed number of threads, say 10. As an simplified example, assume we have a List of 10 million File objects that need to be parsed and those that contain some string of characters are to be deleted. One simple way would be to divide the List into 10 new Lists, each containing 1 million objects; create an instance of the FileProcessor class, which implements Runnable, passing its allocation of the first 1 million File objects to it; then calling ThreadPoolExecutor's execute method, passing in that instance of FileProcessor; then create a new List of the next one million, etc.
My question is, is there a better way to do this that does not require creating those 10 new "sub-lists?" For one thing, it gets more complicated when the division does not yield an even number; for another, it predetermines what each thread is responsible for, and if a thread happens to crash, its one million objects (or some portion thereof) don't get processed.