How to implement multi-threaded polling

I have a small single threaded app that continually polls a remote server for new items. If any items are found, it processes a batch of 50 items (max).  After the items are processed it sends a request to the remote server, telling it to mark those 50 items as processed (if successful). Finally it closes the connection and sleeps for a few minutes. Then grabs the next batch of items. In psuedo-code it looks something like this:

    while (!canceled) {

       SomeConnection conn = getConnectionToRemoteServer();
       List<Foo> items = conn.getNewItems(50);

       List<Foo> processed = new ArrayList<Foo>();
       for (Foo foo: items) {
           successful = processItem(foo);

           if (successful) {
              processed.add(foo);
           }
           else {
              doSomethingElse();
           }

           if (canceled) {
               break;
           }
       }  

       conn.removeItems(processed);              
       
       //... close connection
      Thread.sleep(someInterval);
    }

This all works fine. However, I would like to speed up the process by splitting up the work across multiple threads. Ideally, I'd like to submit the 50 tasks for processing, and do something when all tasks are finished. (Then begin again, when the next batch is received). But also support the ability to cancel the current batch at any time and shutdown the app if needed.

I'm looking for general ideas on how to accomplish this. I've been reading about ExcecutorService and CompletionService, and it seems promising, but I'm not clear on how to structure the overall process.  

Again, I'm just looking for high level advice on fitting the various pieces together rather than actual code.  Any thoughts or suggestions would be greatly appreciated!