Web Service method call returning 404

It definitely sounds as if you are saturating the service. Either your bandwidth, or the remote image service that you are connecting to.

404 errors are pretty generic HTTP errors. They basically just say that the service you are trying to get to cannot be reached or is temporarily unavailable. That's why even if you don't have connection to the internet at all sometimes when you open your browser and try to get to any webpage you'll get a 404 error. Although, browsers are getting better now at reporting your connection status.

Some services will can be lazy and report 404 errors even when more a more appropriate HTTP error code would better describe the situation.

Here's a link to a list of possible HTTP error codes: http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Thanks for this information.  Since 404 is a pretty generic error code, as you said, can you suggest a method that I could use to verify what is going on behind the scenes to cause this?  Could the web service call be timing out (which would be directly related to low or insufficient bandwidth)?  

Bandwidth issues are some of the harder ones to fix. If I were you I would take the following action:

1. I would call the service I'm uploading the images to and see if they have a maximum capacity that they allow and if so, I would change my uploading program to adhere to them.
2. If not, I would figure out how much upload bandwidth I am currently allotted by my ISP. There are quite a few sites out there that can help you do this, but I would recommend either: http://www.speakeasy.net/speedtest/ or http://www.speedtest.net/ Since you are uploading images, you'll want to make sure to watch for your "Upload Speed" not your download speed. Also, note that speed tests are done in "bits" per second note "bytes" the difference is the lower case b as opposed to the upper case B. The difference between the two is 8 times (in other words, there are 8 bits in a byte)
   
3. I would then monitor your system in a high upload period, sometime when there are around 20 page calls at a time (as you mentioned in your post), and check to see how much you are uploading at a time. There are quite a few tools that monitor this, recent versions of Windows (Vista or 7) and Windows Server have a tool called "Resource Monitor" which you can find by opening the start menu and typing in "resource monitor" and then go to the Network tab. If your server is a Linux server, I would recommend installing a package called "bmon" (found here: http://freshmeat.net/projects/bmon/) to monitor network bandwidth. Notice that the upper case B in both of these tools indicating that they are reporting in Bytes per second, divide it by 8 to get bits per second. Again, you'll want to pay attention to the uploaded bytes (Send in the resource monitor, TX in bmon).
   
4. The end result of steps 2 and 3 is that you want to see how much of your total upload bandwidth you are using. If the numbers from step 3 are close to the numbers from step 2 then you know that you are saturating your total upload bandwidth. Most ISPs give you significantly less upload bandwidth than download bandwidth, so its a high possibility.

One other test that I could think of, is if you have access to two computers and two different bandwidth pipes you could set both of them to uploading a large amount of photos and see whether you're still getting caped out at approximately 20 calls combined. This would prove that it is the service that is limiting you, not the bandwidth.

In my previous post I wrote to divide the bytes by 8 to get bits. This is wrong. You want to multiple the bytes by 8 to get bits. Sorry for the confusion.

maybe you could pause a little between page uploads giving the server time to catch up..

Its true, all of my posts have been about diagnosis. If you wanted an easy out, you could just try pausing a little as GeneralTackett mentioned. Or you could try to queue up you image uploads so that you are only uploading 10-15 at a time. In heavier loads, the images would queue until it reached a time of lower load when the program could flush the queue.

Thanks for the suggestions.

First, I wrote both the client and the web service and while I cannot tell you the exact capacity of the web service method, it has successfully uploaded hundreds of pages at a time in a test environment with "good bandwidth".  In this case less than 50 pages is being uploaded at a time.  So it is not capacity of the web service method.

The production web service is installed on a web server in a central location and the client is at remote site in a different city.  Both the web server and the client workstation are on the customer's private network.  We are a software vendor who created the software but we are not going to be the ones to determine how to correct the bandwidth issue (if it is a bandwidth issue).  We just want to be able to report to the customer the "root cause" of the issue.  From what has been said, do you feel more confident that this is a bandwidth-related issue?

Regarding pausing, I think I need to clarify something.   The client makes a single web service call that uploads all of the pages of a document at one time.  It does not make one call per page.  All of the pages are sent in one block of bytes that is then separated on the web server end.  So I do not see how pausing applies to that.

And the number of pages sent with each upload is the number of pages for a scanned document.  All the pages are sent at once, along with meta data, in order to create a document.  Splitting up the files into smaller chunks is something we could consider doing if the issue is due to bandwidth and the customer cannot find a way to resolve it.  It would be a significant change in the existing software architecture so we would want to avoid that if possible.

Ah.. you couljd also post them via a state machine making sure only one posted at a time instead of sending them all at once.  (if you are sending them all at once)  as shadowcat said..

I am accepting Shadowcat's solution, who I think has offered all the feedback he can at this point.  We are going to further monitor this situation.  Thanks.