asked on

file upload from windows working to one linux server, but not another

I have 3 linux servers running with Ubuntu. All have JBoss 1.5.0 and Java 1.6. All have the save file structures, users, and permissions.

The following code breaks on item.getInputStream:

      private String processFile(DiskFileItem item) {

            File tempDir = null;
            ResourceBundle rBundle = null;

            try {
                  rBundle = ResourceBundle.getBundle(SPREADSHEET_PROPERTIES_FILE_NAME);
                  tempDir = new File(rBundle.getString(FILE_DIR));
            } catch (MissingResourceException mre) {
                  log.warning("in missing resource sxception");
                  rBundle = new MyResources();
                  tempDir = new File(rBundle.getString(FILE_DIR));
            }
            boolean ret = tempDir.mkdirs();
            if (!ret) {
                  log.warning("failed to create directories for "+tempDir.getPath());
            }
            fileName = item.getName().toLowerCase();
            log.info("fileName from item (client) = "+fileName);
            String separator = FORWARD_SLASH;
            if (fileName.indexOf(separator) != (-1))
                  fileName = fileName.substring(fileName.lastIndexOf(separator) + 1);
            separator = DOUBLE_BACK_SLASH;
            if (fileName.indexOf(separator) != (-1))
                  fileName = fileName.substring(fileName.lastIndexOf(separator) + 1);

            String newFileName = SERVER_FILE_PREFIX + new Date().getTime() + fileName;
            //String newFileName = SERVER_FILE_PREFIX + "_" + fileName;
            File newFile = null;
            try {// tempDir.getPath() + File.separator +
                  String fname = tempDir.getPath() + File.separator + newFileName;
                  log.info("filename = " + fname);
                  newFile = new File(fname);
                  newFile.setExecutable(true,false);
                  newFile.setReadable(true,false);
                  newFile.setWritable(true,false);

       if (DevTestProdConstants.DEV_PROD_TEST.equals(DevTestProdConstants.LOCAL))
            FileUtils.copyFile(new File(item.getName()), newFile);
       else {
            newFile.createNewFile();
            //item.write(newFile);
BufferedInputStream in = new BufferedInputStream(
item.getInputStream());
BufferedOutputStream out = new BufferedOutputStream(
new FileOutputStream(newFile));
IOUtils.copy(in, out);
in.close();
out.close();
            newFile.setReadable(true, false);
            log.info("is readable="+newFile.canRead());
            log.info(newFile.getPath() + (newFile.exists() ? " does " : " does NOT ") + " exist");
       }

                  if (c_or_m)
                        saveFile(fileName, newFile.getPath());
                  else
                        updateFile(fileName, newFile.getPath());

                  long id = -1;
                  id = parseFile(newFile);
                  if (id < 0) {
                        workbookId = id;
                  }
//watchWindow("processFile","627");

            } catch (Exception e) {
                  log.warning("Error ingesting spreadsheet file! File: "
                              + newFile.getPath() + "; Error: " + e);
                  String stack = getStackTrace(e);
                  log.warning(stack);
                  workbookId = -1;
            }
            return newFile.getPath();

with the following trace:

2011-05-16 15:32:05,097 INFO [STDOUT] (http-10.20.5.61-8880-4) INFO [gov.nrel.nbc.spreadsheet.server.SpreadSheetUploadServiceImpl.processFile()] fileName from item (client) = us_city_wages.xls
2011-05-16 15:32:05,097 INFO [STDOUT] (http-10.20.52011-05-16 15:32:05,098 INFO [STDOUT] (http-10.20.5.61-8880-4) WARNING [gov.nrel.nbc.spreadsheet.server.SpreadSheetUploadServiceImpl.processFile()] Error ingesting spreadsheet file! File: /usr/local/NREL_ARCHIVE/NBC/SPREADSHEET/spreadsheet1305581525097us_city_wages.xls; Error: java.io.FileNotFoundException: /usr/local/NREL_ARCHIVE/NBC/SPREADSHEET/tmp/upload_1fde2b39_12ff9cbcac0__7fb7_00000011.tmp (No such file or directory)
2011-05-16 15:32:05,098 INFO [STDOUT] (http-10.20.5.61-8880-4) WARNING [gov.nrel.nbc.spreadsheet.server.SpreadSheetUploadServiceImpl.processFile()] java.io.FileNotFoundException: /usr/local/NREL_ARCHIVE/NBC/SPREADSHEET/tmp/upload_1fde2b39_12ff9cbcac0__7fb7_00000011.tmp (No such file or directory)
      at java.io.FileInputStream.open(Native Method)
      at java.io.FileInputStream.<init>(FileInputStream.java:106)
      at org.apache.commons.fileupload.disk.DiskFileItem.getInputStream(DiskFileItem.java:230)
      at gov.nrel.nbc.spreadsheet.server.SpreadSheetUploadServiceImpl.processFile(SpreadSheetUploadServiceImpl.java:865)
      at gov.nrel.nbc.spreadsheet.server.SpreadSheetUploadServiceImpl.doPost(SpreadSheetUploadServiceImpl.java:255)

etc.....

The problem looks like the commons.fileupload.DiskFileItem.write is trying to create a temporary file with a unique GUID in the filename. The directory that it is trying to create the file in is totally open (777) and owned by the process user (jboss).

Any clues would be very much appreciated.

jalbersh2

ASKER

Are there any experts on linux file uploading out there?

jalbersh2

ASKER

Please, please, please try to help me. Start a dialog to hunt down the problem. Anyone!

ASKER CERTIFIED SOLUTION

David Johnson, CD

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

David Johnson, CD

also what are the differences between the working and non working machines?

jalbersh2

ASKER

That's the wierd thing. I can't find any differences. I'm using the same war file on all three machines, running jboss, ubuntu linus, and it works on one but not the others.

jalbersh2

ASKER

Creating the temporary directory didn't work either. I still think it is some weird permission, closed port, or protocol issue on the server-side. Why else would creating the temp file work on one server and not another?

jalbersh2

ASKER

Hello out there! Where are all the experts?

jalbersh2

ASKER

Turns out that the solution was a combination of using the temp directory as well as an increased size for the upload

David Johnson, CD

could you amplify your solution?

jalbersh2

ASKER

Here is what the code was before:

                  FileItemFactory itemFactory = new DiskFileItemFactory();
                  ServletFileUpload upload = new ServletFileUpload(itemFactory);
                  try {
                        List<DiskFileItem> items = (List<DiskFileItem>)upload.parseRequest(req);
                        Iterator<DiskFileItem> it = items.iterator();
                        while (it.hasNext()) {
                              DiskFileItem item = it.next();
                              if (item.isFormField() == false) {
              String fileName = item.getName();
String newFileName = "upload" + new Date().getTime() + fileName;
              File newFile = null;
                   String fname = System.getProperty("java.io.tmpdir") + File.separator + newFileName;
                   log.info("filename = " + fname);
                   newFile = new File(fname);
                   newFile.setExecutable(true);
                   newFile.setReadable(true);
                   newFile.setWritable(true);
                   item.write(newFile);
                              }
                        }
...

Here is the new (Working) code:

       File tdir = createTempDir(tempDir.getPath()); // from suggested method
                  FileItemFactory itemFactory = new DiskFileItemFactory(1000*1024*1024,tdir);
                  ServletFileUpload upload = new ServletFileUpload(itemFactory);
                  try {
                        List<DiskFileItem> items = (List<DiskFileItem>)upload.parseRequest(req);
                        Iterator<DiskFileItem> it = items.iterator();
                        while (it.hasNext()) {
                              DiskFileItem item = it.next();
                              if (item.isFormField() == false) {
              String fileName = item.getName();
String newFileName = "upload" +fileName;
                   String fname = tdir.getPath() + File.separator + newFileName;
                   log.info("filename = " + fname);
                   File newFile = new File(fname);
                   newFile.setExecutable(true);
                   newFile.setReadable(true);
                   newFile.setWritable(true);
                   item.write(newFile);
                              }
                        }
...