Solved

Increasing memory allotted for a Java application when the application starts up

Posted on 2006-11-21
31
245 Views
Last Modified: 2013-11-23
I know that you can raise the memory allowed for your Java application when you run it on a command line by:
>java -Xmx1400m MyApp

But is there a way to do this in the Java code instead, maybe in the main method before you start creating objects and using memory in your Java application?

Thanks
0
Comment
Question by:gdkinney_2
  • 9
  • 8
  • 6
  • +2
31 Comments
 
LVL 14

Accepted Solution

by:
hoomanv earned 125 total points
Comment Utility
I don't think so
0
 
LVL 23

Assisted Solution

by:Ajay-Singh
Ajay-Singh earned 125 total points
Comment Utility
There is a hack you can do to achieve this. Before giving you that idea, I recommend you to evaluate again why you want to increase heap size once jvm has started.


sun.misc.Unsafe.getUnsafe().allocateMemory(1024); // will allocate 1024 bytes more


Note this api is JVM specific and subject to change without prior notice and its discouraged to use this class.


0
 
LVL 26

Assisted Solution

by:ksivananth
ksivananth earned 125 total points
Comment Utility
why do you need to do so?

another option would be launch your pplication thru another java class in it's main method using Runtime!
0
 
LVL 24

Expert Comment

by:sciuriware
Comment Utility
Once I was in a place where I couldn't give commandline arguments so I wrote
a method that verifies the allocation limit and then invokes the application again with the same
JVM and the same commandline but more memory.
It works, but maybe you don't want that.
I could publish the code.
The hint by Ajay-Singh is an experts solution.

Your choice ......

;JOOP!
0
 
LVL 14

Expert Comment

by:hoomanv
Comment Utility
I dont think the memory allocated by Unsafe.allocateMemory() will be counted toward the heap size
Ajay, can you construct objects in the allocated space as before (with 'new' keyword) ?
0
 
LVL 24

Assisted Solution

by:sciuriware
sciuriware earned 125 total points
Comment Utility
This at least works, even if you don't like it. It was deployed for a year by the bank INTERPAY, The Netherlands:

import java.io.File;
import java.io.IOException;
import java.net.URL;

/**
 * Class to have the current application relaunched with enough memory.
 * <p>
 * Usage:
 *
 *         public static void main(String[] argv)
 *         {
 *            try
 *            {
 *               new needMemory().needMemory(400, argv); // 400Mb limit, 200Mb initial.
 *              .......................
 * </p>
 *
 * @author     J.F.Lanting
 * @since      07-Oct-2002
 */
public final class NeedMemory
{
   /**
    * Environment variable to check for recursive re-launching (->'needMemory()').
    */
   private final static String launchToken = "RELAUNCHEDFORHEAP";


   /**
    * Checks if enough memory is available to the VM for this application and optionally re-launches it.
    * <p>
    * If not, the application is re-launched from this copy with:<BR>
    * - the same commandline and<BR>
    * - the same JAVA release.<BR>
    * The operation fails or is blocked because:<BR>
    * 1) this is already a re-launch (something must be wrong),<BR>
    * 2) the amount of memory asked for is > 1500Mb which is out of reach.<BR>
    * The first situation is controlled by a property: 'launchToken'.<BR>
    * We relaunch the application with initially 50% of the heap limit allocated.
    * </p>
    *
    * @param needed # of Mb to establish.
    * @param programCommandLine the command line as appearing in 'main()'.
    * @throws Exception for excess needs or recursive launches (a bug?).
    */
   public void needMemory(int needed, String[] programCommandLine) throws Exception
   {
   long available;                     // What we've got.
   String javaPath;                    // The path to java or java.exe
   URL u;                              // The URL to this class from the classloader.
   String programPath = null;          // The path to this .jar or .class if any.
   File directory;                     // Directory of the .jar or .class file.
   boolean isJarFile;                  // If this is run from a .jar file.
   String[] commandVector;             // The new commandline for re-launch.
   String s;
   int i;

      if((available = toMegaBytes(Runtime.getRuntime().maxMemory())) < needed)
      {
         if(needed > 1500)
         {
            throw new Exception
               ("IMPOSSIBLE TO MEET REQUEST FOR " + needed + " Mb MEMORY.");
         }
         if(System.getProperty(launchToken) != null)
         {
            throw new Exception("FATAL RECURSION IN RE-LAUNCH");
         }

// Where did I come from ?

         u = getClass().getResource("needMemory.class");
         if(u == null)
         {
            throw new Exception("CAN'T FIND MY OWN CLASS FILE.");
         }

// Determine the path to a class-file or to a jar-file from the classloader URL:

         s = u.getPath();
         if(s.startsWith("file:/"))    // It's a jar-file.
         {
            isJarFile = true;
            if(File.separatorChar == '/') // UNIX|LINUX
            {
               s = s.substring(5);     // Leaves a / alone.
            }
            else
            {
               s = s.substring(6);     // Strips the / away.
            }

            if((i = s.indexOf("!/")) > 0)
            {
               s = s.substring(0, i);  // Deletes the internal jar path.
            }
            commandVector = new String[6 + programCommandLine.length];
         }
         else                          // In a class file.
         {
            isJarFile = false;
            if(s.startsWith("/"))
            {
               s = s.substring(1);     // Waste the /.
            }
            if((i = s.indexOf(".class")) > 0)
            {
               s = s.substring(0, i);  // Drop the extension.
            }
            commandVector = new String[5 + programCommandLine.length];
         }

         if(s.indexOf("%20") >= 0) // Incase the URL contained them.
         {
            s = s.replaceAll("%20", " ");
         }
         programPath = s;

// Where to find java[w] :

         javaPath = System.getProperty("java.home");
         if(File.separatorChar == '/') // UNIX|LINUX
         {
            javaPath += "/bin/java";
         }
         else
         {
            javaPath += "\\bin\\javaw.exe";
         }

// Prepare new commandline :

         commandVector[0] = javaPath;

         needed = (int)(needed * 1.01 + 1); // Allow 1% more plus one.

         commandVector[1] = "-Xms" + (needed / 2) + "m"; // Initial heap.
         commandVector[2] = "-Xmx" + needed + "m"; // Maximum heap.
         commandVector[3] = "-D" + "RELAUNCHEDFORHEAP" + "=1";

         directory = new File(programPath);
         if(isJarFile)
         {
            commandVector[4] = "-jar";
            commandVector[5] = directory.getName();
            i = 6;
         }
         else
         {
            commandVector[4] = directory.getName();
            i = 5;
         }
         directory = directory.getParentFile();

         for(int j = 0;  j < programCommandLine.length;  ++j, ++i)
         {
            commandVector[i] = programCommandLine[j];
         }

// Log the action :

         for(i = 0;  i < commandVector.length;  ++i)
         {
            if(i == 0)
            {
               s = commandVector[0];
            }
            else
            {
               s += " " + commandVector[i];
            }
         }
         System.out.println
         (
            "+++ Re-launch, heap=" + available
            + "Mb, requested=" + needed
            + "Mb, from directory " + directory.getAbsolutePath()
            + ", commandline="
         );
         System.out.println(s);

// Re-launch :

         i = launch(commandVector, directory);
         if(i < 0)
         {
            System.out.println("*** RE-LAUNCH ATTEMPT FAILED.");
         }
         else
         {
            System.out.println("### End of re-launch.");
         }
         System.exit(i);
      }
   }


   /**
    * Executes an application and waits for completion.
    *
    * @param commandVector  Command line as array.
    * @param dir directory in which the process runs.
    * @return Exit status from the command.
    */
   public int launch(String[] commandVector, File dir)
   {
   Process p;
   int exitValue = -1;

      if((p = execute(commandVector, dir)) != null)
      {
         try
         {
            exitValue = p.waitFor();
         }
         catch(InterruptedException e)
         {
            exitValue = -1;
         }
      }
      return(exitValue);
   }


   /**
    * Executes an application in a directory, without waiting for it and returns the Process object.
    *
    * @param commandVector  command line as String array.
    * @param dir directory in which the process runs.
    * @return Process object for the child process.
    */
   public static Process execute(String[] commandVector, File dir)
   {
   Process p;

      try
      {
         p = Runtime.getRuntime().exec(commandVector, null, dir);
      }
      catch(IOException e)
      {
         p = null;
      }
      return(p);
   }


   /**
    * Returns a number of bytes as Mb, rounded up.
    *
    * @param m # of bytes
    * @return the # of Mb.
    */
   public static long toMegaBytes(long m)
   {
      return((m / 1024 + 512) / 1024);
   }
}

/*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*/
/*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*/

;JOOP!
0
 
LVL 23

Expert Comment

by:Ajay-Singh
Comment Utility
> Ajay, can you construct objects in the allocated space as before (with 'new' keyword) ?

Yes
0
 
LVL 24

Expert Comment

by:sciuriware
Comment Utility
hoomanv did not provide a solution or prove his case.

;JOOP!
0
 
LVL 35

Expert Comment

by:girionis
Comment Utility
Why do you say that? I think hoomanv is the only 100% correct answer, which is simply "you cannot do it". All the other comments are simply hacks.
0
 
LVL 24

Expert Comment

by:sciuriware
Comment Utility
"You can't" means "give up."

I think that the other workarounds or hacks (as you name them) are faithful
 solutions that can even be applied in a critical environment.
Please put yourself in the place of gdkinney_2: the work must go on anyhow.
What should gdkinney_2 say to his/her boss?
                  "I can't because they say I can't?"
Or:
                   "some guys came forward with several solutions; let's decide upon one".

I'm still content with my solution that I applied in a very critical banking environment.

Concluding: it should have been gdkinney_2's choice; now it is up to you!

I rest my case.

;JOOP!
0
 
LVL 23

Expert Comment

by:Ajay-Singh
Comment Utility
I agree. The features java doesn't provide and there is a need for it - we do need to do hacks!
0
 
LVL 35

Expert Comment

by:girionis
Comment Utility
sciuriware, actually it's not up to me, I am just recommending *possible* (note the word "possible") solutions. It is up to you (the experts) to have a chat whether my suggestion is valid or not and up to a moderator to decide (taking in mind of course all the comments after my recommendation) if my comment will be followed or not. So I *do not* decide (to make things clear) :)

As for the question, I still insist that hoomanv's is a valid answer. Your comment is also valid, since it would work for a *new* application loaded from another java application. Ajay-Singh's is also valid since it could work for some versions of the JVM.
0
 
LVL 14

Expert Comment

by:hoomanv
Comment Utility
> a method that verifies the allocation limit and then invokes the application again with the same JVM and the same commandline but more memory.

Q> Can the memory allocated to JVM increase ?
A> Yes you can restart the JVM with options to get more memory
This is not really a solution nor a hack (because he could launch jvm with maximum available memory in the first step, without first encountering an out of memory and then deciding to allocate it more). We could also buy a new RAM :) and allocate even more

Also Ajay's solution is not fine to me until he proves that the statement { new Object() } will put the new object in the new address space allocated by Unsafe class. otherwise the allocated space does not belong to heap in any case, so you did not actually increase the heap size, you only requested it from OS just like when you call malloc() in C, So another would be to call malloc from JNI and pretend that the heap has been grown
0
 
LVL 24

Expert Comment

by:sciuriware
Comment Utility
To girionis: may be a misunderstanding?
Please review my code: it should be called very early in the application to start a 2nd
instance of the same application with the same java version and commandline.
The 1st instance remains very small and can be swapped.

To hoomanv: my applications were started by clicking an icon. I was not allowed
to set the registry entry to use 1.5Gb every time.
And: I knew how much every application needed, so finally all applications
ran in 2 instances: a small one spawning and a real one with the correct size.
If a mayor bank accepts that I think it is a good solution.

;JOOP!
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 23

Expert Comment

by:Ajay-Singh
Comment Utility
> Also Ajay's solution is not fine to me until he proves that the statement { new Object() } will put the new object in the new address space allocated by Unsafe class.

Have tested it myself (will encourage you to prove it otherwise). As i mentioned in my comment, i don't recommend to use it unless there is really need for it.
0
 
LVL 35

Expert Comment

by:girionis
Comment Utility
Yes but it is not the *very same* instance that is being given more memory, it is another one. If a major bank accepts your code, then it is a solution, but if another organization accepts the "there is no way to do it" quote it is also equally a solution.
0
 
LVL 35

Expert Comment

by:girionis
Comment Utility
My comment was for sciuriware.
0
 
LVL 14

Expert Comment

by:hoomanv
Comment Utility
> will encourage you to prove it otherwise
putObject is for putting objects into allocated space, if { new object() } also does the same, what was the need for those low level put methods ? also why sun leaved such a needful class undocumented ?
http://www.docjar.com/docs/api/ClassLib/Common/sun/misc/Unsafe.html
0
 
LVL 23

Expert Comment

by:Ajay-Singh
Comment Utility
> putObject is for putting objects into allocated space

No, it puts a object only when you create a object otherwise (meaning new XXX(...)) i.e. using reflection or serialization
0
 
LVL 14

Expert Comment

by:hoomanv
Comment Utility
Yes the object should first be created in heap using {new Object()}, then it can travel to that space by calling putObject and the previous heap location can be garbage collected. Therefore the source code needs to be changed in order to store objects into that out of heap memory. garbage collector will not take care of there too.

In this situation I think storing objects in a permanent storage seems more reasonable (even though it reduces the speed)
- using OS virtual memory
- a database approach like hibernate
0
 
LVL 23

Expert Comment

by:Ajay-Singh
Comment Utility
> Yes the object should first be created in heap using {new Object()}, then it can travel to that space by calling putObject and the previous heap location can be garbage collected.

I don't think its true. the call to new XXX(...) takes care of keeping the object to heap for gc - will be same when you call allocateMemory() function.

As i said earlier, putObject is used only to create object from reflection, cloning, reflection and associating fields with the object created
0
 
LVL 14

Expert Comment

by:hoomanv
Comment Utility
And I said you have to prove it

since you have low level access to that space, I don't think this is used as part of the heap, Perhaps thats why it is called unsafe, but why does it allow us to put a single byte anywhere there ? if this space is used by the object allocator simultaneously, how does it make sure that my single byte will not overwritten ?
0
 
LVL 23

Expert Comment

by:Ajay-Singh
Comment Utility
Sun has GPL'ed the source code of Java, look at the source code, and let me know if its otherwise
0
 
LVL 14

Expert Comment

by:hoomanv
Comment Utility
Should I prove the incorrectness ... or YOU prove the correctness ... of your statement ?
0
 
LVL 23

Expert Comment

by:Ajay-Singh
Comment Utility
> Should I prove the incorrectness
please
0
 
LVL 14

Expert Comment

by:hoomanv
Comment Utility
And if I be right ?
0
 
LVL 35

Expert Comment

by:girionis
Comment Utility
Ajay-Singh I think you should prove the correctness of your statement, not hoomanv the incorrectness of it. This is a simple principal of logic, the one who makes an assumption also bears the burden of proving it. If I say that in a small planet one million years away from Earth live ten little green men, I should prove it is correct, not the astrologers that it is wrong.
0
 
LVL 23

Expert Comment

by:Ajay-Singh
Comment Utility
> Ajay-Singh I think you should prove the correctness of your statement
I have already done it. I am not sure what you are looking for. are you looking for code of Unsafe.java?
0
 
LVL 35

Expert Comment

by:girionis
Comment Utility
I personally am not looking for something, I believe you and I think it works (that's why I recommended you also get some part of the points). I am just saying that once you make a statement and asked to prove it you shouldn't transfer the effort of unproving it to someone else.
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
Viewers will learn one way to get user input in Java. Introduce the Scanner object: Declare the variable that stores the user input: An example prompting the user for input: Methods you need to invoke in order to properly get  user input:
Viewers will learn about the regular for loop in Java and how to use it. Definition: Break the for loop down into 3 parts: Syntax when using for loops: Example using a for loop:

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now