Solved

Handling large text files with JTextPane

Posted on 2004-04-07
13
989 Views
Last Modified: 2007-12-19
Is there a restriction concerning the size of (text) files when trying to display them in a JTextPane?

I have trace files which can have a size of 10MB, 20MB, or even 100MB and when I try to display the files content within a JTextPane I get an OutOfMemoryError. How can I handle this best?

By now I just read the file's content with a BufferedReader into a StringBuffer to display the StringBuffer's content within a JTextPane. On later stages I want to group the source file's content and display these groups in different JTextPanes.

The error occurs when executing this statement:

this.replaceSelection(sb + "\n");

(Where 'sb' is the StringBuffer instance obviuosly).

How can I avoid getting this OutOfMemoryError?
0
Comment
Question by:dirku
  • 6
  • 5
  • 2
13 Comments
 

Expert Comment

by:RobCSP
Comment Utility
In my opinion you can  split the text in files of 1Mb size

  File df = new File(getDirName() + java.io.File.separator + getFileName());
        if (df.length()/1024>sizeKb)
             df.renameTo(new File("nameOftheLogfile"));


You can control the name with a counter or something like that.......
0
 

Expert Comment

by:RobCSP
Comment Utility
I´have been testing in my computer and when a file exceed 5Mb the OutofMemory error occurs.

Your application must control the size of the files that will be opened to avoid this error.


SizeKb=1024;
getFileName() --> get the name of the file
getDirName()  --> get the name of the dir

nameOftheLogfile--> name of the new log file
i.e.
logfile1
logfile2
logfile3, etc...

Hope it helps ;-P
0
 
LVL 92

Accepted Solution

by:
objects earned 125 total points
Comment Utility
> Is there a restriction concerning the size of (text) files when trying to display them in a JTextPane?

As you have discivered the limitation is effectively memory.

you can use the -Xm command line option to increase the amount of memory available.
eg. to make 100Mb avbailable use:
-Xmx100m
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
0
 

Author Comment

by:dirku
Comment Utility
Oops. My acceptance maybe was too quick?
I shortened the trace file for testing purposes and then tried your suggestion.
Now, after using the original trace file with ~16MB I get the OutOfMemoryError again although I passed -Xmx60MB as a command line option.

:-(
0
 

Author Comment

by:dirku
Comment Utility
Even with -Xmx100m...
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 92

Expert Comment

by:objects
Comment Utility
Can u produce a small example that reproduces the problem that i could try here?
0
 

Author Comment

by:dirku
Comment Utility
How could I do that?
I could zip my four small classes and the 16MB trace file and send it to you via email if you want.
0
 

Author Comment

by:dirku
Comment Utility
BTW:
I am currently running the app from within JBuilder 9 but since I pass the command line parameter this should not be the problem.
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
just post the code, without the 16mb trace
0
 

Author Comment

by:dirku
Comment Utility
Main class:

package advantage.utils;

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Dimension;

import java.io.File;

import javax.swing.Box;
import javax.swing.BoxLayout;
import javax.swing.JFileChooser;
import javax.swing.JFrame;
import javax.swing.JButton;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;

import javax.swing.text.BadLocationException;

//import java.awt.HeadlessException;
import java.util.ResourceBundle;
import java.util.Locale;

/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p><br>This TraceAnalyzer uses a {@link TraceFileParser} to group the traces
 * within a trace file. These grouped traces will then be displayed with an
 * alternating background color to identify the {@link TraceGroup}s easier.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceAnalyzer extends JFrame
{
  private transient ResourceBundle rbundle = null;
  private transient TraceFileParser tfParser = null;
  private transient TraceGroup[] traceGroups = null;
  private Color bgColor = Color.gray;
  private Color box1Color = Color.white;
  private Color box2Color = Color.gray.brighter();

  private JPanel mainPane = new JPanel();
  private JButton testButton = null;
  private JFileChooser fileChooser = null;
  //----------------------------------------------------------------------------

  /**
   * The <code>TraceAnalyzer</code> default constructor initializes the
   * application and make it visible.
   */
  public TraceAnalyzer() //throws HeadlessException
  {
    init();
    this.setVisible(true);
  }//end default constructor
  //----------------------------------------------------------------------------

  /**
   * Initializes the <code>TraceAnalyzer</code>.
   */
  private void init()
  {
    //Handle the window closing event.
    addWindowListener(new java.awt.event.WindowAdapter()
    {
      public void windowClosing(java.awt.event.WindowEvent evt)
      {
        exitAnalyzer(evt);
      }
    });

    initConfiguration();
    initGuiComponents();
  }//end init
  //----------------------------------------------------------------------------

  /**
   * Initializes this <code>TraceAnalyzer</code>'s configuration using underlying
   * property files/ResourceBundles.
   */
  private void initConfiguration()
  {
   try
   {
     rbundle = ResourceBundle.getBundle("traceAnalyzerConf");
     this.bgColor = Color.decode(rbundle.getString("TraceAnalyzerBgColor"));
     this.box1Color = Color.decode(rbundle.getString("TraceViewer1Color"));
     this.box2Color = Color.decode(rbundle.getString("TraceViewer2Color"));
     String locale_lang = rbundle.getString("locale_language");
     String locale_country = rbundle.getString("locale_country");
     Locale locale = new Locale(locale_lang, locale_country);
     rbundle = ResourceBundle.getBundle("traceAnalyzer", locale);
   }
   catch(Exception e)
   {
      e.printStackTrace();
   }
    int width = Integer.parseInt(rbundle.getString("width"));
    int height = Integer.parseInt(rbundle.getString("height"));

    this.setSize(width, height);
  }//end initConfiguration
  //----------------------------------------------------------------------------

  /**
   * Initializes the GUI components of the <code>TraceAnalyzer</code>.
   */
  private void initGuiComponents()
  {
    //The layout of the application itself.
    this.getContentPane().setLayout(new BorderLayout());

    testButton = new JButton("test");
    this.getContentPane().add(testButton, BorderLayout.PAGE_START);

    //BoxLayout bl = new BoxLayout(mainPane, BoxLayout.PAGE_AXIS);
    //mainPane.setLayout(bl);
    mainPane.setBackground(Color.orange);

    JScrollPane scrollPane = new JScrollPane(mainPane);

    this.getContentPane().add(scrollPane, BorderLayout.CENTER);

    addListeners();
  }//end initGuiComponents
  //----------------------------------------------------------------------------

  private void initTraceGroups()
  {
    //traceGroups = new TraceGroup[1];
    //traceGroups[0] = tfParser.parseFileByString("");
//traceGroups = tfParser.parseFileByString("");
TraceGroup[] trGrps = tfParser.parseFileByString("");
//System.out.println("traceGroups.length = " + traceGroups.length);
System.out.println("trGrps.length = " + trGrps.length);

traceGroups = new TraceGroup[trGrps.length+1];
    System.out.println("traceGroups.length = " + traceGroups.length);
System.arraycopy(trGrps, 0, traceGroups, 0, trGrps.length);
traceGroups[traceGroups.length-1] = new TraceGroup(0, new StringBuffer("qqqqqqqq\n"));

/*
    traceGroups = new TraceGroup[] {
            new TraceGroup(0, new StringBuffer("qqqqqqqq")),
            new TraceGroup(1, new StringBuffer("rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr")),
            new TraceGroup(2, new StringBuffer("sssssssssssss\ntttttttttttttttt\n\nsfghdfsaba\nsadfgafega\n\nsdfbgadfg\ndasfgadfgd\nasdffsadsadfgsdfg\nadfvbaddbsdbsadbsdfbsbnkdfckmgadfbknbjadnbjadnfbjnsadjfbnandfbalsndfbjsandfbjnlsa\n\nkjnfvkjanfkjadnfbvkjadnkja\nskdfnkjsfb\nakvnkjfgnkjqaw\nadnfkj\n"))
        };
*/
  }//end initTraceGroups
  //----------------------------------------------------------------------------

  /**
   * Registers listeners to particular components to handle events.
   */
  private void addListeners()
  {
    testButton.addActionListener(new ActionListener()
    {
      public void actionPerformed(ActionEvent e)
      {
        fileChooser = new JFileChooser();
        if (e.getSource() == testButton) {
          int returnVal = fileChooser.showOpenDialog(TraceAnalyzer.this);
          if (returnVal == JFileChooser.APPROVE_OPTION) {
            File file = fileChooser.getSelectedFile();
            tfParser = new TraceFileParser(file);
            initTraceGroups();
            runTraceViewer();
          } //end if(returnVal == JFileChooser.APPROVE_OPTION)
        } //end if(e.getSource() == testButton)
      } //end actionPerformed
    });
  } //end addListeners
  //----------------------------------------------------------------------------

  /**
   *
   */
  private void runTraceViewer()
  {
    System.out.println("traceGroups.lenght = " + traceGroups.length);
      if (traceGroups != null)
      {
        for (int i = 0; i < traceGroups.length; i++)
        {
          Box box = Box.createVerticalBox();
          box.setAlignmentY(BoxLayout.PAGE_AXIS);
          box.setBackground(Color.darkGray);

          TraceGroup traceGroup = traceGroups[i];
          TraceViewer tv = new TraceViewer(traceGroup);
System.out.println("tv" + i + " created.");
          System.out.println(tv.getTraceGroup().getTraces().toString());
          if (i % 2 == 0)
          {
            tv.setBackground(this.box1Color);
          }
          else
          {
            tv.setBackground(this.box2Color);
          }

          box.add(tv);
          mainPane.add(box);
          mainPane.add(Box.createVerticalStrut(5));

          mainPane.revalidate();
          mainPane.repaint();
        } //end for i
System.out.println("Each of " + traceGroups.length + " Boxes created.");
      } //end if(traceGroups != null)
  }//end runTraceViewer
  //----------------------------------------------------------------------------

  /**
   * Returns the {@link TraceGroup}s which are in fact the instantiated
   * {@link TraceViewer}s.
   * @return The {@link TraceGroup}s handled by this <code>TraceAnalyzer</code>.
   */
  public TraceGroup[] getTraceGroups()
  {
    return this.traceGroups;
  }//end getTraceGroups
  //----------------------------------------------------------------------------

  private void exitAnalyzer(java.awt.event.WindowEvent evt)
  {
    System.gc();
    System.exit(0);
  }//end exitAnalyzer
  //----------------------------------------------------------------------------

  public static void main(String[] args) //throws HeadlessException
  {
    TraceAnalyzer tracy = new TraceAnalyzer();
  }//end main
}//end class TraceAnalyzer

//*********************************************************************

Class intended to parse the trace file in the future:

package advantage.utils;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.StringTokenizer;


/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p><br>This <code>TraceFileParser</code> takes a trace file which is conform
 * to the Siemens AG conventions.</p>
 * A trace con consist of the following parts although it is not mandatory that
 * each of these components are in a particular trace:</p>
 * <ul>
 *   <li>Timestamp (date and time with milliseconds)</li>
 *   <li>Process ID</li>
 *   <li>Thread ID</li>
 *   <li>Trace topic</li>
 *   <li>Trace level</li>
 *   <li>CFRAME tag</li>
 *   <li>container_name::CAT type name</li>
 *   <li>container_name::CAT instance ID</li>
 *   <li>CAT type</li>
 *   <li>Component ID</li>
 *   <li>Context ID</li>
 *   <li>Context type ID</li>
 * </ul>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceFileParser
{
  private File tFile = null;
  private TraceGroup[] traceGroups = null;
  //----------------------------------------------------------------------------

  public TraceFileParser()
  {
  }//end default constructor
  //----------------------------------------------------------------------------

  public TraceFileParser(File file)
  {
    tFile = file;
  }//end standard constructor
  //----------------------------------------------------------------------------

  /**
   * Parses a file to find matches for the given pattern.
   * @param pattern The pattern to look for within the file to parse.
   */
  public TraceGroup[] parseFileByString(String pattern)
  {
    TraceGroup tGroup = new TraceGroup();
    traceGroups = new TraceGroup[1];
    StringBuffer sb = new StringBuffer();
    FileReader fr = null;
    BufferedReader bufr = null;
    try
    {
      fr = new FileReader(tFile);
      bufr = new BufferedReader(fr);
      String line = "";
      while ((line = bufr.readLine()) != null)
      {
        sb.append(line);
        sb.append("\n");
      } //end
      tGroup.setId(1);
      tGroup.setTraces(sb);
System.out.println(sb.length());
System.out.println(tFile.length());
    }
    catch(FileNotFoundException fnfE)
    {

    }
    catch(IOException ioE)
    {

    }
    finally
    {
      try
      {
        bufr.close();
        fr.close();
      }
      catch(IOException innerIoE)
      {

      }
    }
traceGroups[0] = tGroup;
    //return tGroup;
    return traceGroups;
  }//end parseFileByString
  //----------------------------------------------------------------------------
}//end class TraceFileParser

//**********************************************************************

Class to display the text within a JTextPane:

package advantage.utils;

import java.awt.Color;

import javax.swing.SwingUtilities;
import javax.swing.JTextPane;

import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;
import javax.swing.text.DefaultStyledDocument;
import javax.swing.text.MutableAttributeSet;
import javax.swing.text.SimpleAttributeSet;
import javax.swing.text.Style;
import javax.swing.text.StyleConstants;
import javax.swing.text.StyleContext;
import javax.swing.text.StyledDocument;

/**
 * <p>Title: TraceViewer</p>
 * <p>Description: The viewing component of the {@link TraceAnalyzer}.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceViewer extends JTextPane
{
  private transient DefaultStyledDocument doc = null;
  private transient StyleContext sCtxt = null;
  private TraceGroup traceGroup = null;
  //----------------------------------------------------------------------------

  public TraceViewer(TraceGroup tGroup)
  {
    traceGroup = tGroup;
    sCtxt = new StyleContext();
    doc = new DefaultStyledDocument(sCtxt);
    this.setDocument(doc);

    final StringBuffer sb = traceGroup.getTraces();
this.setBackground(Color.green);
final MutableAttributeSet par_1 = new SimpleAttributeSet();
StyleConstants.setForeground(par_1, Color.black);
StyleConstants.setBackground(par_1, Color.gray);

/*
    final Style pStyle_1 = sContext.addStyle("paragraph_1_bg", null);
    final Style pStyle_2 = sContext.addStyle("paragraph_2_bg", null);
    final Style mainStyle = sContext.addStyle("mainStyle", sContext.getStyle(StyleContext.DEFAULT_STYLE));

    StyleConstants.setLeftIndent(mainStyle, 0);
    StyleConstants.setRightIndent(mainStyle, 80);

    pStyle_1.addAttribute(StyleConstants.FontSize, new Integer(20));
    pStyle_1.addAttribute(StyleConstants.Foreground, Color.blue);
    pStyle_1.addAttribute(StyleConstants.Background, Color.red);
*/
doc.setParagraphAttributes(0, 1, par_1, true);
this.replaceSelection(sb.toString());
//System.out.println(sb.toString());
/*
    try
    {
      SwingUtilities.invokeAndWait(new Runnable() {
        public void run() {
          try {
          }
          catch (BadLocationException ble) {

          }
        }
      });
    }
    catch(Exception e)
    {

    }
*/
    this.setEditable(false);
  }//end default constructor
  //----------------------------------------------------------------------------

  /**
   * Feeds the <code>TraceViewer</code> with the {@link TraceGroup} to be shown.
   * @param group The {@link TraceGroup} the be displayed through this
   * <code>TraceViewer</code>.
   * @author Dirk Ulrich
   */
  public void setTraceGroup(TraceGroup group)
  {
    this.traceGroup = group;
  }//end setTraceGroup
  //----------------------------------------------------------------------------

  public TraceGroup getTraceGroup()
  {
    return this.traceGroup;
  }//end getTraceGroup
  //----------------------------------------------------------------------------
}//end class TraceViewer

//**********************************************************************

Class to group the parts of the trace file (grouped by particular criterias in the future):

package advantage.utils;

/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceGroup
{
  private int id = -1;
  private StringBuffer traces = null;
  //----------------------------------------------------------------------------

  public TraceGroup()
  {
  }//end default constructor
  //----------------------------------------------------------------------------

  public TraceGroup(int id)
  {
    this.id = id;
  }//end constructor
  //----------------------------------------------------------------------------

  public TraceGroup(int id, StringBuffer text)
  {
    this.id = id;
    this.traces = text;
  }//end constructor
  //----------------------------------------------------------------------------

  public void setId(int id)
  {
    this.id = id;
  }//end setId
  //----------------------------------------------------------------------------

  public int getId()
  {
    return this.id;
  }//end getId
  //----------------------------------------------------------------------------

  public void setTraces(StringBuffer text)
  {
    this.traces = text;
  }//end setTraces
  //----------------------------------------------------------------------------

  public StringBuffer getTraces()
  {
    return this.traces;
  }//end getTraces
  //----------------------------------------------------------------------------
}//end class TraceGroup

//**********************************************************************

Necessary property file TraceAnalyzerConf.properties:

width = 800
height = 600

locale_language = en
locale_country = GB
#locale_language = en
#locale_country = US
#locale_language = de
#locale_country = DE

TraceAnalyzerBgColor = #c0c0c0
TraceAnalyzerBgColor_R = 225
TraceAnalyzerBgColor_G = 225
TraceAnalyzerBgColor_B = 225

TraceViewer1Color = #ffffff
TraceViewer2Color = #e1e1e1
0
 

Author Comment

by:dirku
Comment Utility
There is a System.arraycops(...); in method initTraceGroups() but the OutOfMemoryError occurs even without the arrayCopy statement.
BTW: Is there a way to load a large text file into the Java app (JTextPane) and show the data already read and continue loading the remaining data in the background?
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
the setPage() method loads the page asynchronously i think.
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
return in catch statement 1 39
pairs challenge 5 44
unix in java example 9 37
JUnit 4 @Before and @BeforeClass differences 3 19
By the end of 1980s, object oriented programming using languages like C++, Simula69 and ObjectPascal gained momentum. It looked like programmers finally found the perfect language. C++ successfully combined the object oriented principles of Simula w…
Are you developing a Java application and want to create Excel Spreadsheets? You have come to the right place, this article will describe how you can create Excel Spreadsheets from a Java Application. For the purposes of this article, I will be u…
Video by: Michael
Viewers learn about how to reduce the potential repetitiveness of coding in main by developing methods to perform specific tasks for their program. Additionally, objects are introduced for the purpose of learning how to call methods in Java. Define …
The viewer will learn how to implement Singleton Design Pattern in Java.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now