We help IT Professionals succeed at work.

Check out our new AWS podcast with Certified Expert, Phil Phillips! Listen to "How to Execute a Seamless AWS Migration" on EE or on your favorite podcast platform. Listen Now

x

Handling large text files with JTextPane

dirku
dirku asked
on
Medium Priority
1,047 Views
Last Modified: 2007-12-19
Is there a restriction concerning the size of (text) files when trying to display them in a JTextPane?

I have trace files which can have a size of 10MB, 20MB, or even 100MB and when I try to display the files content within a JTextPane I get an OutOfMemoryError. How can I handle this best?

By now I just read the file's content with a BufferedReader into a StringBuffer to display the StringBuffer's content within a JTextPane. On later stages I want to group the source file's content and display these groups in different JTextPanes.

The error occurs when executing this statement:

this.replaceSelection(sb + "\n");

(Where 'sb' is the StringBuffer instance obviuosly).

How can I avoid getting this OutOfMemoryError?
Comment
Watch Question

Commented:
In my opinion you can  split the text in files of 1Mb size

  File df = new File(getDirName() + java.io.File.separator + getFileName());
        if (df.length()/1024>sizeKb)
             df.renameTo(new File("nameOftheLogfile"));


You can control the name with a counter or something like that.......

Commented:
IĀ“have been testing in my computer and when a file exceed 5Mb the OutofMemory error occurs.

Your application must control the size of the files that will be opened to avoid this error.


SizeKb=1024;
getFileName() --> get the name of the file
getDirName()  --> get the name of the dir

nameOftheLogfile--> name of the new log file
i.e.
logfile1
logfile2
logfile3, etc...

Hope it helps ;-P
Java Developer
CERTIFIED EXPERT
Top Expert 2010
Commented:
Unlock this solution with a free trial preview.
(No credit card required)
Get Preview
Mick BarryJava Developer
CERTIFIED EXPERT
Top Expert 2010

Commented:

Author

Commented:
Oops. My acceptance maybe was too quick?
I shortened the trace file for testing purposes and then tried your suggestion.
Now, after using the original trace file with ~16MB I get the OutOfMemoryError again although I passed -Xmx60MB as a command line option.

:-(

Author

Commented:
Even with -Xmx100m...
Mick BarryJava Developer
CERTIFIED EXPERT
Top Expert 2010

Commented:
Can u produce a small example that reproduces the problem that i could try here?

Author

Commented:
How could I do that?
I could zip my four small classes and the 16MB trace file and send it to you via email if you want.

Author

Commented:
BTW:
I am currently running the app from within JBuilder 9 but since I pass the command line parameter this should not be the problem.
Mick BarryJava Developer
CERTIFIED EXPERT
Top Expert 2010

Commented:
just post the code, without the 16mb trace

Author

Commented:
Main class:

package advantage.utils;

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Dimension;

import java.io.File;

import javax.swing.Box;
import javax.swing.BoxLayout;
import javax.swing.JFileChooser;
import javax.swing.JFrame;
import javax.swing.JButton;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;

import javax.swing.text.BadLocationException;

//import java.awt.HeadlessException;
import java.util.ResourceBundle;
import java.util.Locale;

/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p><br>This TraceAnalyzer uses a {@link TraceFileParser} to group the traces
 * within a trace file. These grouped traces will then be displayed with an
 * alternating background color to identify the {@link TraceGroup}s easier.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceAnalyzer extends JFrame
{
  private transient ResourceBundle rbundle = null;
  private transient TraceFileParser tfParser = null;
  private transient TraceGroup[] traceGroups = null;
  private Color bgColor = Color.gray;
  private Color box1Color = Color.white;
  private Color box2Color = Color.gray.brighter();

  private JPanel mainPane = new JPanel();
  private JButton testButton = null;
  private JFileChooser fileChooser = null;
  //----------------------------------------------------------------------------

  /**
   * The <code>TraceAnalyzer</code> default constructor initializes the
   * application and make it visible.
   */
  public TraceAnalyzer() //throws HeadlessException
  {
    init();
    this.setVisible(true);
  }//end default constructor
  //----------------------------------------------------------------------------

  /**
   * Initializes the <code>TraceAnalyzer</code>.
   */
  private void init()
  {
    //Handle the window closing event.
    addWindowListener(new java.awt.event.WindowAdapter()
    {
      public void windowClosing(java.awt.event.WindowEvent evt)
      {
        exitAnalyzer(evt);
      }
    });

    initConfiguration();
    initGuiComponents();
  }//end init
  //----------------------------------------------------------------------------

  /**
   * Initializes this <code>TraceAnalyzer</code>'s configuration using underlying
   * property files/ResourceBundles.
   */
  private void initConfiguration()
  {
   try
   {
     rbundle = ResourceBundle.getBundle("traceAnalyzerConf");
     this.bgColor = Color.decode(rbundle.getString("TraceAnalyzerBgColor"));
     this.box1Color = Color.decode(rbundle.getString("TraceViewer1Color"));
     this.box2Color = Color.decode(rbundle.getString("TraceViewer2Color"));
     String locale_lang = rbundle.getString("locale_language");
     String locale_country = rbundle.getString("locale_country");
     Locale locale = new Locale(locale_lang, locale_country);
     rbundle = ResourceBundle.getBundle("traceAnalyzer", locale);
   }
   catch(Exception e)
   {
      e.printStackTrace();
   }
    int width = Integer.parseInt(rbundle.getString("width"));
    int height = Integer.parseInt(rbundle.getString("height"));

    this.setSize(width, height);
  }//end initConfiguration
  //----------------------------------------------------------------------------

  /**
   * Initializes the GUI components of the <code>TraceAnalyzer</code>.
   */
  private void initGuiComponents()
  {
    //The layout of the application itself.
    this.getContentPane().setLayout(new BorderLayout());

    testButton = new JButton("test");
    this.getContentPane().add(testButton, BorderLayout.PAGE_START);

    //BoxLayout bl = new BoxLayout(mainPane, BoxLayout.PAGE_AXIS);
    //mainPane.setLayout(bl);
    mainPane.setBackground(Color.orange);

    JScrollPane scrollPane = new JScrollPane(mainPane);

    this.getContentPane().add(scrollPane, BorderLayout.CENTER);

    addListeners();
  }//end initGuiComponents
  //----------------------------------------------------------------------------

  private void initTraceGroups()
  {
    //traceGroups = new TraceGroup[1];
    //traceGroups[0] = tfParser.parseFileByString("");
//traceGroups = tfParser.parseFileByString("");
TraceGroup[] trGrps = tfParser.parseFileByString("");
//System.out.println("traceGroups.length = " + traceGroups.length);
System.out.println("trGrps.length = " + trGrps.length);

traceGroups = new TraceGroup[trGrps.length+1];
    System.out.println("traceGroups.length = " + traceGroups.length);
System.arraycopy(trGrps, 0, traceGroups, 0, trGrps.length);
traceGroups[traceGroups.length-1] = new TraceGroup(0, new StringBuffer("qqqqqqqq\n"));

/*
    traceGroups = new TraceGroup[] {
            new TraceGroup(0, new StringBuffer("qqqqqqqq")),
            new TraceGroup(1, new StringBuffer("rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr")),
            new TraceGroup(2, new StringBuffer("sssssssssssss\ntttttttttttttttt\n\nsfghdfsaba\nsadfgafega\n\nsdfbgadfg\ndasfgadfgd\nasdffsadsadfgsdfg\nadfvbaddbsdbsadbsdfbsbnkdfckmgadfbknbjadnbjadnfbjnsadjfbnandfbalsndfbjsandfbjnlsa\n\nkjnfvkjanfkjadnfbvkjadnkja\nskdfnkjsfb\nakvnkjfgnkjqaw\nadnfkj\n"))
        };
*/
  }//end initTraceGroups
  //----------------------------------------------------------------------------

  /**
   * Registers listeners to particular components to handle events.
   */
  private void addListeners()
  {
    testButton.addActionListener(new ActionListener()
    {
      public void actionPerformed(ActionEvent e)
      {
        fileChooser = new JFileChooser();
        if (e.getSource() == testButton) {
          int returnVal = fileChooser.showOpenDialog(TraceAnalyzer.this);
          if (returnVal == JFileChooser.APPROVE_OPTION) {
            File file = fileChooser.getSelectedFile();
            tfParser = new TraceFileParser(file);
            initTraceGroups();
            runTraceViewer();
          } //end if(returnVal == JFileChooser.APPROVE_OPTION)
        } //end if(e.getSource() == testButton)
      } //end actionPerformed
    });
  } //end addListeners
  //----------------------------------------------------------------------------

  /**
   *
   */
  private void runTraceViewer()
  {
    System.out.println("traceGroups.lenght = " + traceGroups.length);
      if (traceGroups != null)
      {
        for (int i = 0; i < traceGroups.length; i++)
        {
          Box box = Box.createVerticalBox();
          box.setAlignmentY(BoxLayout.PAGE_AXIS);
          box.setBackground(Color.darkGray);

          TraceGroup traceGroup = traceGroups[i];
          TraceViewer tv = new TraceViewer(traceGroup);
System.out.println("tv" + i + " created.");
          System.out.println(tv.getTraceGroup().getTraces().toString());
          if (i % 2 == 0)
          {
            tv.setBackground(this.box1Color);
          }
          else
          {
            tv.setBackground(this.box2Color);
          }

          box.add(tv);
          mainPane.add(box);
          mainPane.add(Box.createVerticalStrut(5));

          mainPane.revalidate();
          mainPane.repaint();
        } //end for i
System.out.println("Each of " + traceGroups.length + " Boxes created.");
      } //end if(traceGroups != null)
  }//end runTraceViewer
  //----------------------------------------------------------------------------

  /**
   * Returns the {@link TraceGroup}s which are in fact the instantiated
   * {@link TraceViewer}s.
   * @return The {@link TraceGroup}s handled by this <code>TraceAnalyzer</code>.
   */
  public TraceGroup[] getTraceGroups()
  {
    return this.traceGroups;
  }//end getTraceGroups
  //----------------------------------------------------------------------------

  private void exitAnalyzer(java.awt.event.WindowEvent evt)
  {
    System.gc();
    System.exit(0);
  }//end exitAnalyzer
  //----------------------------------------------------------------------------

  public static void main(String[] args) //throws HeadlessException
  {
    TraceAnalyzer tracy = new TraceAnalyzer();
  }//end main
}//end class TraceAnalyzer

//*********************************************************************

Class intended to parse the trace file in the future:

package advantage.utils;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.StringTokenizer;


/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p><br>This <code>TraceFileParser</code> takes a trace file which is conform
 * to the Siemens AG conventions.</p>
 * A trace con consist of the following parts although it is not mandatory that
 * each of these components are in a particular trace:</p>
 * <ul>
 *   <li>Timestamp (date and time with milliseconds)</li>
 *   <li>Process ID</li>
 *   <li>Thread ID</li>
 *   <li>Trace topic</li>
 *   <li>Trace level</li>
 *   <li>CFRAME tag</li>
 *   <li>container_name::CAT type name</li>
 *   <li>container_name::CAT instance ID</li>
 *   <li>CAT type</li>
 *   <li>Component ID</li>
 *   <li>Context ID</li>
 *   <li>Context type ID</li>
 * </ul>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceFileParser
{
  private File tFile = null;
  private TraceGroup[] traceGroups = null;
  //----------------------------------------------------------------------------

  public TraceFileParser()
  {
  }//end default constructor
  //----------------------------------------------------------------------------

  public TraceFileParser(File file)
  {
    tFile = file;
  }//end standard constructor
  //----------------------------------------------------------------------------

  /**
   * Parses a file to find matches for the given pattern.
   * @param pattern The pattern to look for within the file to parse.
   */
  public TraceGroup[] parseFileByString(String pattern)
  {
    TraceGroup tGroup = new TraceGroup();
    traceGroups = new TraceGroup[1];
    StringBuffer sb = new StringBuffer();
    FileReader fr = null;
    BufferedReader bufr = null;
    try
    {
      fr = new FileReader(tFile);
      bufr = new BufferedReader(fr);
      String line = "";
      while ((line = bufr.readLine()) != null)
      {
        sb.append(line);
        sb.append("\n");
      } //end
      tGroup.setId(1);
      tGroup.setTraces(sb);
System.out.println(sb.length());
System.out.println(tFile.length());
    }
    catch(FileNotFoundException fnfE)
    {

    }
    catch(IOException ioE)
    {

    }
    finally
    {
      try
      {
        bufr.close();
        fr.close();
      }
      catch(IOException innerIoE)
      {

      }
    }
traceGroups[0] = tGroup;
    //return tGroup;
    return traceGroups;
  }//end parseFileByString
  //----------------------------------------------------------------------------
}//end class TraceFileParser

//**********************************************************************

Class to display the text within a JTextPane:

package advantage.utils;

import java.awt.Color;

import javax.swing.SwingUtilities;
import javax.swing.JTextPane;

import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;
import javax.swing.text.DefaultStyledDocument;
import javax.swing.text.MutableAttributeSet;
import javax.swing.text.SimpleAttributeSet;
import javax.swing.text.Style;
import javax.swing.text.StyleConstants;
import javax.swing.text.StyleContext;
import javax.swing.text.StyledDocument;

/**
 * <p>Title: TraceViewer</p>
 * <p>Description: The viewing component of the {@link TraceAnalyzer}.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceViewer extends JTextPane
{
  private transient DefaultStyledDocument doc = null;
  private transient StyleContext sCtxt = null;
  private TraceGroup traceGroup = null;
  //----------------------------------------------------------------------------

  public TraceViewer(TraceGroup tGroup)
  {
    traceGroup = tGroup;
    sCtxt = new StyleContext();
    doc = new DefaultStyledDocument(sCtxt);
    this.setDocument(doc);

    final StringBuffer sb = traceGroup.getTraces();
this.setBackground(Color.green);
final MutableAttributeSet par_1 = new SimpleAttributeSet();
StyleConstants.setForeground(par_1, Color.black);
StyleConstants.setBackground(par_1, Color.gray);

/*
    final Style pStyle_1 = sContext.addStyle("paragraph_1_bg", null);
    final Style pStyle_2 = sContext.addStyle("paragraph_2_bg", null);
    final Style mainStyle = sContext.addStyle("mainStyle", sContext.getStyle(StyleContext.DEFAULT_STYLE));

    StyleConstants.setLeftIndent(mainStyle, 0);
    StyleConstants.setRightIndent(mainStyle, 80);

    pStyle_1.addAttribute(StyleConstants.FontSize, new Integer(20));
    pStyle_1.addAttribute(StyleConstants.Foreground, Color.blue);
    pStyle_1.addAttribute(StyleConstants.Background, Color.red);
*/
doc.setParagraphAttributes(0, 1, par_1, true);
this.replaceSelection(sb.toString());
//System.out.println(sb.toString());
/*
    try
    {
      SwingUtilities.invokeAndWait(new Runnable() {
        public void run() {
          try {
          }
          catch (BadLocationException ble) {

          }
        }
      });
    }
    catch(Exception e)
    {

    }
*/
    this.setEditable(false);
  }//end default constructor
  //----------------------------------------------------------------------------

  /**
   * Feeds the <code>TraceViewer</code> with the {@link TraceGroup} to be shown.
   * @param group The {@link TraceGroup} the be displayed through this
   * <code>TraceViewer</code>.
   * @author Dirk Ulrich
   */
  public void setTraceGroup(TraceGroup group)
  {
    this.traceGroup = group;
  }//end setTraceGroup
  //----------------------------------------------------------------------------

  public TraceGroup getTraceGroup()
  {
    return this.traceGroup;
  }//end getTraceGroup
  //----------------------------------------------------------------------------
}//end class TraceViewer

//**********************************************************************

Class to group the parts of the trace file (grouped by particular criterias in the future):

package advantage.utils;

/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceGroup
{
  private int id = -1;
  private StringBuffer traces = null;
  //----------------------------------------------------------------------------

  public TraceGroup()
  {
  }//end default constructor
  //----------------------------------------------------------------------------

  public TraceGroup(int id)
  {
    this.id = id;
  }//end constructor
  //----------------------------------------------------------------------------

  public TraceGroup(int id, StringBuffer text)
  {
    this.id = id;
    this.traces = text;
  }//end constructor
  //----------------------------------------------------------------------------

  public void setId(int id)
  {
    this.id = id;
  }//end setId
  //----------------------------------------------------------------------------

  public int getId()
  {
    return this.id;
  }//end getId
  //----------------------------------------------------------------------------

  public void setTraces(StringBuffer text)
  {
    this.traces = text;
  }//end setTraces
  //----------------------------------------------------------------------------

  public StringBuffer getTraces()
  {
    return this.traces;
  }//end getTraces
  //----------------------------------------------------------------------------
}//end class TraceGroup

//**********************************************************************

Necessary property file TraceAnalyzerConf.properties:

width = 800
height = 600

locale_language = en
locale_country = GB
#locale_language = en
#locale_country = US
#locale_language = de
#locale_country = DE

TraceAnalyzerBgColor = #c0c0c0
TraceAnalyzerBgColor_R = 225
TraceAnalyzerBgColor_G = 225
TraceAnalyzerBgColor_B = 225

TraceViewer1Color = #ffffff
TraceViewer2Color = #e1e1e1

Author

Commented:
There is a System.arraycops(...); in method initTraceGroups() but the OutOfMemoryError occurs even without the arrayCopy statement.
BTW: Is there a way to load a large text file into the Java app (JTextPane) and show the data already read and continue loading the remaining data in the background?
Mick BarryJava Developer
CERTIFIED EXPERT
Top Expert 2010

Commented:
the setPage() method loads the page asynchronously i think.
Unlock the solution to this question.
Thanks for using Experts Exchange.

Please provide your email to receive a free trial preview!

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.