Handling large text files with JTextPane

In my opinion you can  split the text in files of 1Mb size

  File df = new File(getDirName() + java.io.File.separator + getFileName());
        if (df.length()/1024>sizeKb)
             df.renameTo(new File("nameOftheLogfile"));


You can control the name with a counter or something like that.......

I´have been testing in my computer and when a file exceed 5Mb the OutofMemory error occurs.

Your application must control the size of the files that will be opened to avoid this error.


SizeKb=1024;
getFileName() --> get the name of the file
getDirName()  --> get the name of the dir

nameOftheLogfile--> name of the new log file
i.e.
logfile1
logfile2
logfile3, etc...

Hope it helps ;-P

:-)

http://www.objects.com.au/staff/mick

Oops. My acceptance maybe was too quick?
I shortened the trace file for testing purposes and then tried your suggestion.
Now, after using the original trace file with ~16MB I get the OutOfMemoryError again although I passed -Xmx60MB as a command line option.

:-(

Even with -Xmx100m...

Can u produce a small example that reproduces the problem that i could try here?

How could I do that?
I could zip my four small classes and the 16MB trace file and send it to you via email if you want.

BTW:
I am currently running the app from within JBuilder 9 but since I pass the command line parameter this should not be the problem.

just post the code, without the 16mb trace

Main class:

package advantage.utils;

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Dimension;

import java.io.File;

import javax.swing.Box;
import javax.swing.BoxLayout;
import javax.swing.JFileChooser;
import javax.swing.JFrame;
import javax.swing.JButton;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;

import javax.swing.text.BadLocationException;

//import java.awt.HeadlessException;
import java.util.ResourceBundle;
import java.util.Locale;

/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p><br>This TraceAnalyzer uses a {@link TraceFileParser} to group the traces
 * within a trace file. These grouped traces will then be displayed with an
 * alternating background color to identify the {@link TraceGroup}s easier.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceAnalyzer extends JFrame
{
  private transient ResourceBundle rbundle = null;
  private transient TraceFileParser tfParser = null;
  private transient TraceGroup[] traceGroups = null;
  private Color bgColor = Color.gray;
  private Color box1Color = Color.white;
  private Color box2Color = Color.gray.brighter();

  private JPanel mainPane = new JPanel();
  private JButton testButton = null;
  private JFileChooser fileChooser = null;
  //----------------------------------------------------------------------------

  /**
   * The <code>TraceAnalyzer</code> default constructor initializes the
   * application and make it visible.
   */
  public TraceAnalyzer() //throws HeadlessException
  {
    init();
    this.setVisible(true);
  }//end default constructor
  //----------------------------------------------------------------------------

  /**
   * Initializes the <code>TraceAnalyzer</code>.
   */
  private void init()
  {
    //Handle the window closing event.
    addWindowListener(new java.awt.event.WindowAdapter()
    {
      public void windowClosing(java.awt.event.WindowEvent evt)
      {
        exitAnalyzer(evt);
      }
    });

    initConfiguration();
    initGuiComponents();
  }//end init
  //----------------------------------------------------------------------------

  /**
   * Initializes this <code>TraceAnalyzer</code>'s configuration using underlying
   * property files/ResourceBundles.
   */
  private void initConfiguration()
  {
   try
   {
     rbundle = ResourceBundle.getBundle("traceAnalyzerConf");
     this.bgColor = Color.decode(rbundle.getString("TraceAnalyzerBgColor"));
     this.box1Color = Color.decode(rbundle.getString("TraceViewer1Color"));
     this.box2Color = Color.decode(rbundle.getString("TraceViewer2Color"));
     String locale_lang = rbundle.getString("locale_language");
     String locale_country = rbundle.getString("locale_country");
     Locale locale = new Locale(locale_lang, locale_country);
     rbundle = ResourceBundle.getBundle("traceAnalyzer", locale);
   }
   catch(Exception e)
   {
      e.printStackTrace();
   }
    int width = Integer.parseInt(rbundle.getString("width"));
    int height = Integer.parseInt(rbundle.getString("height"));

    this.setSize(width, height);
  }//end initConfiguration
  //----------------------------------------------------------------------------

  /**
   * Initializes the GUI components of the <code>TraceAnalyzer</code>.
   */
  private void initGuiComponents()
  {
    //The layout of the application itself.
    this.getContentPane().setLayout(new BorderLayout());

    testButton = new JButton("test");
    this.getContentPane().add(testButton, BorderLayout.PAGE_START);

    //BoxLayout bl = new BoxLayout(mainPane, BoxLayout.PAGE_AXIS);
    //mainPane.setLayout(bl);
    mainPane.setBackground(Color.orange);

    JScrollPane scrollPane = new JScrollPane(mainPane);

    this.getContentPane().add(scrollPane, BorderLayout.CENTER);

    addListeners();
  }//end initGuiComponents
  //----------------------------------------------------------------------------

  private void initTraceGroups()
  {
    //traceGroups = new TraceGroup[1];
    //traceGroups[0] = tfParser.parseFileByString("");
//traceGroups = tfParser.parseFileByString("");
TraceGroup[] trGrps = tfParser.parseFileByString("");
//System.out.println("traceGroups.length = " + traceGroups.length);
System.out.println("trGrps.length = " + trGrps.length);

traceGroups = new TraceGroup[trGrps.length+1];
    System.out.println("traceGroups.length = " + traceGroups.length);
System.arraycopy(trGrps, 0, traceGroups, 0, trGrps.length);
traceGroups[traceGroups.length-1] = new TraceGroup(0, new StringBuffer("qqqqqqqq\n"));

/*
    traceGroups = new TraceGroup[] {
            new TraceGroup(0, new StringBuffer("qqqqqqqq")),
            new TraceGroup(1, new StringBuffer("rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr")),
            new TraceGroup(2, new StringBuffer("sssssssssssss\ntttttttttttttttt\n\nsfghdfsaba\nsadfgafega\n\nsdfbgadfg\ndasfgadfgd\nasdffsadsadfgsdfg\nadfvbaddbsdbsadbsdfbsbnkdfckmgadfbknbjadnbjadnfbjnsadjfbnandfbalsndfbjsandfbjnlsa\n\nkjnfvkjanfkjadnfbvkjadnkja\nskdfnkjsfb\nakvnkjfgnkjqaw\nadnfkj\n"))
        };
*/
  }//end initTraceGroups
  //----------------------------------------------------------------------------

  /**
   * Registers listeners to particular components to handle events.
   */
  private void addListeners()
  {
    testButton.addActionListener(new ActionListener()
    {
      public void actionPerformed(ActionEvent e)
      {
        fileChooser = new JFileChooser();
        if (e.getSource() == testButton) {
          int returnVal = fileChooser.showOpenDialog(TraceAnalyzer.this);
          if (returnVal == JFileChooser.APPROVE_OPTION) {
            File file = fileChooser.getSelectedFile();
            tfParser = new TraceFileParser(file);
            initTraceGroups();
            runTraceViewer();
          } //end if(returnVal == JFileChooser.APPROVE_OPTION)
        } //end if(e.getSource() == testButton)
      } //end actionPerformed
    });
  } //end addListeners
  //----------------------------------------------------------------------------

  /**
   *
   */
  private void runTraceViewer()
  {
    System.out.println("traceGroups.lenght = " + traceGroups.length);
      if (traceGroups != null)
      {
        for (int i = 0; i < traceGroups.length; i++)
        {
          Box box = Box.createVerticalBox();
          box.setAlignmentY(BoxLayout.PAGE_AXIS);
          box.setBackground(Color.darkGray);

          TraceGroup traceGroup = traceGroups[i];
          TraceViewer tv = new TraceViewer(traceGroup);
System.out.println("tv" + i + " created.");
          System.out.println(tv.getTraceGroup().getTraces().toString());
          if (i % 2 == 0)
          {
            tv.setBackground(this.box1Color);
          }
          else
          {
            tv.setBackground(this.box2Color);
          }

          box.add(tv);
          mainPane.add(box);
          mainPane.add(Box.createVerticalStrut(5));

          mainPane.revalidate();
          mainPane.repaint();
        } //end for i
System.out.println("Each of " + traceGroups.length + " Boxes created.");
      } //end if(traceGroups != null)
  }//end runTraceViewer
  //----------------------------------------------------------------------------

  /**
   * Returns the {@link TraceGroup}s which are in fact the instantiated
   * {@link TraceViewer}s.
   * @return The {@link TraceGroup}s handled by this <code>TraceAnalyzer</code>.
   */
  public TraceGroup[] getTraceGroups()
  {
    return this.traceGroups;
  }//end getTraceGroups
  //----------------------------------------------------------------------------

  private void exitAnalyzer(java.awt.event.WindowEvent evt)
  {
    System.gc();
    System.exit(0);
  }//end exitAnalyzer
  //----------------------------------------------------------------------------

  public static void main(String[] args) //throws HeadlessException
  {
    TraceAnalyzer tracy = new TraceAnalyzer();
  }//end main
}//end class TraceAnalyzer

//*********************************************************************

Class intended to parse the trace file in the future:

package advantage.utils;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.StringTokenizer;


/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p><br>This <code>TraceFileParser</code> takes a trace file which is conform
 * to the Siemens AG conventions.</p>
 * A trace con consist of the following parts although it is not mandatory that
 * each of these components are in a particular trace:</p>
 * <ul>
 *   <li>Timestamp (date and time with milliseconds)</li>
 *   <li>Process ID</li>
 *   <li>Thread ID</li>
 *   <li>Trace topic</li>
 *   <li>Trace level</li>
 *   <li>CFRAME tag</li>
 *   <li>container_name::CAT type name</li>
 *   <li>container_name::CAT instance ID</li>
 *   <li>CAT type</li>
 *   <li>Component ID</li>
 *   <li>Context ID</li>
 *   <li>Context type ID</li>
 * </ul>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceFileParser
{
  private File tFile = null;
  private TraceGroup[] traceGroups = null;
  //----------------------------------------------------------------------------

  public TraceFileParser()
  {
  }//end default constructor
  //----------------------------------------------------------------------------

  public TraceFileParser(File file)
  {
    tFile = file;
  }//end standard constructor
  //----------------------------------------------------------------------------

  /**
   * Parses a file to find matches for the given pattern.
   * @param pattern The pattern to look for within the file to parse.
   */
  public TraceGroup[] parseFileByString(String pattern)
  {
    TraceGroup tGroup = new TraceGroup();
    traceGroups = new TraceGroup[1];
    StringBuffer sb = new StringBuffer();
    FileReader fr = null;
    BufferedReader bufr = null;
    try
    {
      fr = new FileReader(tFile);
      bufr = new BufferedReader(fr);
      String line = "";
      while ((line = bufr.readLine()) != null)
      {
        sb.append(line);
        sb.append("\n");
      } //end
      tGroup.setId(1);
      tGroup.setTraces(sb);
System.out.println(sb.length());
System.out.println(tFile.length());
    }
    catch(FileNotFoundException fnfE)
    {

    }
    catch(IOException ioE)
    {

    }
    finally
    {
      try
      {
        bufr.close();
        fr.close();
      }
      catch(IOException innerIoE)
      {

      }
    }
traceGroups[0] = tGroup;
    //return tGroup;
    return traceGroups;
  }//end parseFileByString
  //----------------------------------------------------------------------------
}//end class TraceFileParser

//**********************************************************************

Class to display the text within a JTextPane:

package advantage.utils;

import java.awt.Color;

import javax.swing.SwingUtilities;
import javax.swing.JTextPane;

import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;
import javax.swing.text.DefaultStyledDocument;
import javax.swing.text.MutableAttributeSet;
import javax.swing.text.SimpleAttributeSet;
import javax.swing.text.Style;
import javax.swing.text.StyleConstants;
import javax.swing.text.StyleContext;
import javax.swing.text.StyledDocument;

/**
 * <p>Title: TraceViewer</p>
 * <p>Description: The viewing component of the {@link TraceAnalyzer}.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceViewer extends JTextPane
{
  private transient DefaultStyledDocument doc = null;
  private transient StyleContext sCtxt = null;
  private TraceGroup traceGroup = null;
  //----------------------------------------------------------------------------

  public TraceViewer(TraceGroup tGroup)
  {
    traceGroup = tGroup;
    sCtxt = new StyleContext();
    doc = new DefaultStyledDocument(sCtxt);
    this.setDocument(doc);

    final StringBuffer sb = traceGroup.getTraces();
this.setBackground(Color.green);
final MutableAttributeSet par_1 = new SimpleAttributeSet();
StyleConstants.setForeground(par_1, Color.black);
StyleConstants.setBackground(par_1, Color.gray);

/*
    final Style pStyle_1 = sContext.addStyle("paragraph_1_bg", null);
    final Style pStyle_2 = sContext.addStyle("paragraph_2_bg", null);
    final Style mainStyle = sContext.addStyle("mainStyle", sContext.getStyle(StyleContext.DEFAULT_STYLE));

    StyleConstants.setLeftIndent(mainStyle, 0);
    StyleConstants.setRightIndent(mainStyle, 80);

    pStyle_1.addAttribute(StyleConstants.FontSize, new Integer(20));
    pStyle_1.addAttribute(StyleConstants.Foreground, Color.blue);
    pStyle_1.addAttribute(StyleConstants.Background, Color.red);
*/
doc.setParagraphAttributes(0, 1, par_1, true);
this.replaceSelection(sb.toString());
//System.out.println(sb.toString());
/*
    try
    {
      SwingUtilities.invokeAndWait(new Runnable() {
        public void run() {
          try {
          }
          catch (BadLocationException ble) {

          }
        }
      });
    }
    catch(Exception e)
    {

    }
*/
    this.setEditable(false);
  }//end default constructor
  //----------------------------------------------------------------------------

  /**
   * Feeds the <code>TraceViewer</code> with the {@link TraceGroup} to be shown.
   * @param group The {@link TraceGroup} the be displayed through this
   * <code>TraceViewer</code>.
   * @author Dirk Ulrich
   */
  public void setTraceGroup(TraceGroup group)
  {
    this.traceGroup = group;
  }//end setTraceGroup
  //----------------------------------------------------------------------------

  public TraceGroup getTraceGroup()
  {
    return this.traceGroup;
  }//end getTraceGroup
  //----------------------------------------------------------------------------
}//end class TraceViewer

//**********************************************************************

Class to group the parts of the trace file (grouped by particular criterias in the future):

package advantage.utils;

/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceGroup
{
  private int id = -1;
  private StringBuffer traces = null;
  //----------------------------------------------------------------------------

  public TraceGroup()
  {
  }//end default constructor
  //----------------------------------------------------------------------------

  public TraceGroup(int id)
  {
    this.id = id;
  }//end constructor
  //----------------------------------------------------------------------------

  public TraceGroup(int id, StringBuffer text)
  {
    this.id = id;
    this.traces = text;
  }//end constructor
  //----------------------------------------------------------------------------

  public void setId(int id)
  {
    this.id = id;
  }//end setId
  //----------------------------------------------------------------------------

  public int getId()
  {
    return this.id;
  }//end getId
  //----------------------------------------------------------------------------

  public void setTraces(StringBuffer text)
  {
    this.traces = text;
  }//end setTraces
  //----------------------------------------------------------------------------

  public StringBuffer getTraces()
  {
    return this.traces;
  }//end getTraces
  //----------------------------------------------------------------------------
}//end class TraceGroup

//**********************************************************************

Necessary property file TraceAnalyzerConf.properties:

width = 800
height = 600

locale_language = en
locale_country = GB
#locale_language = en
#locale_country = US
#locale_language = de
#locale_country = DE

TraceAnalyzerBgColor = #c0c0c0
TraceAnalyzerBgColor_R = 225
TraceAnalyzerBgColor_G = 225
TraceAnalyzerBgColor_B = 225

TraceViewer1Color = #ffffff
TraceViewer2Color = #e1e1e1

There is a System.arraycops(...); in method initTraceGroups() but the OutOfMemoryError occurs even without the arrayCopy statement.
BTW: Is there a way to load a large text file into the Java app (JTextPane) and show the data already read and continue loading the remaining data in the background?

the setPage() method loads the page asynchronously i think.