Solved

Handling large text files with JTextPane

Posted on 2004-04-07
13
1,007 Views
Last Modified: 2007-12-19
Is there a restriction concerning the size of (text) files when trying to display them in a JTextPane?

I have trace files which can have a size of 10MB, 20MB, or even 100MB and when I try to display the files content within a JTextPane I get an OutOfMemoryError. How can I handle this best?

By now I just read the file's content with a BufferedReader into a StringBuffer to display the StringBuffer's content within a JTextPane. On later stages I want to group the source file's content and display these groups in different JTextPanes.

The error occurs when executing this statement:

this.replaceSelection(sb + "\n");

(Where 'sb' is the StringBuffer instance obviuosly).

How can I avoid getting this OutOfMemoryError?
0
Comment
Question by:dirku
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 5
  • 2
13 Comments
 

Expert Comment

by:RobCSP
ID: 10775405
In my opinion you can  split the text in files of 1Mb size

  File df = new File(getDirName() + java.io.File.separator + getFileName());
        if (df.length()/1024>sizeKb)
             df.renameTo(new File("nameOftheLogfile"));


You can control the name with a counter or something like that.......
0
 

Expert Comment

by:RobCSP
ID: 10775715
I´have been testing in my computer and when a file exceed 5Mb the OutofMemory error occurs.

Your application must control the size of the files that will be opened to avoid this error.


SizeKb=1024;
getFileName() --> get the name of the file
getDirName()  --> get the name of the dir

nameOftheLogfile--> name of the new log file
i.e.
logfile1
logfile2
logfile3, etc...

Hope it helps ;-P
0
 
LVL 92

Accepted Solution

by:
objects earned 125 total points
ID: 10778857
> Is there a restriction concerning the size of (text) files when trying to display them in a JTextPane?

As you have discivered the limitation is effectively memory.

you can use the -Xm command line option to increase the amount of memory available.
eg. to make 100Mb avbailable use:
-Xmx100m
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 92

Expert Comment

by:objects
ID: 10781582
0
 

Author Comment

by:dirku
ID: 10781624
Oops. My acceptance maybe was too quick?
I shortened the trace file for testing purposes and then tried your suggestion.
Now, after using the original trace file with ~16MB I get the OutOfMemoryError again although I passed -Xmx60MB as a command line option.

:-(
0
 

Author Comment

by:dirku
ID: 10781627
Even with -Xmx100m...
0
 
LVL 92

Expert Comment

by:objects
ID: 10781655
Can u produce a small example that reproduces the problem that i could try here?
0
 

Author Comment

by:dirku
ID: 10781682
How could I do that?
I could zip my four small classes and the 16MB trace file and send it to you via email if you want.
0
 

Author Comment

by:dirku
ID: 10781696
BTW:
I am currently running the app from within JBuilder 9 but since I pass the command line parameter this should not be the problem.
0
 
LVL 92

Expert Comment

by:objects
ID: 10781735
just post the code, without the 16mb trace
0
 

Author Comment

by:dirku
ID: 10781757
Main class:

package advantage.utils;

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Dimension;

import java.io.File;

import javax.swing.Box;
import javax.swing.BoxLayout;
import javax.swing.JFileChooser;
import javax.swing.JFrame;
import javax.swing.JButton;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;

import javax.swing.text.BadLocationException;

//import java.awt.HeadlessException;
import java.util.ResourceBundle;
import java.util.Locale;

/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p><br>This TraceAnalyzer uses a {@link TraceFileParser} to group the traces
 * within a trace file. These grouped traces will then be displayed with an
 * alternating background color to identify the {@link TraceGroup}s easier.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceAnalyzer extends JFrame
{
  private transient ResourceBundle rbundle = null;
  private transient TraceFileParser tfParser = null;
  private transient TraceGroup[] traceGroups = null;
  private Color bgColor = Color.gray;
  private Color box1Color = Color.white;
  private Color box2Color = Color.gray.brighter();

  private JPanel mainPane = new JPanel();
  private JButton testButton = null;
  private JFileChooser fileChooser = null;
  //----------------------------------------------------------------------------

  /**
   * The <code>TraceAnalyzer</code> default constructor initializes the
   * application and make it visible.
   */
  public TraceAnalyzer() //throws HeadlessException
  {
    init();
    this.setVisible(true);
  }//end default constructor
  //----------------------------------------------------------------------------

  /**
   * Initializes the <code>TraceAnalyzer</code>.
   */
  private void init()
  {
    //Handle the window closing event.
    addWindowListener(new java.awt.event.WindowAdapter()
    {
      public void windowClosing(java.awt.event.WindowEvent evt)
      {
        exitAnalyzer(evt);
      }
    });

    initConfiguration();
    initGuiComponents();
  }//end init
  //----------------------------------------------------------------------------

  /**
   * Initializes this <code>TraceAnalyzer</code>'s configuration using underlying
   * property files/ResourceBundles.
   */
  private void initConfiguration()
  {
   try
   {
     rbundle = ResourceBundle.getBundle("traceAnalyzerConf");
     this.bgColor = Color.decode(rbundle.getString("TraceAnalyzerBgColor"));
     this.box1Color = Color.decode(rbundle.getString("TraceViewer1Color"));
     this.box2Color = Color.decode(rbundle.getString("TraceViewer2Color"));
     String locale_lang = rbundle.getString("locale_language");
     String locale_country = rbundle.getString("locale_country");
     Locale locale = new Locale(locale_lang, locale_country);
     rbundle = ResourceBundle.getBundle("traceAnalyzer", locale);
   }
   catch(Exception e)
   {
      e.printStackTrace();
   }
    int width = Integer.parseInt(rbundle.getString("width"));
    int height = Integer.parseInt(rbundle.getString("height"));

    this.setSize(width, height);
  }//end initConfiguration
  //----------------------------------------------------------------------------

  /**
   * Initializes the GUI components of the <code>TraceAnalyzer</code>.
   */
  private void initGuiComponents()
  {
    //The layout of the application itself.
    this.getContentPane().setLayout(new BorderLayout());

    testButton = new JButton("test");
    this.getContentPane().add(testButton, BorderLayout.PAGE_START);

    //BoxLayout bl = new BoxLayout(mainPane, BoxLayout.PAGE_AXIS);
    //mainPane.setLayout(bl);
    mainPane.setBackground(Color.orange);

    JScrollPane scrollPane = new JScrollPane(mainPane);

    this.getContentPane().add(scrollPane, BorderLayout.CENTER);

    addListeners();
  }//end initGuiComponents
  //----------------------------------------------------------------------------

  private void initTraceGroups()
  {
    //traceGroups = new TraceGroup[1];
    //traceGroups[0] = tfParser.parseFileByString("");
//traceGroups = tfParser.parseFileByString("");
TraceGroup[] trGrps = tfParser.parseFileByString("");
//System.out.println("traceGroups.length = " + traceGroups.length);
System.out.println("trGrps.length = " + trGrps.length);

traceGroups = new TraceGroup[trGrps.length+1];
    System.out.println("traceGroups.length = " + traceGroups.length);
System.arraycopy(trGrps, 0, traceGroups, 0, trGrps.length);
traceGroups[traceGroups.length-1] = new TraceGroup(0, new StringBuffer("qqqqqqqq\n"));

/*
    traceGroups = new TraceGroup[] {
            new TraceGroup(0, new StringBuffer("qqqqqqqq")),
            new TraceGroup(1, new StringBuffer("rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr")),
            new TraceGroup(2, new StringBuffer("sssssssssssss\ntttttttttttttttt\n\nsfghdfsaba\nsadfgafega\n\nsdfbgadfg\ndasfgadfgd\nasdffsadsadfgsdfg\nadfvbaddbsdbsadbsdfbsbnkdfckmgadfbknbjadnbjadnfbjnsadjfbnandfbalsndfbjsandfbjnlsa\n\nkjnfvkjanfkjadnfbvkjadnkja\nskdfnkjsfb\nakvnkjfgnkjqaw\nadnfkj\n"))
        };
*/
  }//end initTraceGroups
  //----------------------------------------------------------------------------

  /**
   * Registers listeners to particular components to handle events.
   */
  private void addListeners()
  {
    testButton.addActionListener(new ActionListener()
    {
      public void actionPerformed(ActionEvent e)
      {
        fileChooser = new JFileChooser();
        if (e.getSource() == testButton) {
          int returnVal = fileChooser.showOpenDialog(TraceAnalyzer.this);
          if (returnVal == JFileChooser.APPROVE_OPTION) {
            File file = fileChooser.getSelectedFile();
            tfParser = new TraceFileParser(file);
            initTraceGroups();
            runTraceViewer();
          } //end if(returnVal == JFileChooser.APPROVE_OPTION)
        } //end if(e.getSource() == testButton)
      } //end actionPerformed
    });
  } //end addListeners
  //----------------------------------------------------------------------------

  /**
   *
   */
  private void runTraceViewer()
  {
    System.out.println("traceGroups.lenght = " + traceGroups.length);
      if (traceGroups != null)
      {
        for (int i = 0; i < traceGroups.length; i++)
        {
          Box box = Box.createVerticalBox();
          box.setAlignmentY(BoxLayout.PAGE_AXIS);
          box.setBackground(Color.darkGray);

          TraceGroup traceGroup = traceGroups[i];
          TraceViewer tv = new TraceViewer(traceGroup);
System.out.println("tv" + i + " created.");
          System.out.println(tv.getTraceGroup().getTraces().toString());
          if (i % 2 == 0)
          {
            tv.setBackground(this.box1Color);
          }
          else
          {
            tv.setBackground(this.box2Color);
          }

          box.add(tv);
          mainPane.add(box);
          mainPane.add(Box.createVerticalStrut(5));

          mainPane.revalidate();
          mainPane.repaint();
        } //end for i
System.out.println("Each of " + traceGroups.length + " Boxes created.");
      } //end if(traceGroups != null)
  }//end runTraceViewer
  //----------------------------------------------------------------------------

  /**
   * Returns the {@link TraceGroup}s which are in fact the instantiated
   * {@link TraceViewer}s.
   * @return The {@link TraceGroup}s handled by this <code>TraceAnalyzer</code>.
   */
  public TraceGroup[] getTraceGroups()
  {
    return this.traceGroups;
  }//end getTraceGroups
  //----------------------------------------------------------------------------

  private void exitAnalyzer(java.awt.event.WindowEvent evt)
  {
    System.gc();
    System.exit(0);
  }//end exitAnalyzer
  //----------------------------------------------------------------------------

  public static void main(String[] args) //throws HeadlessException
  {
    TraceAnalyzer tracy = new TraceAnalyzer();
  }//end main
}//end class TraceAnalyzer

//*********************************************************************

Class intended to parse the trace file in the future:

package advantage.utils;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.StringTokenizer;


/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p><br>This <code>TraceFileParser</code> takes a trace file which is conform
 * to the Siemens AG conventions.</p>
 * A trace con consist of the following parts although it is not mandatory that
 * each of these components are in a particular trace:</p>
 * <ul>
 *   <li>Timestamp (date and time with milliseconds)</li>
 *   <li>Process ID</li>
 *   <li>Thread ID</li>
 *   <li>Trace topic</li>
 *   <li>Trace level</li>
 *   <li>CFRAME tag</li>
 *   <li>container_name::CAT type name</li>
 *   <li>container_name::CAT instance ID</li>
 *   <li>CAT type</li>
 *   <li>Component ID</li>
 *   <li>Context ID</li>
 *   <li>Context type ID</li>
 * </ul>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceFileParser
{
  private File tFile = null;
  private TraceGroup[] traceGroups = null;
  //----------------------------------------------------------------------------

  public TraceFileParser()
  {
  }//end default constructor
  //----------------------------------------------------------------------------

  public TraceFileParser(File file)
  {
    tFile = file;
  }//end standard constructor
  //----------------------------------------------------------------------------

  /**
   * Parses a file to find matches for the given pattern.
   * @param pattern The pattern to look for within the file to parse.
   */
  public TraceGroup[] parseFileByString(String pattern)
  {
    TraceGroup tGroup = new TraceGroup();
    traceGroups = new TraceGroup[1];
    StringBuffer sb = new StringBuffer();
    FileReader fr = null;
    BufferedReader bufr = null;
    try
    {
      fr = new FileReader(tFile);
      bufr = new BufferedReader(fr);
      String line = "";
      while ((line = bufr.readLine()) != null)
      {
        sb.append(line);
        sb.append("\n");
      } //end
      tGroup.setId(1);
      tGroup.setTraces(sb);
System.out.println(sb.length());
System.out.println(tFile.length());
    }
    catch(FileNotFoundException fnfE)
    {

    }
    catch(IOException ioE)
    {

    }
    finally
    {
      try
      {
        bufr.close();
        fr.close();
      }
      catch(IOException innerIoE)
      {

      }
    }
traceGroups[0] = tGroup;
    //return tGroup;
    return traceGroups;
  }//end parseFileByString
  //----------------------------------------------------------------------------
}//end class TraceFileParser

//**********************************************************************

Class to display the text within a JTextPane:

package advantage.utils;

import java.awt.Color;

import javax.swing.SwingUtilities;
import javax.swing.JTextPane;

import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;
import javax.swing.text.DefaultStyledDocument;
import javax.swing.text.MutableAttributeSet;
import javax.swing.text.SimpleAttributeSet;
import javax.swing.text.Style;
import javax.swing.text.StyleConstants;
import javax.swing.text.StyleContext;
import javax.swing.text.StyledDocument;

/**
 * <p>Title: TraceViewer</p>
 * <p>Description: The viewing component of the {@link TraceAnalyzer}.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceViewer extends JTextPane
{
  private transient DefaultStyledDocument doc = null;
  private transient StyleContext sCtxt = null;
  private TraceGroup traceGroup = null;
  //----------------------------------------------------------------------------

  public TraceViewer(TraceGroup tGroup)
  {
    traceGroup = tGroup;
    sCtxt = new StyleContext();
    doc = new DefaultStyledDocument(sCtxt);
    this.setDocument(doc);

    final StringBuffer sb = traceGroup.getTraces();
this.setBackground(Color.green);
final MutableAttributeSet par_1 = new SimpleAttributeSet();
StyleConstants.setForeground(par_1, Color.black);
StyleConstants.setBackground(par_1, Color.gray);

/*
    final Style pStyle_1 = sContext.addStyle("paragraph_1_bg", null);
    final Style pStyle_2 = sContext.addStyle("paragraph_2_bg", null);
    final Style mainStyle = sContext.addStyle("mainStyle", sContext.getStyle(StyleContext.DEFAULT_STYLE));

    StyleConstants.setLeftIndent(mainStyle, 0);
    StyleConstants.setRightIndent(mainStyle, 80);

    pStyle_1.addAttribute(StyleConstants.FontSize, new Integer(20));
    pStyle_1.addAttribute(StyleConstants.Foreground, Color.blue);
    pStyle_1.addAttribute(StyleConstants.Background, Color.red);
*/
doc.setParagraphAttributes(0, 1, par_1, true);
this.replaceSelection(sb.toString());
//System.out.println(sb.toString());
/*
    try
    {
      SwingUtilities.invokeAndWait(new Runnable() {
        public void run() {
          try {
          }
          catch (BadLocationException ble) {

          }
        }
      });
    }
    catch(Exception e)
    {

    }
*/
    this.setEditable(false);
  }//end default constructor
  //----------------------------------------------------------------------------

  /**
   * Feeds the <code>TraceViewer</code> with the {@link TraceGroup} to be shown.
   * @param group The {@link TraceGroup} the be displayed through this
   * <code>TraceViewer</code>.
   * @author Dirk Ulrich
   */
  public void setTraceGroup(TraceGroup group)
  {
    this.traceGroup = group;
  }//end setTraceGroup
  //----------------------------------------------------------------------------

  public TraceGroup getTraceGroup()
  {
    return this.traceGroup;
  }//end getTraceGroup
  //----------------------------------------------------------------------------
}//end class TraceViewer

//**********************************************************************

Class to group the parts of the trace file (grouped by particular criterias in the future):

package advantage.utils;

/**
 * <p>Title: TraceAnalyzer</p>
 * <p>Description: An analyzing tool for trace files.</p>
 * <p>Copyright: Copyright (c) 2004 by du-it</p>
 * <p>Company: du-it Dirk Ulrich-Informationstechnologie</p>
 * @author Dirk Ulrich d.ulrich@du-it.de
 * @version 1.0
 */

public class TraceGroup
{
  private int id = -1;
  private StringBuffer traces = null;
  //----------------------------------------------------------------------------

  public TraceGroup()
  {
  }//end default constructor
  //----------------------------------------------------------------------------

  public TraceGroup(int id)
  {
    this.id = id;
  }//end constructor
  //----------------------------------------------------------------------------

  public TraceGroup(int id, StringBuffer text)
  {
    this.id = id;
    this.traces = text;
  }//end constructor
  //----------------------------------------------------------------------------

  public void setId(int id)
  {
    this.id = id;
  }//end setId
  //----------------------------------------------------------------------------

  public int getId()
  {
    return this.id;
  }//end getId
  //----------------------------------------------------------------------------

  public void setTraces(StringBuffer text)
  {
    this.traces = text;
  }//end setTraces
  //----------------------------------------------------------------------------

  public StringBuffer getTraces()
  {
    return this.traces;
  }//end getTraces
  //----------------------------------------------------------------------------
}//end class TraceGroup

//**********************************************************************

Necessary property file TraceAnalyzerConf.properties:

width = 800
height = 600

locale_language = en
locale_country = GB
#locale_language = en
#locale_country = US
#locale_language = de
#locale_country = DE

TraceAnalyzerBgColor = #c0c0c0
TraceAnalyzerBgColor_R = 225
TraceAnalyzerBgColor_G = 225
TraceAnalyzerBgColor_B = 225

TraceViewer1Color = #ffffff
TraceViewer2Color = #e1e1e1
0
 

Author Comment

by:dirku
ID: 10781781
There is a System.arraycops(...); in method initTraceGroups() but the OutOfMemoryError occurs even without the arrayCopy statement.
BTW: Is there a way to load a large text file into the Java app (JTextPane) and show the data already read and continue loading the remaining data in the background?
0
 
LVL 92

Expert Comment

by:objects
ID: 10788368
the setPage() method loads the page asynchronously i think.
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
In this post we will learn different types of Android Layout and some basics of an Android App.
Viewers learn how to read error messages and identify possible mistakes that could cause hours of frustration. Coding is as much about debugging your code as it is about writing it. Define Error Message: Line Numbers: Type of Error: Break Down…
The viewer will learn how to implement Singleton Design Pattern in Java.

732 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question