Solved

Splitting a file into batches

Posted on 2012-03-15
6
271 Views
Last Modified: 2012-03-16
Hi:
I have a text file, extracted from a database. It is a set of instructions for a saw.
Note that column eight, beginning 169... changes from 169022 to 169023, etc.
Is there any way of breaking the file at each change of batch; creating a new file, and each file having the name of the  batch it contains. So currently the file is called 169022. I need that to be JUST 169022, with different files for each subsequent batch.
I can from the database create a text file with a list of the batches if needs be.
Java is good - but will accept anything (c++, c#, vb, etc!)

Thanks!
0
Comment
Question by:ClaytonGlass
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 

Author Comment

by:ClaytonGlass
ID: 37724384
Sorry...forgot the file...
0
 

Author Comment

by:ClaytonGlass
ID: 37724396
Sorry...forgot the file...
169022.txt
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 37725647
You might try this class, which contains a rather crude implementation of T for your file (but it seems to work)
import java.io.*;

import java.util.*;


public class ComparableSublistRetriever<T extends Comparable<T>>
    implements Iterator<List<T>> {
    private List<T> delegate;
    private T previous;
    private T current;
    private int index;

    public ComparableSublistRetriever(List<T> items) {
        delegate = new ArrayList<T>(items);
        Collections.sort(delegate);
    }

    @Override
    public boolean hasNext() {
        return index < delegate.size();
    }

    @Override
    public List<T> next() {
        List<T> sublist = new ArrayList<T>();
        T current = delegate.get(index++);
        sublist.add(current);
	previous = current;
	while(index < delegate.size() && (current = delegate.get(index)).equals(previous)) {
            sublist.add(current);
            previous = current;
	    index++;

        }
        return sublist;
    }

    @Override
    public void remove() {
        ;
    }

    static class CsvLine implements Comparable<CsvLine> {
	private String[] array;

	public CsvLine(String[] array) {
	    this.array = array;
	}

	public int compareTo(CsvLine other) {
	    return array[7].compareTo(other.array[7]);
	}

	public boolean equals(Object other) {
	    return array[7].equals(((CsvLine)other).array[7]);
	}

	public String toString() {
	    return Arrays.toString(array);
	}
    }

    public static void main(String[] args) throws Exception {
	List<CsvLine> csvLines = new ArrayList<CsvLine>();
	Scanner s = new Scanner(new File(args[0]));
	while(s.hasNextLine()) {
	    csvLines.add(new CsvLine(s.nextLine().split("\\s+", 9)));
	}
	s.close();
        ComparableSublistRetriever<CsvLine> r = new ComparableSublistRetriever<CsvLine>(csvLines);

        while (r.hasNext()) {
            System.out.println(r.next());
        }
    }
}

Open in new window

0
Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

 
LVL 47

Expert Comment

by:for_yan
ID: 37726247
This is the code for the complete program which does it;
See attached the files produeced from your input file
import java.io.BufferedReader;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.HashMap;

public class SplitFile {
    public SplitFile() {
        try{
            HashMap <String, ArrayList<String>> map = new HashMap<String, ArrayList<String>>();
            ArrayList<String> names = new ArrayList<String>();

            BufferedReader br = new BufferedReader((new FileReader("169022.txt")));

            String buff = "";

            while((buff=br.readLine()) != null){
                String [] items = buff.split("[\\s]+");
                if(items.length < 3)continue;



                 if(map.get(items[7]) != null){
                     ArrayList<String> ar = map.get(items[7]);
                     ar.add(buff);
                       map.put(items[7], ar);

                 }      else
                 {
                     ArrayList<String> ar = new ArrayList<String>();
                     ar.add(buff);
                     System.out.println(items[7]);
                     names.add(items[7]);
                       map.put(items[7], ar);
                 }

                


            }

            br.close();

            for(int j=0; j< names.size(); j++){
                String name = names.get(j);
                PrintStream ps = new PrintStream(new FileOutputStream(name + "_out.txt"));

                ArrayList<String> ar = map.get(name);
                for(String s: ar){
                    ps.println(s);
                }

                 ps.close();
            }






        }  catch(Exception ex){
            ex.printStackTrace();

        }


    }

    public static void main(String[] args) {
        new SplitFile();
        }
}

Open in new window

169022-out.txt
169023-out.txt
169024-out.txt
169025-out.txt
169026-out.txt
169027-out.txt
0
 
LVL 47

Accepted Solution

by:
for_yan earned 300 total points
ID: 37726390
This is even simpler varian which does not depend on the size fo the file and can handle files of any size.

Output files are attached.

import java.io.*;
import java.util.ArrayList;
import java.util.HashMap;

public class SplitFile1 {

       public SplitFile1() {
        try{

            String sep = System.getProperty("line.separator");

            BufferedReader br = new BufferedReader((new FileReader("169022.txt")));
            String curFile = null;
            FileWriter fw = null;

            String buff = "";

            while((buff=br.readLine()) != null){
                String [] items = buff.split("[\\s]+");
                if(items.length < 3)continue;
                String name = items[7];
                if(!name.equals(curFile)){

                    curFile = name;
                    if(fw != null)fw.close();
                    fw = new FileWriter(new File(curFile+ "_out1.txt"), true);
                    

                }
               if(fw != null) fw.write(buff + sep);


            }

            br.close();
            if(fw != null) fw.close();

        } catch(Exception ex){
            ex.printStackTrace();
        }




}

    public static void main(String[] args) {
        new SplitFile1();
    }


}

Open in new window

169022-out1.txt
169023-out1.txt
169024-out1.txt
169025-out1.txt
169026-out1.txt
169027-out1.txt
0
 

Author Closing Comment

by:ClaytonGlass
ID: 37728308
Thank You Very Much!
0

Featured Post

Revamp Your Training Process

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
The article shows the basic steps of integrating an HTML theme template into an ASP.NET MVC project
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question