Solved

Splitting a file into batches

Posted on 2012-03-15
6
268 Views
Last Modified: 2012-03-16
Hi:
I have a text file, extracted from a database. It is a set of instructions for a saw.
Note that column eight, beginning 169... changes from 169022 to 169023, etc.
Is there any way of breaking the file at each change of batch; creating a new file, and each file having the name of the  batch it contains. So currently the file is called 169022. I need that to be JUST 169022, with different files for each subsequent batch.
I can from the database create a text file with a list of the batches if needs be.
Java is good - but will accept anything (c++, c#, vb, etc!)

Thanks!
0
Comment
Question by:ClaytonGlass
  • 3
  • 2
6 Comments
 

Author Comment

by:ClaytonGlass
ID: 37724384
Sorry...forgot the file...
0
 

Author Comment

by:ClaytonGlass
ID: 37724396
Sorry...forgot the file...
169022.txt
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 37725647
You might try this class, which contains a rather crude implementation of T for your file (but it seems to work)
import java.io.*;

import java.util.*;


public class ComparableSublistRetriever<T extends Comparable<T>>
    implements Iterator<List<T>> {
    private List<T> delegate;
    private T previous;
    private T current;
    private int index;

    public ComparableSublistRetriever(List<T> items) {
        delegate = new ArrayList<T>(items);
        Collections.sort(delegate);
    }

    @Override
    public boolean hasNext() {
        return index < delegate.size();
    }

    @Override
    public List<T> next() {
        List<T> sublist = new ArrayList<T>();
        T current = delegate.get(index++);
        sublist.add(current);
	previous = current;
	while(index < delegate.size() && (current = delegate.get(index)).equals(previous)) {
            sublist.add(current);
            previous = current;
	    index++;

        }
        return sublist;
    }

    @Override
    public void remove() {
        ;
    }

    static class CsvLine implements Comparable<CsvLine> {
	private String[] array;

	public CsvLine(String[] array) {
	    this.array = array;
	}

	public int compareTo(CsvLine other) {
	    return array[7].compareTo(other.array[7]);
	}

	public boolean equals(Object other) {
	    return array[7].equals(((CsvLine)other).array[7]);
	}

	public String toString() {
	    return Arrays.toString(array);
	}
    }

    public static void main(String[] args) throws Exception {
	List<CsvLine> csvLines = new ArrayList<CsvLine>();
	Scanner s = new Scanner(new File(args[0]));
	while(s.hasNextLine()) {
	    csvLines.add(new CsvLine(s.nextLine().split("\\s+", 9)));
	}
	s.close();
        ComparableSublistRetriever<CsvLine> r = new ComparableSublistRetriever<CsvLine>(csvLines);

        while (r.hasNext()) {
            System.out.println(r.next());
        }
    }
}

Open in new window

0
Courses: Start Training Online With Pros, Today

Brush up on the basics or master the advanced techniques required to earn essential industry certifications, with Courses. Enroll in a course and start learning today. Training topics range from Android App Dev to the Xen Virtualization Platform.

 
LVL 47

Expert Comment

by:for_yan
ID: 37726247
This is the code for the complete program which does it;
See attached the files produeced from your input file
import java.io.BufferedReader;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.HashMap;

public class SplitFile {
    public SplitFile() {
        try{
            HashMap <String, ArrayList<String>> map = new HashMap<String, ArrayList<String>>();
            ArrayList<String> names = new ArrayList<String>();

            BufferedReader br = new BufferedReader((new FileReader("169022.txt")));

            String buff = "";

            while((buff=br.readLine()) != null){
                String [] items = buff.split("[\\s]+");
                if(items.length < 3)continue;



                 if(map.get(items[7]) != null){
                     ArrayList<String> ar = map.get(items[7]);
                     ar.add(buff);
                       map.put(items[7], ar);

                 }      else
                 {
                     ArrayList<String> ar = new ArrayList<String>();
                     ar.add(buff);
                     System.out.println(items[7]);
                     names.add(items[7]);
                       map.put(items[7], ar);
                 }

                


            }

            br.close();

            for(int j=0; j< names.size(); j++){
                String name = names.get(j);
                PrintStream ps = new PrintStream(new FileOutputStream(name + "_out.txt"));

                ArrayList<String> ar = map.get(name);
                for(String s: ar){
                    ps.println(s);
                }

                 ps.close();
            }






        }  catch(Exception ex){
            ex.printStackTrace();

        }


    }

    public static void main(String[] args) {
        new SplitFile();
        }
}

Open in new window

169022-out.txt
169023-out.txt
169024-out.txt
169025-out.txt
169026-out.txt
169027-out.txt
0
 
LVL 47

Accepted Solution

by:
for_yan earned 300 total points
ID: 37726390
This is even simpler varian which does not depend on the size fo the file and can handle files of any size.

Output files are attached.

import java.io.*;
import java.util.ArrayList;
import java.util.HashMap;

public class SplitFile1 {

       public SplitFile1() {
        try{

            String sep = System.getProperty("line.separator");

            BufferedReader br = new BufferedReader((new FileReader("169022.txt")));
            String curFile = null;
            FileWriter fw = null;

            String buff = "";

            while((buff=br.readLine()) != null){
                String [] items = buff.split("[\\s]+");
                if(items.length < 3)continue;
                String name = items[7];
                if(!name.equals(curFile)){

                    curFile = name;
                    if(fw != null)fw.close();
                    fw = new FileWriter(new File(curFile+ "_out1.txt"), true);
                    

                }
               if(fw != null) fw.write(buff + sep);


            }

            br.close();
            if(fw != null) fw.close();

        } catch(Exception ex){
            ex.printStackTrace();
        }




}

    public static void main(String[] args) {
        new SplitFile1();
    }


}

Open in new window

169022-out1.txt
169023-out1.txt
169024-out1.txt
169025-out1.txt
169026-out1.txt
169027-out1.txt
0
 

Author Closing Comment

by:ClaytonGlass
ID: 37728308
Thank You Very Much!
0

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
HashTable highest marks enumeration alternative 9 37
2 questions 10 25
pagenation logic how it is working in my code 1 27
How come this regular expression allows "e"? 14 35
This article shows you how to optimize memory allocations in C++ using placement new. Applicable especially to usecases dealing with creation of large number of objects. A brief on problem: Lets take example problem for simplicity: - I have a G…
Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
This video teaches viewers about errors in exception handling.
The viewer will learn how to pass data into a function in C++. This is one step further in using functions. Instead of only printing text onto the console, the function will be able to perform calculations with argumentents given by the user.

776 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question