asked on

How to concatenate all files from one folder into one, ignoring headers?

Hi Experts,
I have this question after solving the following..
https://www.experts-exchange.com/questions/29108803/Script-to-read-all-files-of-specific-folder.html?anchorAnswerId=cceptAnswerByMember#acceptAnswerByMember
In that question I got a script how to count all records of files in one folder.
However now I need to know from that count how many are under one specific category.
Therefore I thought the best way would be to concatenate all files into one, just ignoring the headers.
And from there I will link to MS Access and get the results from a query.
Anyone has either a script for that or another way to accomplish the task?
Thanks in advance.

PatHartman

Import the files into the same table.

ASKER CERTIFIED SOLUTION

bfuchs

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

PatHartman

Why does it matter if there are a few thousand? Only the total size matters. Will it exceed 2G? At least if you import them into a single table, you would be able to log the file source and therefore be able to count the individual rows and do whatever other analysis you need to do.

bfuchs

ASKER

@Pat,

My question was how to import the contents of thousand files into one table.

Thanks,
Ben

sarabande

dir /b *.csv > output.txt

Open in new window

the above command would produce the same output as your for command granted the files are in the current directory.

note, if using the for /r command you have to use %%F within a batch file. a single %F is only valid in the console window.

if you would use

dir /s *.csv /b

Open in new window

the output contains the full paths and would search in sub folders as well.

My question was how to import the contents of thousand files into one table.

not quite clear how a list of all csv files would help to achieve that much bigger goal?

Sara

bfuchs

ASKER

@sarabande,

not quite clear how a list of all csv files would help to achieve that much bigger goal?

I had many files containing records of same table all in one folder.
When using the code I posted, I can get all files concatenated into one single file, and then when used Access to link to that file I was able to filter out the records containing just headers and the rest where the record count.
Try it-:)

Yours only outputs file names which does not really help here.

Thanks,
Ben

bfuchs

ASKER

Resolved.