How to concatenate all files from one folder into one, ignoring headers?

bfuchs
bfuchs used Ask the Experts™
on
Hi Experts,
I have this question after solving the following..
https://www.experts-exchange.com/questions/29108803/Script-to-read-all-files-of-specific-folder.html#acceptAnswerByMember
In that question I got a script how to count all records of files in one folder.
However now I need to know from that count how many are under one specific category.
Therefore I thought the best way would be to concatenate all files into one, just ignoring the headers.
And from there I will link to MS Access and get the results from a query.
Anyone has either a script for that or another way to accomplish the task?
Thanks in advance.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Distinguished Expert 2017

Commented:
Import the files into the same table.
Hi,
They're few thousand files-:)
But I think the following is the solution.
for /R %F in (*.csv) do type "%F" >> output.txt

Open in new window

Thanks,
Ben
Distinguished Expert 2017

Commented:
Why does it matter if there are a few thousand?  Only the total size matters.  Will it exceed 2G?  At least if you import them into a single table, you would be able to log the file source and therefore be able to count the individual rows and do whatever other analysis you need to do.
Acronis in Gartner 2019 MQ for datacenter backup

It is an honor to be featured in Gartner 2019 Magic Quadrant for Datacenter Backup and Recovery Solutions. Gartner’s MQ sets a high standard and earning a place on their grid is a great affirmation that Acronis is delivering on our mission to protect all data, apps, and systems.

@Pat,

My question was how to import the contents of thousand files into one table.

Thanks,
Ben
Top Expert 2016

Commented:
dir /b *.csv > output.txt

Open in new window


the above command would produce the same output as your for command granted the files are in the current directory.

note, if using the for /r command you have to use %%F within a batch file. a single %F is only valid in the console window.

if you would use

dir /s *.csv /b

Open in new window


the output contains the full paths and would search in sub folders as well.

My question was how to import the contents of thousand files into one table.
not quite clear how a list of all csv files would help to achieve that much bigger goal?

Sara
@sarabande,

not quite clear how a list of all csv files would help to achieve that much bigger goal?
I had many files containing records of same table all in one folder.
When using the code I posted, I can get all files concatenated into one single file, and then when used Access to link to that file I was able to filter out the records containing just headers and the rest where the record count.
Try it-:)

Yours only outputs file names which does not really help here.

Thanks,
Ben
Resolved.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial