merge multiple files into one automatically.

Is there any way via powershell, to supply a folder as a parameter, and it 'merge' all files in that folder into a single file? I have a batch of files with extention .log, which when you open one up actually represent comma seperated rows of data (.csv). But rather than manually import and append the data into a master xlsx for example, I was hoping for an automated way of merging all files into one, and then just manually deleting the duplicate 'field headers' that will be brought in from each individual file.
LVL 4
pma111Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

AlanConsultantCommented:
Maybe:

Cat file1.log + file2.log + ... + fileN.log > MasterFile.log

Does that do what you want?

Alan.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Hasin Ahmed ChoudharyExchange AdministratorCommented:
Try this:
Get-ChildItem -recurse -include "*.txt" | % { Get-Content $_ -ReadCount 0 | Add-Content .combined_files.txt }

Once You have Combine_Files, you can search for header line and delete using search-replace option.
0
oBdACommented:
Why would you want to delete the header lines manually?
This will sort the files be LastWriteTime before merging; or are the names already in a format that allows them to be sorted in the correct order?
$sourceFolder = 'C:\Temp\log'
$filter = '*.log'
$outFile = 'C:\Temp\merged.log'
$skip = 0
Get-ChildItem -Path $sourceFolder -Filter $filter |
	Sort-Object -Property $_.LastWriteTime |
	ForEach-Object {
		Get-Content -Path $_.FullName | Select-Object -Skip $skip
		$skip = 1
	} | Set-Content -Path $outFile

Open in new window

1
Challenges in Government Cyber Security

Has cyber security been a challenge in your government organization? Are you looking to improve your government's network security? Learn more about how to improve your government organization's security by viewing our on-demand webinar!

pma111Author Commented:
I presume cat is some form of Linux based command?
0
AlanConsultantCommented:
I'm not at a computer right now, but I'm pretty sure it will work in PowerShell.

In fact, probably:

cat *.log > MasterLogFile.log

will probably work too.

Alan.
0
Shaun VermaakTechnical Specialist/DeveloperCommented:
I think cat is just an alias for type in Powershell so this
cat *.log > MasterLogFile.log

Open in new window

should be the same as
type *.log > MasterLogFile.log

Open in new window


You also should be able to do this (copy is alias of copy-item)
copy *.log MasterLogFile.log

Open in new window

or
copy-item *.log MasterLogFile.log

Open in new window

0
Brent ChallisPrincipal: ITCommented:
Is there a header row in each file?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.