Solved

Powershell Memory Usage / Leakage?

Posted on 2013-12-07
2
334 Views
Last Modified: 2013-12-14
Hi,

I am having an issue with a long running powershell script with a the memory increasing over time, like a leak. I am creating collection variables but after doing a connection to sql. Later on I am writing the output to a CSV file.  However in my functions I am setting the collection variables to $null. But the memory keeps going up and up.

Is there something special I am missing on how powershell frees it memory. So for example if I call function a and I have a variable called $test should the memory be free for what was allocated to that function.

So is there any general advice somebody can offer how powershell allocates and frees it's memory, or any other advice that you can offer?


I am even calling the GC::Collect() and it is still not helping

thanks,

Ward

Below is a simplified version of the code.

function Test1($machine_list)

$data_list = @()

foreach ($pc in $machine_list)
{
	$obj = New-Object System.Object
	$obj | Add-Member -MemberType NoteProperty Data1 $pc
	$obj | Add-Member -MemberType NoteProperty Data2 $pc
	$obj | Add-Member -MemberType NoteProperty Data3 $pc
	$data_list += $obj
}

$data_list | Export-Csv c:\scripts\test.txt -notype

$data_list = $null
}

$machine_list = @()

#select Name from Table where ProductID = 'XX'

# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

0
Comment
Question by:whorsfall
2 Comments
 
LVL 68

Accepted Solution

by:
Qlemo earned 500 total points
ID: 39705047
It is bad style (but commonly used) in PowerShell to collect output into a var in a loop, just to push the results out without further processing. That way PS has to allocate and deallocate small chunks of memory in a bigger, non-continous memory area, which leads to more memory consumption and excessive memory fragmentation.
Local vars (used only in functions or script blocks) are marked for GC whenever their scope is left, of course, but that doesn't mean GC will jump in immediately.

Whenever possible, use the pipeline, and no intermediate objects, to lessen memory impact.
function Test1($machine_list)
{
  $machine_list | foreach-object
  {
    New-Object PsObject -Property @{
      Data1 = $pc
      Data2 = $pc
      Data3 = $pc
    }
  } | Export-Csv c:\scripts\test.txt -notype
}


$machine_list = @()
#select Name from Table where ProductID = 'XX'
# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

Considering to combine SQL SELECT and the test1 function with external Export-CSV might be better, but that depends on what you do in fact in the function. Hard to tell with the overly simplified code you showed ;-).
(#select expression resulting in machine (string) objects put into the pipe) | test1 | export-csv c:\scripts\test.txt -notype

Open in new window

(with the export-csv removed from test1, of course).
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

In this previous article (https://oddytee.wordpress.com/2016/05/05/provision-new-office-365-user-and-mailbox-from-exchange-hybrid-via-powershell/), we made basic license assignments to users in O365. When I say basic, the method is the simplest way …
Set OWA language and time zone in Exchange for individuals, all users or per database.
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now