Solved

Powershell Memory Usage / Leakage?

Posted on 2013-12-07
2
335 Views
Last Modified: 2013-12-14
Hi,

I am having an issue with a long running powershell script with a the memory increasing over time, like a leak. I am creating collection variables but after doing a connection to sql. Later on I am writing the output to a CSV file.  However in my functions I am setting the collection variables to $null. But the memory keeps going up and up.

Is there something special I am missing on how powershell frees it memory. So for example if I call function a and I have a variable called $test should the memory be free for what was allocated to that function.

So is there any general advice somebody can offer how powershell allocates and frees it's memory, or any other advice that you can offer?


I am even calling the GC::Collect() and it is still not helping

thanks,

Ward

Below is a simplified version of the code.

function Test1($machine_list)

$data_list = @()

foreach ($pc in $machine_list)
{
	$obj = New-Object System.Object
	$obj | Add-Member -MemberType NoteProperty Data1 $pc
	$obj | Add-Member -MemberType NoteProperty Data2 $pc
	$obj | Add-Member -MemberType NoteProperty Data3 $pc
	$data_list += $obj
}

$data_list | Export-Csv c:\scripts\test.txt -notype

$data_list = $null
}

$machine_list = @()

#select Name from Table where ProductID = 'XX'

# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

0
Comment
Question by:whorsfall
2 Comments
 
LVL 68

Accepted Solution

by:
Qlemo earned 500 total points
ID: 39705047
It is bad style (but commonly used) in PowerShell to collect output into a var in a loop, just to push the results out without further processing. That way PS has to allocate and deallocate small chunks of memory in a bigger, non-continous memory area, which leads to more memory consumption and excessive memory fragmentation.
Local vars (used only in functions or script blocks) are marked for GC whenever their scope is left, of course, but that doesn't mean GC will jump in immediately.

Whenever possible, use the pipeline, and no intermediate objects, to lessen memory impact.
function Test1($machine_list)
{
  $machine_list | foreach-object
  {
    New-Object PsObject -Property @{
      Data1 = $pc
      Data2 = $pc
      Data3 = $pc
    }
  } | Export-Csv c:\scripts\test.txt -notype
}


$machine_list = @()
#select Name from Table where ProductID = 'XX'
# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

Considering to combine SQL SELECT and the test1 function with external Export-CSV might be better, but that depends on what you do in fact in the function. Hard to tell with the overly simplified code you showed ;-).
(#select expression resulting in machine (string) objects put into the pipe) | test1 | export-csv c:\scripts\test.txt -notype

Open in new window

(with the export-csv removed from test1, of course).
0

Featured Post

Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This is a PowerShell web interface I use to manage some task as a network administrator. Clicking an action button on the left frame will display a form in the middle frame to input some data in textboxes, process this data in PowerShell and display…
A brief introduction to what I consider to be the best editor for PowerShell.
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now