Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Powershell Memory Usage / Leakage?

Posted on 2013-12-07
2
Medium Priority
?
362 Views
Last Modified: 2013-12-14
Hi,

I am having an issue with a long running powershell script with a the memory increasing over time, like a leak. I am creating collection variables but after doing a connection to sql. Later on I am writing the output to a CSV file.  However in my functions I am setting the collection variables to $null. But the memory keeps going up and up.

Is there something special I am missing on how powershell frees it memory. So for example if I call function a and I have a variable called $test should the memory be free for what was allocated to that function.

So is there any general advice somebody can offer how powershell allocates and frees it's memory, or any other advice that you can offer?


I am even calling the GC::Collect() and it is still not helping

thanks,

Ward

Below is a simplified version of the code.

function Test1($machine_list)

$data_list = @()

foreach ($pc in $machine_list)
{
	$obj = New-Object System.Object
	$obj | Add-Member -MemberType NoteProperty Data1 $pc
	$obj | Add-Member -MemberType NoteProperty Data2 $pc
	$obj | Add-Member -MemberType NoteProperty Data3 $pc
	$data_list += $obj
}

$data_list | Export-Csv c:\scripts\test.txt -notype

$data_list = $null
}

$machine_list = @()

#select Name from Table where ProductID = 'XX'

# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

0
Comment
Question by:whorsfall
1 Comment
 
LVL 71

Accepted Solution

by:
Qlemo earned 2000 total points
ID: 39705047
It is bad style (but commonly used) in PowerShell to collect output into a var in a loop, just to push the results out without further processing. That way PS has to allocate and deallocate small chunks of memory in a bigger, non-continous memory area, which leads to more memory consumption and excessive memory fragmentation.
Local vars (used only in functions or script blocks) are marked for GC whenever their scope is left, of course, but that doesn't mean GC will jump in immediately.

Whenever possible, use the pipeline, and no intermediate objects, to lessen memory impact.
function Test1($machine_list)
{
  $machine_list | foreach-object
  {
    New-Object PsObject -Property @{
      Data1 = $pc
      Data2 = $pc
      Data3 = $pc
    }
  } | Export-Csv c:\scripts\test.txt -notype
}


$machine_list = @()
#select Name from Table where ProductID = 'XX'
# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

Considering to combine SQL SELECT and the test1 function with external Export-CSV might be better, but that depends on what you do in fact in the function. Hard to tell with the overly simplified code you showed ;-).
(#select expression resulting in machine (string) objects put into the pipe) | test1 | export-csv c:\scripts\test.txt -notype

Open in new window

(with the export-csv removed from test1, of course).
0

Featured Post

Creating Active Directory Users from a Text File

If your organization has a need to mass-create AD user accounts, watch this video to see how its done without the need for scripting or other unnecessary complexities.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In threads here at EE, each comment has a unique Identifier (ID). It is easy to get the full path for an ID via the right-click context menu. However, we often want to post a short link within a thread rather than the full link. This article shows a…
Originally, this post was published on Monitis Blog, you can check it here . In business circles, we sometimes hear that today is the “age of the customer.” And so it is. Thanks to the enormous advances over the past few years in consumer techno…
The viewer will learn how to dynamically set the form action using jQuery.
Loops Section Overview
Suggested Courses

877 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question