Solved

Powershell Memory Usage / Leakage?

Posted on 2013-12-07
2
342 Views
Last Modified: 2013-12-14
Hi,

I am having an issue with a long running powershell script with a the memory increasing over time, like a leak. I am creating collection variables but after doing a connection to sql. Later on I am writing the output to a CSV file.  However in my functions I am setting the collection variables to $null. But the memory keeps going up and up.

Is there something special I am missing on how powershell frees it memory. So for example if I call function a and I have a variable called $test should the memory be free for what was allocated to that function.

So is there any general advice somebody can offer how powershell allocates and frees it's memory, or any other advice that you can offer?


I am even calling the GC::Collect() and it is still not helping

thanks,

Ward

Below is a simplified version of the code.

function Test1($machine_list)

$data_list = @()

foreach ($pc in $machine_list)
{
	$obj = New-Object System.Object
	$obj | Add-Member -MemberType NoteProperty Data1 $pc
	$obj | Add-Member -MemberType NoteProperty Data2 $pc
	$obj | Add-Member -MemberType NoteProperty Data3 $pc
	$data_list += $obj
}

$data_list | Export-Csv c:\scripts\test.txt -notype

$data_list = $null
}

$machine_list = @()

#select Name from Table where ProductID = 'XX'

# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

0
Comment
Question by:whorsfall
2 Comments
 
LVL 69

Accepted Solution

by:
Qlemo earned 500 total points
ID: 39705047
It is bad style (but commonly used) in PowerShell to collect output into a var in a loop, just to push the results out without further processing. That way PS has to allocate and deallocate small chunks of memory in a bigger, non-continous memory area, which leads to more memory consumption and excessive memory fragmentation.
Local vars (used only in functions or script blocks) are marked for GC whenever their scope is left, of course, but that doesn't mean GC will jump in immediately.

Whenever possible, use the pipeline, and no intermediate objects, to lessen memory impact.
function Test1($machine_list)
{
  $machine_list | foreach-object
  {
    New-Object PsObject -Property @{
      Data1 = $pc
      Data2 = $pc
      Data3 = $pc
    }
  } | Export-Csv c:\scripts\test.txt -notype
}


$machine_list = @()
#select Name from Table where ProductID = 'XX'
# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

Considering to combine SQL SELECT and the test1 function with external Export-CSV might be better, but that depends on what you do in fact in the function. Hard to tell with the overly simplified code you showed ;-).
(#select expression resulting in machine (string) objects put into the pipe) | test1 | export-csv c:\scripts\test.txt -notype

Open in new window

(with the export-csv removed from test1, of course).
0

Featured Post

Webinar: Aligning, Automating, Winning

Join Dan Russo, Senior Manager of Operations Intelligence, for an in-depth discussion on how Dealertrack, leading provider of integrated digital solutions for the automotive industry, transformed their DevOps processes to increase collaboration and move with greater velocity.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Export DG's members info in Office 365. 2 25
Exchange Inbox Count 4 40
devops career advice 1 33
Execute Remote Powershell command Using System account 3 21
This article will help you understand what HashTables are and how to use them in PowerShell.
This article explains how to prepare an HTML email signature template file containing dynamic placeholders for users' Azure AD data. Furthermore, it explains how to use this file to remotely set up a department-wide email signature policy in Office …
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.

679 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question