Expiring Today—Celebrate National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Powershell Memory Usage / Leakage?

Posted on 2013-12-07
2
Medium Priority
?
358 Views
Last Modified: 2013-12-14
Hi,

I am having an issue with a long running powershell script with a the memory increasing over time, like a leak. I am creating collection variables but after doing a connection to sql. Later on I am writing the output to a CSV file.  However in my functions I am setting the collection variables to $null. But the memory keeps going up and up.

Is there something special I am missing on how powershell frees it memory. So for example if I call function a and I have a variable called $test should the memory be free for what was allocated to that function.

So is there any general advice somebody can offer how powershell allocates and frees it's memory, or any other advice that you can offer?


I am even calling the GC::Collect() and it is still not helping

thanks,

Ward

Below is a simplified version of the code.

function Test1($machine_list)

$data_list = @()

foreach ($pc in $machine_list)
{
	$obj = New-Object System.Object
	$obj | Add-Member -MemberType NoteProperty Data1 $pc
	$obj | Add-Member -MemberType NoteProperty Data2 $pc
	$obj | Add-Member -MemberType NoteProperty Data3 $pc
	$data_list += $obj
}

$data_list | Export-Csv c:\scripts\test.txt -notype

$data_list = $null
}

$machine_list = @()

#select Name from Table where ProductID = 'XX'

# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

0
Comment
Question by:whorsfall
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
2 Comments
 
LVL 71

Accepted Solution

by:
Qlemo earned 2000 total points
ID: 39705047
It is bad style (but commonly used) in PowerShell to collect output into a var in a loop, just to push the results out without further processing. That way PS has to allocate and deallocate small chunks of memory in a bigger, non-continous memory area, which leads to more memory consumption and excessive memory fragmentation.
Local vars (used only in functions or script blocks) are marked for GC whenever their scope is left, of course, but that doesn't mean GC will jump in immediately.

Whenever possible, use the pipeline, and no intermediate objects, to lessen memory impact.
function Test1($machine_list)
{
  $machine_list | foreach-object
  {
    New-Object PsObject -Property @{
      Data1 = $pc
      Data2 = $pc
      Data3 = $pc
    }
  } | Export-Csv c:\scripts\test.txt -notype
}


$machine_list = @()
#select Name from Table where ProductID = 'XX'
# Then put each Name into a machine_list variable.

Test1 $machine_list

Open in new window

Considering to combine SQL SELECT and the test1 function with external Export-CSV might be better, but that depends on what you do in fact in the function. Hard to tell with the overly simplified code you showed ;-).
(#select expression resulting in machine (string) objects put into the pipe) | test1 | export-csv c:\scripts\test.txt -notype

Open in new window

(with the export-csv removed from test1, of course).
0

Featured Post

[Webinar] Lessons on Recovering from Petya

Skyport is working hard to help customers recover from recent attacks, like the Petya worm. This work has brought to light some important lessons. New malware attacks like this can take down your entire environment. Learn from others mistakes on how to prevent Petya like worms.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A procedure for exporting installed hotfix details of remote computers using powershell
This script can help you clean up your user profile database by comparing profiles to Active Directory users in a particular OU, and removing the profiles that don't match.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.

719 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question