asked on

VBScript - Recursion Question (AD Browsing)

This is my first question on this forum, so please forgive any mistakes I may make.

I'm attempting to get the counts of all the groups in an Active Directory Domain by enumerating the groups through the ADSI LDAP provider. I have a routine that is setup to recursively browse the OUs\Containers in the domain, bind to the groups within each OU\Container, and then access the count (functioning code is attached).

The code functions, but runs very inefficiently. After the initial OUs\Containers are enumerated, child objects are enumerated recursively. The executable grows in memory size to almost 1.5GB and doesn't ever shrink back down. It seems as if the executable isn't releasing the memory used by a subroutine when the subroutine has completed.

My Question is this: Can someone tell me why the memory usage for this code is so out of control, and possibly suggest a remedy.

Side Notes: I am aware of other methods for enumerating group memberships, including the ADSI WinNT provider and using ADO Queries. I would like to get the method I am currently using (ADSI LDAP provider) at a more manageable state, because the ADSI LDAP provider has advantages the other methods do not. While I can appreciate other suggestions, my main concern is the high memory usage due to recursion.

Thank you, and Kind Regards.

'===============================================================================
'===============================================================================
Dim objFSO      'File System Object
Dim objShell    'Windows Shell
Dim strDomain   'Domain to Enumerate (LDAP Path)
 
'-->Set the LDAP Path to the Domain
strDomain = "LDAP://FakeDomainName"
'-->Initialize basic variables
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objShell = CreateObject("wscript.shell")
'-->Enumerate Domain Groups
GetBaseContainers
'-->Completion notification
MsgBox "Finito",,"Script Completion Notice"
 
'===============================================================================
'Sub GetBaseContainers
'===============================================================================
Sub GetBaseContainers
    On Error Resume Next
 
    Dim objDomain       'Domain Object instantiated
    Dim objOU           'Organizational Unit or container instantiated
 
    '-->Bind to the Domain
    Set objDomain = GetObject(strDomain)
    '-->DomainDNS object contains no groups, set the filter to OUs
    objDomain.Filter = Array("organizationalUnit")
    '-->Cycle through each OU and enumerate groups
    For Each objOU In objDomain
        EnumLDAPOU objOU.ADSPath,"FakeDomainName/" & Right(objOU.Name,Len(objOU.Name)-3)
    Next
    '-->DomainDNS object contains no groups, set the filter to Containers
    objDomain.Filter = Array("container")
    '-->Cycle through each container and enumerate groups
    For Each objOU In objDomain
        EnumLDAPOU objOU.ADSPath,"FakeDomainName/" & Right(objOU.Name,Len(objOU.Name)-3)
    Next
    '-->DomainDNS object contains no groups, set the filter to builtinDomain
    objDomain.Filter = Array("builtinDomain")
    '-->Cycle through each builtinDomain object and enumerate groups
    For Each objOU In objDomain
        EnumLDAPOU objOU.ADSPath,"FakeDomainName/" & Right(objOU.Name,Len(objOU.Name)-3)
    Next
End Sub
 
'===============================================================================
'Sub EnumLDAPOU
'===============================================================================
Sub EnumLDAPOU(LDAPPath,LDAPTrace)
    On Error Resume Next
 
    Dim objLDAPOU       'Organization unit object
    Dim objGroup        'Group object
    Dim strName         'The Name of the group
    Dim strPath         'The LDAP ADSPath of the group
    Dim intLDAPCount    'The number of members in the group
 
    '-->Bind to an AD OU
    Set objLDAPOU = GetObject(LDAPPath)
    '-->Look at just the groups in the OU
    objLDAPOU.Filter = Array("group")
    '-->Go through each group and get it's attributes
    For Each objGroup In objLDAPOU
        '-->Initilaize attribute variables
        strName = ""
        strPath = ""
        '-->Get the attribute variables
        strName = LCase(objGroup.SamAccountName)
        strPath = LCase(objGroup.ADSPath)
        strPath = Replace(strPath,"ldap://","LDAP://")
        '-->Get the membership count
        intLDAPCount = 0
        intLDAPCount = objGroup.Members.Count
        '--------------->
        'More Code is here to keep track of the counts
        '--------------->
    Next
    objGroup = Null
    '-->Enumerate any groups in this OU's Sub OUs
    objLDAPOU.Filter = Array("organizationalUnit")
    For Each objOU In objLDAPOU
        EnumLDAPOU objOU.ADSPath,LDAPTrace & "/" & Right(objOU.Name,Len(objOU.Name)-3)
    Next
    '-->Enumerate any groups in this OU's Sub OUs
    objLDAPOU.Filter = Array("container")
    For Each objOU In objLDAPOU
        EnumLDAPOU objOU.ADSPath,LDAPTrace & "/" & Right(objOU.Name,Len(objOU.Name)-3)
    Next
End Sub
 
'===============================================================================
'===============================================================================

Open in new window

SOLUTION

exx1976

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

gregcmcse

In your subroutine:
objGroup = Null
should be
set objGroup = Nothing

At the end of the subroutine:
set objOU = Nothing
set objLDAPOU = Nothing

Your problem may be that you're creating many, many connections to LDAP and never clearing them out. Another possibility is that you have a very large domain with many group objects and OU objects and you're simply asking it for more than it can do.

I'd probably do an ADO query instead of iterating through each LDAP object due to performance. ADO is VERY fast, LDAP is VERY slow. So unless you need to edit each and every item, I'd enumerate with ADO and specifically bind with LDAP if you need to query more info. I'll attach a sample shortly.

exx1976

Actually, this code would be incrementally faster than the previous code I posted:

Set oDomain = GetObject("LDAP://DC=fakedomain,DC=local")
For Each item in oDomain
ListGroups(item.adspath)
Next

Function Listgroups(oObj)
On Error Resume Next
Set thing = GetObject(oObj)
For Each child In thing
If child.class = "group" Then
WScript.Echo(vbcrlf & "Group: " & child.adspath)
WScript.Echo(child.members.count)
ElseIf child.class = "organizationalUnit" or child.class = "container" or _
child.class = "builtinDomain" Then Listgroups(child.adspath)
End If
Next
End Function

FirstDecan

ASKER

exx1976,
Yeah, it's a bit much. This is actually parred down from another script that does a lot more, so there's extra "stuff" in there.

I ran your code, and I'm running into the exact same problem. Within 10 minutes, the executable ballooned to 1.5GB.

I appreciate your response, and especially the time you took to simplify the code I had posted. Unfortunately this has not reduced the memory usage.

exx1976

With the code I posted being "barebones", if it still uses the same amount of memory, then it's due to the number of containers in your directory, and thus the number of recursive calls that have to be stored on the heap at runtime.. When I execute that code against my directory, it never gets above 11MB of memory utilization, and it returns 204 groups. OUs in my directory are only nested 4 deep at the max.

Perhaps it is best to look at ADO or something similar for your application.

ASKER CERTIFIED SOLUTION

gregcmcse

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

gregcmcse

Actually, here is a script that I wrote a few years ago... it may not be as pretty as it should be, because I was much less experienced with VBScript & ADSI at that point, but it worked for me then to overcome your issue.

Only the names have been changed to protect the client...


'On Error Resume Next
 
Dim WSH, FSO, fnOutput, myFile
Set WSH = CreateObject("WScript.Shell")
Set FSO = CreateObject("Scripting.FileSystemObject")
fnOutput = "\\server\share\EnumGroupsbyOU.txt"
Set myFile = FSO.CreateTextFile(fnOutput,TRUE)
 
Dim oDSE, strDN, oCN, oChild, oGrpOU
CONST DCName="mydomaincontroller"
CONST DomainName="somedomain.com"
 ' Security enabled group value constants -- for dist list type groups subtract &H80000000
Const GLOBAL_GROUP = &H80000002
Const DOMAIN_LOCAL_GROUP = &H80000004
Const UNIVERSAL_GROUP = &H80000008
 
Set oDSE = GetObject("LDAP://rootDSE")
strDN = oDSE.Get("defaultNamingContext")
 
Set oCN = GetObject("LDAP://" & strDN)
 
Call ListOUs(oCN)
 
myFile.Close
WSH.Run "Notepad.exe " & fnOutput
 
 
Sub ListOUs(oADObj)
	Dim oChild
	For Each oChild in oADObj
		Select Case oChild.Class
			Case "organizationalUnit" 
				Call ListGroups(oChild.Get("distinguishedName"))
 
				Call ListOUs(oChild)
 
			Case "container"
				Call ListGroups(oChild.Get("distinguishedName"))
 
				Call ListOUs(oChild)
		End select
	Next
End Sub
 
 
Sub ListGroups(SDOUString)
	
	Dim SDOU, bOutput, strOutput, oGroup, strGroup, retval
 
	On Error Resume Next
 
	Set SDOU=GetObject("LDAP://" & DomainName & "/" & SDOUString)
	SDOU.Filter = Array("Group")
 
	On Error Goto 0
 
	bOutput = False
	strOutput = SDOUString & vbCrLf & "===========================" & vbCrLf
 
	For Each oGroup In SDOU
 
		Select Case oGroup.GroupType
			Case GLOBAL_GROUP
				strGroup = "Global Group"
			Case DOMAIN_LOCAL_GROUP
				strGroup = "Domain Local Group"
			Case UNIVERSAL_GROUP
				strGroup = "Universal Group"
			Case Else
				strGroup = "Non-security-enabled group or invalid group type"
		End Select
		
		strOutput = strOutput & Right(oGroup.Name,len(oGroup.Name)-3) & " (" & strGroup & ")" & vbCrLf
		
		retval = ListMembers("WinNT://" & DomainName & "/" & Right(oGroup.FullName,len(oGroup.FullName)-3) & ",Group","  ")
 
		bOutPut = True
 
	Next
 
	If bOutput Then myFile.WriteLine strOutput
 
End Sub ' ListGroups
 
Function ListMembers(strGrp,strIndent)
 
	' Beware of evil recursive code contained within!
	Dim oGrp, oMember, oMbrMbr, retval2
	
	On Error Resume Next
	
	Set oGrp = GetObject(strGrp)
	If Err.number Then Wscript.Echo "Error " & err.number & " attempting path " & strGrp
	On Error Goto 0
	
	For Each oMember In oGrp.Members
	
		strOutput = strOutput & strIndent & oMember.Name & vbCrLf
	
		If (oMember.Class = "Group") Then
	
			strOutput = strOutput & strIndent & "---------" & vbCrLf
	
			For Each oMbrMbr In oMember.Members
				retval2 = ListMembers("WinNT://" & DomainName & "/" & oMember.Name & ",Group",strIndent & "  ")
			Next
	
		End If
	
	Next
	
End Function ' ListMembers

Open in new window

SOLUTION

RobSampson

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

RobSampson

Actually, Greg's last code there looks pretty similar in concept.....see how you go....

Regards,

Rob.

FirstDecan

ASKER

Firstly, I'd like to thank everyone for their responses.

exx1976,
It may be an issue of size. I'm running this against a very large AD implementation. I have code that performs the enumerations through ADO and also using the WinNT provider, and this code never peaks above 45MB in either instance. I was hoping someone might have some salacious bit of info that I hadn't come across yet.

gregcmcse,
I tried your suggested method of setting the object to nothing, but that didn't resolve the issue. That same method was suggested when I googled my issue and came across references to memory leaks in VBS. The memory seems to max out whether or not those variables are reset.

RobSampson,
ADO would be much quicker, but it's read only. That would work well to report on findings\group configurations, but if I find something undesirable I'd probably be expected to fix it. While the problem I posted was simply getting group counts, my main concern is actually the out of control memory usage from using the ADSI LDAP provider recursively.

All three of you have suggested that what I'm trying to do is simply not technically feasible, and have suggested using ADO as an alternative. I was hoping that someone would know of some nifty trick that would magically make my code work, but I may be out of luck in that regard.

I'm gonna let this sit for a bit and hope for a miracle. If I don't come across anything else, I will resolve this and split the points amongst you.

Thank you all very kindly for your responses.

RobSampson

>> ADO would be much quicker, but it's read only.

Technically, that's true, but you can still use the LDAP provider to then bind to an object, and make changes to it.

I just noticed a little error in my code. This:
objRecordSet.Fields("distinguishedName").Value
should be
objRecordSet.Fields("adsPath").Value

For example, once ADO has found the required records, I output the adsPath of the object using:
strResults = strResults & VbCrLf & """" & objRecordSet.Fields("Name").Value & """,""" & objRecordSet.Fields("adsPath").Value & """"

So, lets assume that that query pulled a user adsPath, instead of a group, just for illustration. Now, we now that the adsPath is the full LDAP path, such as
LDAP://CN=John Smith,OU=Users,OU=Sites,DC=domain,DC=com

so, we can then bind to that object using
Set objUser = GetObject(objRecordSet.Fields("adsPath").Value)

Then, you can change attributes, by using:
objUser.telephoneNumber = "0311115555"
objUser.SetInfo

to change that user's values.

So yes, while ADO does retrieve values only, you can then use the LDAP provider to bind to an object, and perform operations on it.

Regards,

Rob.

RobSampson

Here's an example of using the same approach....ADO to search, LDAP to change...

How Can I Standardize the Logon Name for All My Users?
http://www.microsoft.com/technet/scriptcenter/resources/qanda/mar08/hey0325.mspx

Rob.

FirstDecan

ASKER

So, a little further experimentation has yielded some interesting results:

It's not the recursion as I thought that was causing the memory to balloon up so much. If I run any of the previous code without accessing the count property, the memory does not balloon up in size.

For example, If I simply access "objGroup.SamAccountName"
VBScript releases the memory used as it cycles through the code.

If I access "objGroup.Members.Count"
VBScript keeps something in memory until the code completes.

It's not just the "count" property itself. If I cycle through the "members" one at a time, I get the same ballooning effect. I've also listed all my groups through ADO, bound to the groups individually through the ADSI LDAP provider and verified the same behavior. I am assuming (possibly incorrectly) that this is a memory leak that happens when the properties of an object are also objects.

Having said all that, if anyone knows how to free up that memory, please let me know.

RobSampson

Interesting......so, when you don't use objGroup.Members.Count, does the script actually finish? My first thought just now, is that because you are recursively enumerating the groups, perhaps there's a chance that you actually have a circular group membership going on?

That is, Group1 is a member of Group2, which is a member of Group3 , which is a member of Group1.

Because of that, it gets back to Group1, and just keeps going in circles.....

See here, this talks about monitoring the groups through a dictionary object, and not counting the group if it's been used in that iteration....

http://www.rlmueller.net/CircularNested.htm

Regards,

Rob.

FirstDecan

ASKER

I actually backend to XML in the full script, it has a lot more flexibility than the dictionary object. And yes, both scripts complete without the use of the Members property (My original and the bare bones version posted). As a matter of fact, the executable stays under 45MB or so when I don't use the Members interface.

I'm not enumerating any nested groups, it's just a flat enumeration from the container structure in AD, so there shouldn't be any infinite recursion. That is something that I had checked for originally (Notice the LDAPTrace parameter in my code, it tracks the levels of enumeration. The actual code runs in an HTA, so I can update the display as it runs and monitor its progress).

I also used ADO to get a list of all the groups in the domain, and then verified the memory leak still happens without the recursion by binding to each group from the ADO query. I have additionally tried to access information from another object property of the group (ntSecurityDescriptor), and the memory leak does not occur with that property. It seems to be particular to the Members property when using the LDAP provider, because it doesn't happen with the WinNT provider.

I'm beginning to think it's just buggy MS behavior. There's enough peculiarities about how the different providers behave, I guess this shouldn't surprise me, even if it does annoy me.

Thanks for your response.

FirstDecan

ASKER

Gentleman,
I appreciate your advice, and have split the points amongst you. ADO was an option I had already been aware of, I was simply hoping to keep the number of methods for accessing the AD info to a minimum to reduce the complexity of the script.

Thank you for your attention and your efforts.