Citrix, multi forest Xenapp servers

Hi

We have a 7 server Xenapp 6.5 farm currently configured in an AD forest (call it Forest1).  We have a file server in the same domain which hosts all of the streamed packages that we publish to users.

Users are in child domains of the above forest and authentication is through SSO and AD group membership.  We currently have three regional domains.

1) europe.forest1.com
2) asia.forest1.com
3) namerica.forest1.com

We have recently provisioned an additional Xenapp server in a different forest (Forest2) along with a different file server to host copies of the same published applications.  No users exist in Forest2.

I need for users to be able to launch a second copy of this application and use the Xenapp server in Forest 2.  So far, when it was joined to the farm, I provided the datastore credentials for Forest1 and the connection was successful.  I can see it inside the AppCenter and all looks good.  I created new published apps and a server desktop for the Forest2 Xenapp server.  Whenever I try to launch them, I get an error (attached).  Citrix Receiver error
I think this is because whatever context its starting the application with, does not have rights over the Forest2 file server, but I am unsure how to find out that context or what log file can help me troubleshoot.

I know this isn't a recommended setup for Xenapp, but I believe it is possible.  We have two way transitive trusts in place already between the forests.

Many thanks

WE
WanderingEngineerAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Will SzymkowskiSenior Solution ArchitectCommented:
The only other thing that i can think of if you have a proper trust in place already is making sure that you have your DNS configured correctly between the two forests.

Also what URL are you using for your Receiver? And is this URL accessible from both domains.

Will.
Brian MurphyIT ArchitectCommented:
Your trying to do SSO with the Receiver client where your XenApp servers exist in another domain?  In this case, a different forest?

You don't need a transitive trust, you can get away with a one-way shortcut trust from that forest root to the child domain where the child domain trusts the forest 2.  It depends on resource use but maximum you can do a two-way trust from forest 2 to child domain.

In your GPO, you must configure: Allow Cross-Forest User Policy and Roaming User Profiles policy

I see what your saying about Forest 2 to Forest 1 and three child domains.  I assume you have a Domain Local Group in each User Domain and that DLG has permissions to the ICA Connection on all the XenApp Servers and member of Remote Desktop Users Group?

These are possible causes.

The short-cut trusts in addition to forest-to-forest trust would help with any slow authentication problems.

It goes much deeper, however. It would be easier if a domain controller from Forest 2 would reside in each of the child domains and in Forest 2 those networks are properly defined under sites and services.  If not, this can lead to authentication problems.
WanderingEngineerAuthor Commented:
Hi

Will - The storefront URL is accessible from the Forest2 Xenapp server.  There is a certificate error because the certificate was issued from a Forest1 CA, but I don't think is hampering us.

Brian - Just to give you a better idea of the setup.

PublishedApp1.
Each regional domain will have a Global Group for this application.
Eahc global group is a member of a Forest1 domain local group with the same name for the app.
The Forest1 app DLG is a member of a Remote Desktop Users DLG which grants RDP access to all Citrix users.

For the Forest2 Xenapp server, I've simply created an additional Remote Desktop Users group, added it to the server Remote Desktop users group.  The Forest1 has a second RDP group which is a member of the Forest2 group.  The Forest1 group then has the regional groups as members.  So in effect, the regional groups are members of two RDP groups that should give them access to all Xenapp servers in both domains.

For testing a published app, I have directly added my user account in the properties.  As I am already a member of the RDP groups, this should cover off my access.

The datastore account is an administrator on all Xenapp services.  

Single zone with a test load balancing policy.

There is a slightly different error when accessing from storefront, this is attached.
Protecting & Securing Your Critical Data

Considering 93 percent of companies file for bankruptcy within 12 months of a disaster that blocked access to their data for 10 days or more, planning for the worst is just smart business. Learn how Acronis Backup integrates security at every stage

WanderingEngineerAuthor Commented:
Here is the error.

storefront.png
Brian MurphyIT ArchitectCommented:
So, you should actually see something in the Application Log on StoreFront Server.  (Windows Logs).

I guess it depends on the version but generally authentication problems would go to Application Event log for StoreFront.

So, not Gateway Direct?  All the STA's the same?  STA's are farm agnostic but must be identical across WI and Storefront.
WanderingEngineerAuthor Commented:
Not finding anything on either SF server in any of the event logs that seems pertinent, certainly no errors.

No Gateway.  I don't know what STA is, but the software version is the same, down to HRP03 being applied to all servers in this farm.

Thanks

WE
WanderingEngineerAuthor Commented:
The Xenapp server in Forest 2 has logged the following in the event viewer (ID 4010)

'No controller could be contacted in zone Default Zone. This XenApp server supports only session-host mode and will continue polling for a controller, but will not accept new connections until one is online.'

All farm servers are in the default zone and there are two controllers.  Is this just an authroity issue with accounts.  What account does the Xenapp server use to communicate with the controller (bear in mind, they are in different forests).

Thanks

WE
Brian MurphyIT ArchitectCommented:
Well, this is a guess only but session hosting mode does not allow that server to act as a Zone Data Collector.

You need at least one non-session-host mode server to act only as ZDC. One is the minimum.

If you have two controllers and they are not running session-host mode the servers speak to each other using the built-in Network Service account.  

It doesn't require a service account on the domain and you should be able to have one farm that crosses multiple forests.

Did you setup a forest to forest trust?  

Citrix running on the Microsoft Operating System and depending on version will usually install at least one service account that is local only.  Those are for local use not domain.

The newer versions use the Network Service account and if you have a trust to answer your question Citrix communicates to other Citrix servers on port 2512.

There is actually a lot of stuff going on their, Citrix has a "transport function" to send the packets between each other.

It is strange that you could add another server to the same Zone.  The default zone or ZDC's do not care about active directory domains.  They are the traffic cop for all things happening in their zone.  

There are a lot of tools to determine certain things but I usually change the Default Zone name to something else.  You can name it after the Citrix or location of the data center but if you change it and all your session hosts recognize the change including the other ones in the other forest that rules out some things.

I would start with going to http://support.citrix.com/

IMA Helper might give you move information
http://support.citrix.com/article/CTX133983

It captures information real time and helps narrow down IMA startup issues.

It doesn't sound like a Farm database issue but you might have other issues that DSCHeck will point out and I would run this at least once a month when I had 6.5
http://support.citrix.com/article/CTX137608

This might be an authentication delay.  It happens sometimes with two forests or two forests with child domains.

You might want to logon to a domain controller and verify the trust.  In 6.5 you need a bidirectional forest trust and afterward you can create domain trusts or "shortcut trusts" but a bidirectional trust covers the forest and all domains.

If not mistaken, those forests must match in regard to function level.

If you run, nltest /trusted_domains

Use NLTest.exe from command prompt

# nltest /trusted_domains (Enter)
0: forest1.local (NT 5) (Direct Outbound) (Direct Inbound) ( Attr: 0x8 )
1: forest2.local (NT 5) (Forest Tree Root) (Primary Domain) (Native)

Is an example.  You need direct outbound and direct inbound on line 0:

#1 will change depending on the forest your run the command.  I would run it in both.

The next thing that comes to mind is if that NOTEPAD is in FOREST1.LOCAL you would need something from FOREST2.LOCAL to grant permissions.

The trust is not enough.  If you want users to access NOTEPAD in Forest 1 you need a Domain Local Group in Forest 1 and a Domain Global Group in Forest 2.

Users from Forest 2 are added to DOMAIN-GLOBAL-GROUP-NOTEPAD

That global group would be added to the Domain Local Group in Forest 1.

That Domain Local Group in Forest 1 is what grants permissions to Notepad.

Hopefully this helps.  I'm trying to help rule things out at this point.
WanderingEngineerAuthor Commented:
Hi Brian

That info is really useful.  Stepping through.

1) There are 2 ZDC's already in Forest 1 managing the Default Zone.  
2) There is a Forest to Forest transitive trust in place.  It has been validated.
3) Forest 1 is 2003 FL.  Forest 2 is 2008 R2 FL.
4) NLTest result

C:\>nltest /trusted_domains
List of domain trusts:
    0: FOREST1 (NT 5) (Direct Outbound) (Direct Inbound) ( Attr: 0x8 )
    1: A N OTHER DOMAIN (NT 5) (Direct Outbound) (Direct Inbound) ( Attr: 0x8 )
    2: FOREST2 (NT 5) (Forest Tree Root) (Primary Domain) (Native)
5) I have the group memberships setup for users.  I can see the published apps on the user desktops, so I know the indirect group enumeration is working.

IMA Helper gives me the following information.

IMA Status
IMA-Status.png
IMA Startup does not even complete.  Sits on step 2.  I'm leaving it running to see if it picks up and I'll post an updated screen if and when it does.
IMA-Helper.png
Thanks

WE
Brian MurphyIT ArchitectCommented:
Oh, did a quick read again.

I'm going back to the Regional Domains where you have the Global Groups and add them to Domain Local Groups where the resources are published.

Which is as I know it, best practice.

But, you mentioned Remote Desktop Users and RDP.

There is a Users group, local to each server, that by default has only that Domain Users group.

Remote Desktop Group alone is not enough to get them rights to logon, just RDP if memory serves.

I seem to remember having to add every MYDOMAIN1\Domain Users group to the local Users Group.  If you look at local security policies, under Local Computer Policy - Computer Configuration - Windows Settings - Security Settings - Local Policies - User Right Assignment

But, not 100% on this one but it can cause issues if those are controlled by GPO.  Usually anything with "Everyone" is removed.

Remote Desktop Group does let you use RDP but user still needs basic User rights.  When there was a bidirectional trust I would remove "Everyone" and set it to Authenticated Users on ICA not RDP.  I removed everything from RDP connection security and that was Administrators local group only which by default adding to a domain is populated with Domain Admins.

Then I used Restricted Groups in GPO to add the Citrix Administrators Group by default once it hit that OU, that group was added and this usually happens before you install Citrix.  Then, some people use service accounts in the domain but bound to local services.  I never liked that and first thing I did was change the Citrix UPD service to the local System account.

Now SQL, different discussion.   There is a performance hit if a domain service account is used instead of SQL account.  And the SQL Team proved it to me but I decided to stick with domain account.  

I mean, that could result in this type of error.  I don't recall if you were using Web Interface or Storefront but all that does is authenticate the user using XML and Netlogon.  So, you in at that point if it returns back a list of applications?

So, you are seeing applications if not mistaken?

If it were a trust issue, or any permission issue at all we know it is from clicking on the App, and what happens next.

That could be simply sending a launch.ica file back that tells their Receiver to "Go to this IP address on 1494".

You get permission error there, no where else? Yes?

The IMA protocol in the connection manager might be set to Remote Desktop Users as well.  So would you not have to put all Domain User groups in Remote Desktop OR Authenticated Users.

Now your past IMA permissions which is protocol.   You can be a local administrator but if someone sets ICA-TCP permissions to Remote Desktop Users only - as admin you cannot connect.

I do this with RDP.  Local Administrators only, remove everything else.

I use Authenticated Users in the "Users" group and "Remote Desktop Users" which is on ICA-TCP connector as being able to connect, and perform pass-through authentication with minimum rights that Users group gives you.

I mean, unless I missed something.  If you can see the application that means your Global groups in the domain local group and that DLG on XYZ application we can rule out a lot of stuff.

And, obviously that is where having Web Interface or Storefront is handy because if they can logon from any domain and I assume you are using one or more web servers behind a VIP and everyone goes to that web interface or SF and sees the application?  Guess I should confirm that before moving past it.
Brian MurphyIT ArchitectCommented:
Yea, as I read back through you good to go until you click the application.

And your not getting a generic error, it knows that is the application.

The Citrix Receiver client of USERNAME in SOMEDOMAIN does not have permissions on the XenApp server where Notepad runs.

You need to apply the same concept as you did with the applications published in Citrix which does not give them ICA or local permission to that server or servers to run notepad.exe

Notepad.exe in C:\Windows probably is Administrators FC and Users read.  By Users I mean the server local group.
WanderingEngineerAuthor Commented:
Hi Brian

Your reasoning above is correct.  We have all the group memberships correct across the two domains.  One thing to bear in mind is that all users are in Forest1.  What we have is a Citrix All Users group which is a member of an equivalent group in Forest2.  This Forest2 group is then a member of the Local RDP Users group on each Forest2 Xenapp server.

As a test, I am a domain admin in Forest2, so I gave myself access to the published application, the test notepad app and the server desktop.  I get the same error on all three connections.

cashbook_fblp_applaunch_error.png
I still think I need to resolve the error with it not being able to see the ZDC.  If I can fix this, I'm pretty confident the apps will launch.  I've gone back over the group security and group policies and they are consistent with what is configured in Forest1.

Are there any logs where I can get more information on why it can't see the ZDC?  The Event Log only has the 4010 errors which aren't yielding many results on the web.

Thanks

WE
WanderingEngineerAuthor Commented:
OK, so more investigation.  The affected xenapp server can't determine a zone collector when queried.

zone-collector-error.png
From the other servers, I get the correct ZDC.  My question is, where do I define this so it can correctly talk to the server.  They are all in a single Default Zone.  My understanding is, it doesn't need a separate ZDC if they are separated by domain/forest, but in a single Citrix zone.

Thanks

WE
Brian MurphyIT ArchitectCommented:
Yea.  Not seeing any primary or secondary servers.  You must be split across subnets.  

What about "query farm"?

Now, "dscheck /full servers"?

Each subnet should have a different zone name.  Regardless of same physical location.

Should list servers with asterick by the servername of ZDC.  Otherwise you have a zone "clash" or database.

Was that server joined as "Session Hosting"? Must be installed as non-session hosting.

If it is ZDC it should show asterick for itself where the subnets differ.  I would change the default zone to something else first.  

It cannot run as ZDC for that subnet.  You might have to Remove then Join again using:
 /ImaWorkerMode: False

http://support.citrix.com/article/ctx112525 

“C:\Program Files (x86)\Citrix\XenApp\ServerConfig\
-XenAppConfigConsole.exe" /ExecutionMode:Join
/OdbcUserName:administrator /OdbcPassword:somepasswd
/LicenseServerName:somelicenseserver
/LicenseServerPort:27000 /LicenseModel:XA
/ZoneName:some_zone_name
/DsnFile:"c:\somepath\to\example.dsn"
/Log:c:\SomewhereConfigLog.txt /CustomXmlServicePort:8080
/AddAnonymousUsersToRemoteDesktopUserGroup:True
/AddUsersGroupToRemoteDesktopUserGroup:True
/AddAuthenticatedUserstoRemoteDesktopUserGroup:True

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
WanderingEngineerAuthor Commented:
The server was originally joined as Session Hosting, but I then went through and removed it and added it back as non session hosting.

The other steps I'll work on tomorrow and update you.

Thanks

WE
Seth SimmonsSr. Systems AdministratorCommented:
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Citrix

From novice to tech pro — start learning today.