XDMCP Connection Timeout from Windows XP to Solaris 8 system when using WRQ Reflection X


At one time - few weeks ago I was always able to access the Solaris system through WRQ's Reflection X product. Now it gives me the XDMCP timeout error. The message pop-up says the following:

"Your XDMCP connection timed out - make sure that hosts on you network are running XDM programs: (RX2102)"

I have even increased the timeout limit from default 15 seconds to 300 seconds - same error.

I was always able to connect to this system successfully from the local LAN as well as VPN connection. It was fast and efficient. Now I have this error condition. I did reboot the Solaris system, yet the problem persists.

In fact this is the same error condition from any system accessing Solaris through Reflextion X session. Although, the regular telent session or WRQ's Reflection Host and Unix connection works without problem. It is the X-Session I am having trouble with from any system.

I seriously need urgent help! Please suggest what may have gone wrong all of the sudden. My assumption is that the IP port used fro Reflection X may be blocked. I have no idea how I can find the port number used., if this is the problem.

I am running Reflection X from my Windows XP system. Yet, it is the same behavior from any Windows OS.

Urgent help required!
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

either your xdm isn't running, check with ps -ef|grep openwin
and look for Xsession / Xsun, if this is running try from your PC
telnet solaris_host 6000
replace solaris_host with name or IP of your solaris server.  If you get a telnet window and no error, it means your PC is connecting on the first X11 port if you get a coonection failed! error, it is almost certainly a firewall.

Are there any firewalls (i.e. have you put XP SP2 on your PC?) between the Sun & PC.  If so you will need to allow X11 ports (6000-6063) and fs (7100)

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
westcloxAuthor Commented:
Hi! liddler,

When check for openwin process on my Solaris 8 system, it is not even running. I tried to look for Openwin or open or Open etc...... dfferent ways as normal user or root user. It is not in the list of process outputs.

Next when I telnet to <IP-Address. 6000 I get the following error:
D:\>telnet 6000
Connecting To not open connection to the host, on port 6000: Connect failed

Thus, both Process not running nor can I telnet succesfully to port 6000. I am assuming the X11 port uses 6000.

I suppose, first of all I need help to determine or strat the openwin process on this version of Solaris. The may be Telnet session to pport 6000 may work. Please help!

westcloxAuthor Commented:
To add more.....

I do not believe that anybody would stop openwin process. This is a pre-configured and pre-set platform. Nobody tampers or would know at that level of Unix to play with openwin.

As I mentioned before, I was able to use Reflection X just about 2-3 weeks ago wihtout any problem. Now, what I am not sure about is, do you need to have openwin process running for Reflection X to work?

When I searched for any X session or process running using the command:

# or $
ps -ef | grep X

There is no process with X  in the listing. Zero rows.

Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

cd /etc/rc2.d
ls *dt*
if the file S99dtlogin exists
./S99dtlogin start
Is there any output? errors? etc?  If not try looking for the process again, have a look at /var/admn/messages and maybe post the output of ps -ef for me to look at
It sounds like someone may have disabled the GUI login on the Solaris system, which would also disable xdm. Is that the case (check the console)?
westcloxAuthor Commented:
Hi! liddler,
Unfortunately, I cannot send you the ps output, as I will be violating security rules.  Sorry! about that. If it was my system, I would not have hesitated ata ll.

According to "jlevie", somebody has definitely doene something whih restricts me to access through Reflection X.

This is a headless system - means no console, no mouse no keyboard. That is why I rely on Eflection X to get to the GUI. Just Sunfire V120 spiining hard drives. I cannot even see console messages.

I did try ./S99dtlogin start

Took the command allright and back to # prompt.

But no output, nor any messages logged not any change in the process table. I do know there is upposed to be X Windows specific process information plus "openwindow " specific entries in the ps -ef display. None of that is there.

I am assuming that something has chnaged before that and S99dtlogin behavior is completely ignored.

I was able to access it 2-weeks ago from the iste directly while on their own LAN. It was instant and no issues. i was also able to connect instantaneously through VPN using Reflection X.

I am thinking the port used by Reflecion X may be blocked.

./S99dtlogin does not seem to do anything at all.

Any more ideas......will be a major help for me. Please!

please post result of:
   ps -ef | egrep -i 'open|dm|dt'

(remove what should not be public as you like)
westcloxAuthor Commented:
johndoe@BIMBO{5}  ps -ef | egrep -i 'open|dm|dt'
    root   244   239  0 15:51:41 ?        0:00 /usr/sadm/lib/smc/bin/smcboot
    root   642     1  0 15:52:41 ?        0:00 /usr/lib/sendmail -bd -q15m
    root   239     1  0 15:51:41 ?        0:00 /usr/sadm/lib/smc/bin/smcboot
    root   281     1  0 15:51:42 ?        0:00 /usr/dt/bin/dtlogin -daemon
    root   506     1  0 15:51:44 ?        0:00 /usr/lib/dmi/dmispd
    johndoe  1417   646  0 17:38:06 pts/1    0:00 egrep -i open|dm|dt

Note: I have edited hostname output for the purpose of security. The above is a complet output listing as requested.

sounds good. If there is also a X (or Xsun) running, then I'd continue with using snoop (probably on port 6000, 6001, etc.).
Are there any syslog messages (/var/adm/messages)?

set -x
as the second line to the S99dtlogin script then save and  run it again and post the output
Sorry missed that output.

Looks like it is being blocked by a firewall ,acl or similar
firewall .. that's what I'm assuming too (hence snoop:)
westcloxAuthor Commented:
I did add set -x as second line within S99dtlogin. here is the outputs:

# ./S99dtlogin start
+ [ ! -d /usr/bin ]
+ /usr/bin/id
+ set uid=0(root) gid=1(other)
+ [ uid=0(root) != uid=0(root) ]
+ update_printers
+ [ -x /usr/dt/bin/dtprintinfo ]
+ [ -x /usr/dt/bin/dtlogin ]
+ /usr/dt/bin/dtprintinfo -populate
+ /usr/dt/bin/dtlogin -daemon
+ exit 0
johndoe@BIMBO{11} ps -ef
    root     0     0  0 15:51:24 ?        0:13 sched
    root     1     0  0 15:51:24 ?        0:00 /etc/init -
    root     2     0  0 15:51:24 ?        0:00 pageout
    root     3     0  0 15:51:24 ?        1:55 fsflush
    root   601     1  0 15:51:46 ?        0:00 /usr/lib/saf/sac -t 300
    root   244   239  0 15:51:41 ?        0:00 /usr/sadm/lib/smc/bin/smcboot
    root   642     1  0 15:52:41 ?        0:00 /usr/lib/sendmail -bd -q15m
    root   114     1  0 15:51:38 ?        0:00 /usr/sbin/aspppd -d 1
    root    61     1  0 15:51:36 ?        0:00 /usr/lib/sysevent/syseventd
    root    70     1  0 15:51:36 ?        0:54 /usr/lib/picl/picld
    root   141     1  0 15:51:39 ?        0:00 /usr/sbin/rpcbind
    root   207     1  0 15:51:40 ?        0:00 /usr/sbin/nscd
    root   199     1  0 15:51:40 ?        0:00 /usr/sbin/cron
    root   212     1  0 15:51:40 ?        0:00 /usr/lib/lpsched
    root   178     1  0 15:51:39 ?        0:00 /usr/lib/nfs/lockd
    root   191     1  0 15:51:39 ?        0:00 /usr/sbin/syslogd
    root   172     1  0 15:51:39 ?        0:00 /usr/sbin/inetd -s  
    daemon 181     1  0 15:51:39 ?        0:00 /usr/lib/nfs/statd
    root   237     1  0 15:51:41 ?        0:00 /usr/sbin/vold
    root   229     1  0 15:51:41 ?        0:00 /usr/lib/utmpd
    root   239     1  0 15:51:41 ?        0:00 /usr/sadm/lib/smc/bin/smcboot
    root   255     1  0 15:51:41 ?        0:00 /usr/sbin/in.rarpd -a
    root   257     1  0 15:51:41 ?        0:00 /usr/sbin/rpc.bootparamd
    root   281     1  0 15:51:42 ?        0:00 /usr/dt/bin/dtlogin -daemon
    root   506     1  0 15:51:44 ?        0:00 /usr/lib/dmi/dmispd
    root  5702   172  0 05:26:44 ?        0:00 in.telnetd
    root  5913  5704  0 05:32:36 pts/1    0:00 ps -ef
    root   609   601  0 15:51:46 ?        0:00 /usr/lib/saf/ttymon
    root   611   609  0 15:51:46 ?        0:00 /usr/lib/saf/ttymon
    root   799   595  0                   0:00 <defunct>

the /var/adm/messages file was not updated at all after manually running S99dtlogin script.
Please Note:  I have deleted those lines that are specific to the customer's application environment.


westcloxAuthor Commented:

Assuming the above looks fine.......
I did download a complete list of Port Number used by WRQ applications and specifically for Reflection-X to give me some idea of the actual port used. Yes, it is port 6000 as "liddler" suggested in his initial response.  Based on his suggestion, telnet to that specific port fails immediately.

I have asked the network support group to confirm if this port is blocked.

westcloxAuthor Commented:
More Info....

Guys! Regretfully,
It is none of the above....................
I fired up my Windows 2003 System and my own Sparc Server which has Solaris 8 installed. I can connect with my own system through Reflection X installed on Win2K3. I then installed VPN cluient on Win2K3 and attempt to connect to customer's Solaris system. It connected immediately and I was able to load the desktop and work with the GUI.

So there is a compelling evidence that both of my WinXP systems has inherited something that is inhibitting the connections using XDMCP, through Reflection X.

I even uninsatlled Reflection Package completely on one of XP system, and cleaned the entire registry and respective directory hierarchies. Re-installed the reflection package again. The exact same symptom persists. XDMCP Timeout through Reflection X but regular Telent session works.

My feelings are that after installing SP2 for WinXP; I am wondering it has hosed something.
Wi2K3 system finds the XDMCP server instantaneously, local or remote through VPN.

One of the test I can do is re-install XP with SP1 and see if it still works. Then test it again with SP2. As I said I was using Reflection X successfully before. It is quite possibel that since I applied SP2 4-6 weeks ago, I have not used X session.

> One of the test I can do is re-install XP with SP1
it's a pain, but that's what M$ allways recommends if something went wrong:
  1. stop then restart application
  2. if 1. fails, reboot system and continue with 1.
  3. if 2. fails, re-install application and continue with 1.
  4. if 3. fails, re-install M$ and continue with 1.
  5. if 4. fails, get new hardware and continue with 1.

so I learned in this question my lession again: if somthing goes wrong in communication between M$ and Unix, see 1. to 5. above (I'm at step 6. now, since years: using reliable software;-)
westcloxAuthor Commented:
Additional Update......as how did I go about fixing the issue. I did three seperate things and am not sure what fixed it.

1.  Remove Reflection software and clean registry.
2.  Uninstall McAfee Virus Professional
3.  Run WsockXPFix to clean the TP stack
4.  Clean Registry
5.  Re-install Reflection Software
6.  WRQ site www.wrq.com has a tip on how SP2 will affect Reflection. It isd all to do with firewall settings and ports allocations. In fact they provide the exact NETSH command syntax to use in order to resolve the issues,
7.  Re-installed McAfee software

Everything works fine. I can access local as well remote Unix ssytems instantaneously. The conclusuion may be that SP2 did something voodoo which took me round the bend.

westclox, did my 4. suggestion not cover all your 1. to 7.?

Anyway, XP's SP2 is know to make trouble with third party firewalls, probably that's the reason
westcloxAuthor Commented:
U R Funny - ahoffman.

I'll remmebr that.

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.