Link to home
Start Free TrialLog in
Avatar of etsellinc
etsellincFlag for United States of America

asked on

ShellExecute Issue within VB6 on Windows 8

We have the following function in VB6..

Public Function OpenWithDefault(ByVal FileName As String) As Boolean

   'ShellExecute returns a value greater than 32 if it was successful
    OpenWithDefault = (ShellExecute(0&, "open", FileName, vbNullString, vbNullString, SW_SHOWNORMAL) > 32)

End Function

Basically we are forcing the opening of a report in a pdf after the user runs a certain process.

In windows XP this code works fine and the report pops up fine in the PDF. HOWEVER, in Windows 8, when the  user runs the process, the report does not come up, All we see is like a flash...

Any ideas on what in Windows 8 is stopping this??
Avatar of Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger)
Flag of Canada image

What happens if you double-click on a pdf file in Windows Explorer on that computer?

This is basically what ShellExecute does. It starts the program registered for the extension of the file. If none is installed, nothing happens.

What is the value in OpenWithDefault after the call?
Avatar of etsellinc

ASKER

When I double click a pdf file it opens right up
And what is the value of OpenWithDefault after the call?

Have you tried with another .PDF viewer? Since this is an external application, the problem can come from an incompatibility between the viewer and Windows 8.

Have you tried running your application in Windows XP compatibility mode? You can set that by right clicking on the .exe file in Windows Explorer, through a Compatibility tab that appears under Properties.

You need to be aware that although Microsoft still supports the runtime of VB6 in Windows 8, it stopped supporting the Visual Basic 6 development environment in 2008, so there might be problems if you develop/compile VB6 applications while working in Windows 8 (http://msdn.microsoft.com/en-us/vstudio/ms788708.aspx).

Is it possible that you were running in 32-bit under XP and 64-bit Under Windows 8? Then you enter another realm. Although 32-bit applications run in an emulation mode for 32-bit on 64-bit computers, there are issues with some programs.
Yes the new computers are 64bit Windows 8  Machines. The XP PCs are all 32bit of course..

I have read online to try this:
replace the "open" with NULL in the command line.

Im going to try that and see if it goes.
Make your changes and compile on one of the old computers. As stated, the Visual Basic development environment is not supported on either Windows 8 or 64-bits and can cause problems when you compile in these.
I doubt its VB6. We have .net of course and we are going to try the same command in it and test it as well to eliminate that its VB6...

I of course blame Windows 8...LOL
Why put VB6 in your question if you are in .NET?

If you are in .NET, do not use ShellExecute. .NET uses the Process class to do the same thing. You would not even need your method with Process, since it already returns a Boolean telling you if the start was a success:

If Process.Start(FileName) Then

If that does not exactly suit your needs, look it up in the documentation. The Start method has 4 other overloads that enable you to pass parameters if needed.
We have an entire application written in VB6. I was just trying to eliminate that it was VB6..Its probably a permission issue on Windows 8...8 is very bad about security these days..
I completely agree with the overdone security. We are to a point where it is like if we had a policeman in front of our house that asks for our ID every time we show at the door.

I would not be so fast putting everything on security however. Calling ShellExecute is the same thing as a double click on the file and thus executes in the same security context. So the issue is probably somewhere else.

Unfortunately, I do not have VB6 installed in my actual environment and cannot make any test.
thats ok, we are going to try the replace the "open" with NULL in the command line and see if that works. Ill let you know if it does...will probably be tomorrow or the next day. We have other pressing issues to deal with first.
ASKER CERTIFIED SOLUTION
Avatar of etsellinc
etsellinc
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Why do you always blame Windows 8? This is useless bashing on an OS that is not perfect but is as solid if not more than XP.

Windows 8 (not to be confused with Windows RT that is new interface and is not Windows 8 although it is presented as such almost everywhere) is Windows 7 with a few more features and new interfaces for some of the old stuff such as the Task Manager. Contrary to Vista, Windows 7 proved to be a very useful and solid Platform, and Windows 8 follows in that step.

Windows 8 has the same file association system as Windows XP. Right click on any file in Windows Explorer, select Open With, and you have the possibility to select the application that will be default for that type of file. Doing so creates the file association.

You should normally not have to do that however when only one application handles a given type of file on a computer.

Problems such as yours come from applications that installed themselves through undocumented features of older operating systems to register their file association, or do not record them at all. Many applications were rushed out to make use of the touch features of Windows 8 and have not been properly tested. And this happens to "big" applications, not only to small stuff done by independant software developers. (Acrobat Reader is one of these. Its Windows RT version is featureless and they broke a lot of useful feature in their touch version for Windows 8 compared to previous ones.)

You can thus end up with an application that installs without registering the association properly. In such a case you need to do it manually.

This is probably what happens with the pdf viewer that you have on your Windows 8 computers. It did not record Open properly. When you call ShellExecute with an empty string, you are calling the default verb, which ends up being Open.

Its not Windows 8 fault, its the pdf viewer fault.
ok, so its Adobe's fault on this one..BUT why it didnt set the open in the association is beyond me. I bet Adobe would blame Microsoft and Microsoft would blame them. Also, I prefer Windows 8 over XP, but there are things in it that have changed that makes no sense, but either way our problem is fixed now..

Also we have 4 Win 8 PCs that we installed Adobe Reader on and all 4 had the SAME problem with the open not being set. So either Adobe or Microsoft has a problem they need to fix!!
Adobe has the problem as they often have. Sometimes they seem to think that they are more important than the operating system. They do not follow the rules.

I have installed many applications on Windows 8, even some that are not supposed to be supported on Windows 8, and they create their file associations when needed.

I have some of my own application that do file associations, they have for years, and they installed perfectly and without any change in Windows 8. I do nothing fancy.

I simply follow the rules.
good deal..Yeah, its seem like such a minor thing...
worked fine