Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 843
  • Last Modified:

Win32 console output

I'm trying to write a simple console output app under Windows XP and having no luck. The WriteConsole procedure wants a handle to the console screen buffer but I can't seem to get one. CreateConsoleScreenBuffer just gives an invalid handle error when I run it (with parameters GENERIC_WRITE, 0, NULL, CONSOLE_TEXTMODE_BUFFER, NULL) so I tried GetStdHandle with the STD_OUTPUT_HANDLE parameter, which doesn't return an error but doesn't give me a handle either. The following is the section of code (compiled with MASM 6.14 and linked with MS incremental linker 5.12). Oh and it's the first thing to run.

; get stdout handle
call GetStdHandle
; eax is now 0

Anybody know why this is happening/how to do it properly?
  • 2
1 Solution
boycyAuthor Commented:
Ok, bit more info. Reading through the documentation for GetStdHandle again (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/getstdhandle.asp) I think the problem is that NULL is being returned because my program doesn't have any associated standard handles. So...how do I get my program to have standard handles? I remember reading that under windows, a bit in the PE header states whether the app is console or windowed...but how can I change this from the assembly source?
boycyAuthor Commented:
Right, managed to answer it myself - finally!
For those interested, the first thing that 'worked' was calling AllocConsole before attempting to retrieve the stdout handle, the only problem was that opened up a new command prompt and closed it as soon as ExitProcess was called.
The bit that did it for me was passing /subsystem:console instead of /subsystem:windows to the linker :D
Question answered by asker or dialog deemed valuable.
Closed, 100 points refunded.
ee_ai_construct (replacement part #xm34)
Community Support Admin

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now