How can I block optimizations made be DJGPP?

Howdid ou turn on optimization? And what exactly do you mean with "the code doesn't work anymore" Do you get wrong results from the reported function or in some other place?
Maybe this function continues to work properly but fat_bus is broken.

What version of gcc are you using ?

use gcc -S  -o fat.s fat.c

to generate assembler code.  Verify that is where problem is occuring.

Why use nasm ?

Pls disregard my last remark

The thing I do is an own operating system for learning purposes. I can't compile the fat.c on my own, it must be within the makefile statements. All the kernel files are linked together. I use NASM because it's easier to code an own OS with startup code in assembler :-) The strange output is unbelievable and I don't know where it come from. But uncommenting the printf-lines and all the code works very well. Is there something wrong in my code or in the compiling setting or is there maybe an error in the declaration?

Bye, TDS.

sometimes stack overruns occur and debug / conditionally compiled code can hide the effects.

fat.c returns 0x00 is that a dword ?

you have multiple return points which is speed over style.

At the end of the day compilers are software and contain bugs like any other software.

Nuclear power stations and aircraft engines must have code verified at machine code level.

as earthamn already suggested, use "gcc -S  -o fat.s fat.c" both with and without the printf and compare the code generated in the two cases. However it's possible that the origin of the problem is somewhere else (stack, fat_buf,...)

Hmm, it can't be the stack. We have defined a stack of 1MB! That should be enough. Maybe I should search for type conversions or something else. The "-S" option I will check today.

I have, on occasion, blamed the compiler for such issues. It has *always* been my fault. Doing an optimized build makes a big difference in how the program will run. The code sizes are all different, the stack is used differently. For example:

void RunBug(int b)
{
    char *p;

    if (b)
        p = &something;

    if (*p == 2)
       DoSomething();
}

In the above code, you'll notice that sometimes p has a garbage value in it (when b is zero). If that garbage value just HAPPENED to be a valid memory address, I might not notice it. (Due to coincidence, the stack just HAPPENED to contain a value in that location that was a valid address.)

Now I switch over to an optimized build. It is more efficient and doesn't use the stack as much. So the stack pointer will be in a different position in this function, which may no longer HAPPEN to have a valid pointer where p is located.

Make sure you initialize EVERY variable in a function. Don't worry about "efficiency", storing zero to a variable is almost free. Especially, initialize EVERY pointer to zero before the first real line of code in the function. It will help you work out the bugs.

I don't see optimizations enabled anywhere in your makefile. Did I miss it? Should be -O or -O1 -O2, etc.

-O is completely safe (with valid code).
-O2 is aggressive
-O3 is maximum.

Since you are writing the operating system itself here, you can't trust printf to work. Who wrote it? Are you linking it in from some library? Deep inside printf it uses file handles to write out it's data - did you implement the file handle code? stdout is file 1, if you know what I mean....

printf also allocates memory. So it will use malloc, and malloc will go to the operating system for actual RAM. How does malloc actually get RAM from the operating system. If you don't know, that is probably your problem(s).

If that code (printf/malloc/etc) was written for some other existing environment it might be doing bad things to your program.

In this sort of environment, you have to write "freestanding" code. That means you can't just pull in whatever is convenient. For example, malloc uses something in the operating system to allocate physical pages, someone has to write the code to give memory to malloc. Same goes for printf. printf hands off the actual "printing" to some other code, deep within the operating system, by using write calls on the stdout handle.

We have already a working memory code with malloc and free. I works very well tested with several functions. The printf function also works very well with and without debugging.
But how can it be that after the two printf's all thing are working perfect? I also see that no optimizations are made.

What calling sequence are you using. Is it possible you are mixing stdcall/cdecl/fastcall calling conventions (with __attribute__ declarations)? This will misalign the stack resulting in a definite crash.

I see you already have maximum warnings enabled. Rebuild the project, and post the warnings you receive here.

When you said "with optimizations" in your original question, you must have meant "not DEBUG" - ie. you removed "-DDEBUG" to get the problem. Is that right? If so, see if you can find what's different when DEBUG is defined.

Ah, nevermind that calling sequence thing... you said printf working fine...

Did you use assembly for something in printf? Maybe you are clobbering ebx, esi, edi, or ebp somewhere? That might hardly show up, and when it does, as really weird bugs.

It might help to compile that source file with -S (as already suggested by ibertacco), with and without the printf calls, and post the assembly for get_fat_entry here. I really really doubt gcc messed up though...

Okay, I've no DEBUG's in the fat.c code. All calls are normal functions calls in this code and data segment. There should be no collision.

My stdio.c:

#include <stdio.h>
#include <datatypes.h>
#include <stdarg.h>
#include <string.h>
#include <drivers/video.h>
#include <drivers/input.h>
#include <nucleus.h>

void gotoxy(word x, word y)
{
      video_set_cursor_pos(x, y);
}

void printc ( char c )
{
      word w1, w2;

      w1 = (video_get_cursor_col() << 8 | video_get_cursor_row());

      if ( c == '\r' )
      {
            w2 = video_get_cursor_row() ;
            gotoxy (0, w2) ;
            return ;
      }
      if ( c == '\n' )
      {
            if(nucleuskl_debug == 0) return;

            if(video_get_cursor_row() == 24)
            {
                  video_scrolldown(1);
            }
            else
            {
                  video_set_cursor_row(video_get_cursor_row() + 1);
                  video_set_cursor_col(0);
            }
            return ;
      }

      if ( c == '\r')
      {
            if(nucleuskl_debug > 0)
                  video_set_cursor_col(0);
            return;
      }
      video_putch(c);
}

void prints ( char* s )
{
      int i1 ;
      i1 = 0 ;
      while ( s[i1] != 0 )
      {
            printc ( s[i1] ) ;
            i1 += 1 ;
      }
}

#define S_SIZE  160

void newline(void)
{
      prints("\n\r");
}

long _formatout(char *outptr, char *fmt, char output, va_list argptr)
{
      char numstk[33], *ptr, justify, zero, minus, chr[2], us, chr2[2];
      unsigned long width, value, i, total, counter;
      int x;

      chr[1] = '\0';      x = 1;
      chr2[1] = '\0';      total = 0;
      while ((chr[0] = *fmt++) != 0)
      {
            if (chr[0] == '%')
            {                        /* format code */
                  chr[0] = *fmt++;
                  ptr = &numstk[32];
                  *ptr = justify = minus = 0;
                  width = value = i = 0;
                  zero = ' ';
                  if (chr[0] == '-')
                  {                  /* left justify */
                        --justify;
                        chr[0] = *fmt++;
                  }
                  if (chr[0] == '0')      /* leading zeros */
                        zero = '0';
                  while (chr[0] >= '0' && chr[0] <= '9')
                  {                  /* field width specifier */
                        width = (width * 10) + (chr[0] - '0');
                        chr[0] = *fmt++;
                  }
                  /* first switch allows for ld Ld ld ud Ud etc... */
                  switch(chr[0])
                  {
                        case 'U' :                                      /* unsigned number */
                        case 'u' :
                              i = 10;
                              us = 1;
                              chr[0] = *fmt++;
                              break;
                        case 'l' :                                      /* Long (it is anyway) */
                        case 'L' :
                              chr[0] = *fmt++;
                              break;
                        default:                                        /* all others */
                              us = 0;                                 /* not unsigned */
                              break;
                  }

                  switch(chr[0])
                  {
                        case 'd' :                                      /* decimal number */
                              value = va_arg(argptr, int);      /* get parameter value */
                              i = 10;
                              if (!us)
                                    if(value & 0x8000000)
                                    {
                                          value = -value;
                                          ++minus;
                                    }
                              break;
                        case 'X' :                                      /* hexidecimal number */
                        case 'x' :                                      /* hexidecimal number */
                              value = va_arg(argptr, long);
                              i = 16;
                              break;
                        case 'o' :                                      /* octal number */
                              value = va_arg(argptr, int);
                              i = 8;
                              break;
                        case 'c' :                                      /* character data */
                              value = (char)va_arg(argptr, int);      /* get parameter value */
                              *--ptr = value;
                              break;
                        case 's':
                                   ptr = va_arg(argptr, char *);
                              break;
                  }

                  if(i)      /* for all numbers, generate the ASCII string */
                  {
                        do {
                              if((chr[0] = (value % i) + '0') > '9')
                                    chr[0] += 7;
                              *--ptr = chr[0]; }
                        while(value /= i);
                  }
                  /* output sign if any */

                  if(minus)
                  {
                        chr2[0]='-';
                        *outptr++ = '-';
                        if (output)
                              prints(chr2);
                        ++total;
                        if(width)
                              --width;
                  }

                  /* pad with 'zero' value if right justify enabled  */

                  if(width && !justify)
                  {
                        chr2[0]=zero;
                        for(i = strlen(ptr); i < width; ++i)
                        {
                              if (output)
                                    prints(chr2);
                              *outptr++ = zero;
                        }
                        ++total;
                  }

                  /* move in data */

                  i = 0;
                  value = width - 1;

                  while((*ptr) && (i <= value))
                  {
                        chr2[0]=*ptr;
                        if (chr2[0] == '\n')
                        {
                              newline();
                        }
                        else
                        if(chr2[0] == '\t')
                        {
                              chr2[0]=' ';
                              if (output)
                              for(counter = 1;counter<=5;counter++)
                                    prints(chr2);
                        }
                        else
                        if (output)
                              prints(chr2);
                        *outptr++ = *ptr++;
                        ++total;
                        ++i;
                  }

                  /* pad with 'zero' value if left justify enabled */

                  if(width && justify)
                  {
                        while(i < width)
                        {
                              chr2[0]=zero;
                              if (output)
                                    prints(chr2);
                              *outptr++ = zero;
                              ++total;
                              ++i;
                        }
                  }
            }
            else
            if (chr[0] == '\n')
            {
                  if (output)
                        newline();
            }      
            else
            if (chr[0] == '\t')
            {
                  chr[0]=' ';
                  if (output)
                  for(counter = 1;counter<=5;counter++)
                          prints(chr);
            }
            else
            {
                  /* not format char, just move into string  */
                  *outptr++ = chr[0];
                  if (output)
                        prints(chr);
                  ++total;
            }
      }

      *outptr = 0;
      return total;
}

/************************************
      Formatted print to stdout
*************************************/
int printf(char *fmt, ...)
{
      va_list ap;
      long total;
      char buffer[S_SIZE];

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, fmt, 1, ap);
      
      va_end(ap);
      return total;
}

int dprintf(char *fmt, ...)
{
#ifdef DEBUG
      va_list ap;
      long total;
      char buffer[S_SIZE];

      if (nucleuskl_debug == 0) return 0;

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, fmt, 1, ap);
      
      va_end(ap);
      return total;
#else
      return 0;
#endif
}

int sprintf(char* str, const char *fmt, ...)
{
      va_list ap;
      long total;
      char buffer[S_SIZE];

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, (char *)fmt, 0, ap);
      strcpy(str, buffer);
      
      va_end(ap);
      return total;
}

int scanf(const char *format, ...)
{
      va_list ap;
      char *result, input[120];
      int i = 0, j = 0;

      va_start(ap, format);

      while(!(format[i] == '\0'))
      {
            switch(format[i++])
            {
            case '%':
                  switch(format[i++])
                  {
                  case 's':
                        while(j < 120)
                        {
                              input[j++] = input_getch();
                              if(input[j - 1] == 0x08)
                              {
                                    if(j == 1)
                                    {
                                          j = 0;
                                    }
                                    else
                                    {
                                          j -= 2;
                                          video_set_cursor_col(video_get_cursor_col() - 1);
                                          printf(" ");
                                          video_set_cursor_col(video_get_cursor_col() - 1);
                                    }
                              }
                              else if(input[j - 1] == 0x0A)
                              {
                                    input[--j] = '\0';
                                    break;
                              }
                              else printf("%c", input[j - 1]);
                        }
                        j = 0;
                        result = va_arg(ap, char *);
                        
                        strcpy(result, input);
                        break;

                  default:
                        break;
                  }
                  break;

            default:
                  video_putch(format[i] - 1);
            }
      }

      va_end(ap);

      return 1;
}

As you will see there is no assembly... If I want to compile fat.c with -S options I get errors because it uses functions from other files. So I can't use it.

I notice printf uses a fixed size buffer of S_SIZE. What is S_SIZE. Are you overrunning it with a large message?

For every place where you use _formatout, do this:

if (total >= S_SIZE) {
    panic();
}

where panic is your "that was really bad" function. :)

If you have no panic, just do "*(char*)0 = 0;" to catch these extremely fatal overruns.

Consider implementing _snprintf:

int _snprintf(char *buf, size_t bufsize, char *format, ...)

where it returns count of output characters excluding null terminator, always terminates output string, returns -1 if it would overflow and truncates string at max buf size (terminated).

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore98/HTML/_crt__snprintf.2c_._snwprintf.asp

Notice that in the middle of the code:

#define S_SIZE  160

An the var "total" is set up with "_formatout":

total = _formatout(buffer, fmt, 1, ap);

So I can't use the panic statement.

Where should I past the "*(char*)0 = 0;" statement. Maybe here?
int printf(char *fmt, ...)
{
      va_list ap;
      long total;
      char buffer[S_SIZE];

      va_start(ap, fmt);              /* set up ap pointer */
      
      *(char*)0 = 0;
      total = _formatout(buffer, fmt, 1, ap);
      
      va_end(ap);
      return total;
}

Can you explain me the advantages of _snprintf ?

It's also ugly that if I use printf in the get_fat_entry function all things work fine, if not I have 0-values.

I'm saying that if S_SIZE is 160 bytes, then any printf that generates more than 160 bytes will spill over and corrupt other variables located after buffer. The advantage of _snprintf is it won't allow a buffer overflow, you can set a limit to the output so it won't spill over the buffer and corrupt memory. This overrun might happen in some other area of the program entirely, and cause a problem later in somewhere seemingly unrelated.

Consider this:

#define panic() dopanic(__FILE__, __LINE__)

void dopanic(char *pFile, int nLine)
{
    clear_the_screen_somehow();
    printf("panic! %s(%u)", pFile, nLine);

    // freeze the program here. This never returns
    while (1) {
    }
}

int printf(char *fmt, ...)
{
     va_list ap;
     long total;
     char buffer[S_SIZE];

     va_start(ap, fmt);              /* set up ap pointer */
     
     total = _formatout(buffer, fmt, 1, ap);

     if (total >= S_SIZE)          
          panic();    // I just overwrote memory, die here, where it happened
     
     va_end(ap);
     return total;
}

You'd use the panic macro in places in the program where something really bad happened and you can't trust program execution to continue. In this case, an overrun of buffer will cause stack corruption. printf can no longer be trusted to return. It may have had registers saved on the stack that will not be restored correctly.

See what I mean? In memory, this is how the local variables are laid out:

[buffer]
[total]
[ap]
[saved registers]
return address
[parameters]

If you overrun buffer, you will spill over and corrupt the "total", "ap", and finally, the really bad part, you will corrupt registers saved on the stack. If you overrun a bit further you will destroy the return address, and printf will return to some invalid address and control is lost.

My suggestion is to catch the overrun (after each call to _formatout). When the panic() macro is executed, it will pass the source file name and line number to dopanic. dopanic clears the screen shows the panic location and freezes the program.

If you had _snprintf, you could block the overrun from occuring at all, it would simply truncate the string at 159 characters, saving one for null terminator.

Hmm, it's difficult to understand why this thing should happen if the printf function isn't called by get_fat_entry. All seems to work. I inserted the panic code, but nothing happens. All things are working pretty well.

Here's a rule of thumb I keep in my head when working with local variables - a way to mentally picture buffer overflows on the stack:

 - as you fill forward in memory, you are going backwards in source lines. If you overflow a buffer, you overwrite the variables declared before the buffer (before as in in-the-source, not memory). if you overwrite further - into the opening brace of the function, you destroyed saved registers. If you overwrite all the way up to the function name, you overwrote the return address. Overwriting the return address is completely fatal. You can't return. When you do, it's to some invalid address. Overwriting just saved registers will cause very very weird problems because stuff that seems impossible can happen.

I suspect that slight overflows are corrupting the stack somewhere else (maybe another source file entirely), causing weird stuff.

You mean I shouldn't call so many procs and funcs from main()? Should it be a problem that I call a function on 5th position?

main -> block_init -> fat_init -> fat_read -> get_fat

You put...

     if (total >= S_SIZE)          
          panic();

...in dprintf and printf, right?

Okay, I've tested it -> no success.
I screw up now, it's unbelievable.

int printf(char *fmt, ...)
{
      va_list ap;
      long total;
      char buffer[S_SIZE];

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, fmt, 1, ap);
      
    if (total >= S_SIZE)          
          panic();    // I just overwrote memory, die here, where it happened
    va_end(ap);
         
      return total;
}

int dprintf(char *fmt, ...)
{
#ifdef DEBUG
      va_list ap;
      long total;
      char buffer[S_SIZE];

      if (nucleuskl_debug == 0) return 0;

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, fmt, 1, ap);
    if (total >= S_SIZE)          
          panic();    // I just overwrote memory, die here, where it happened
      
      va_end(ap);
      return total;
#else
      return 0;
#endif
}

BUT IS IT REALLY THE PRINTF FUNCTION?

Maybe not. It could be lots of things. It just looked like it was something global that had potential to overrun when called from somewhere in the program. Some day, you are going to call printf with more than 160 bytes to print, and the panic will catch that before the program crashes.

That might be right. I looked into severyl printf source but I couldn't find anything else that could make troubles.

How do you boot it right now? You are using a custom linker script (nucleus.ld). Can you show it? Also, did you write the loader? Is it clearing uninitialized memory?

I was doing a project very much like yours (freestanding kernel written with gcc). I ran into a bug in ld that caused it to miscalculate addresses because of a custom linker script.

I had a problem in an earlier project, where I attempted to use separate sections for locked code and locked data (interrupt handlers and their data). I used the __attribute__((section, ...))  declarations and a custom linker script, but it miscalculated the address of some stuff. Would have been cool if it worked because I could lock all data touched in interrupt handlers in one nice linear block of memory.

That was a long time ago, on DJGPP though. I would assume they fixed the linker script bugs by now.

I wanna answer all your questions but look at the code first. There are difficult questions about each load oder run process.

http://www.tds-net.de/nucleuskl.rar

Sure we use a linker script, because another chance to compile we don't have :-(
I also played with __attribute__ ((packed)) option, without success. The bead thing is that I can't trace back the calling funcs in a kernel which isn't ready.

Did you notice that there are two source files with the same name? fat.c

I see... not using LFN one.

I noticed a HUGE error in header.asm. It initializes esp *misaligned*!!! That means the cpu is struggling with every stack access. Not that it would cause an actual bug, but it slows the cpu right down.. Fix this line in header.asm:

  mov esp,0x002fffff

It should be

  mov esp,0x00300000

Before you jump and say I am wrong, consider this: The cpu subtracts 4 from esp BEFORE writing a value to the stack. So if it starts at 0x300000 as I suggest, then the first push will be to 0x002ffffc. Since the push is 4 bytes, the first push will be to

0x002ffffc
0x002ffffd
0x002ffffe
0x002fffff

I am looking into a technique using gcc's function instrumentation capbility, to add a stack overflow check at every function entry. See the gcc docs for the -finstrument-functions switch.

Ok, I have completed the changes necessary to automatically add stack checking to every function.

Add the following lines in the appropriate places in support.asm:

----

  global ___cyg_profile_func_enter
  global ___cyg_profile_func_exit

  extern _dopanic

----

And add this near the bottom, before "section .data"

----

; Stack checking code. (Same function used on func entry AND exit)
; Stack pointer must be between 0x00300000 and 0x00200000
; This is only used when you compile with -finstrument-functions
; Raises panic if stack pointer is out-of-bounds
; Completely transparent. Even preserves flags.
___cyg_profile_func_enter:
      nop              ; This is so disassembly shows proper different symbol for entry/exit
___cyg_profile_func_exit:
      pushfd
      cmp esp,0x00200000
      jbe _stack_overflowed
      cmp esp,0x00300000
      ja _stack_overflowed
      popfd
      ret

_stackoverflowmsg db 'Stack overflow!',0

_stack_overflowed:
      ; Due to stack overflow, I can't push parameters for dopanic,
      ; so flush/reset stack, dopanic freezes execution anyways
      mov esp,0x00300000
      push dword 0
      push _stackoverflowmsg
      jmp _dopanic

----

And, add change the CFLAGS in your makefile to this:

----

CFLAGS = -I include -Wall -fno-builtin -g -DDEBUG -finstrument-functions

----

OOPS! Change

jmp _dopanic

to

call _dopanic

I forgot that the called function needs the return address there to get the parameters properly

@adg: Thanx for helping me so much. You see errors I can't see because I don't know (e.g. system optimizations). But the error isn't solved. I wrote lines and I removed lines in the code without success. Where do you came from, maybe I can have your email? My is: webmaster@tds-net.de

I increased the value of points. Maybe it helps.

Wow, you are too fast for me :-)

I have not completely looked through the threading code, but I suggest you add some sort of "thread local storage". What I mean is, for each thread, there should be a special storage area for information related to that thread.

For each thread created, allocate a small structure for data related to that thread. Make a selector in the GDT for that thread's fs register (allocate a range of selectors for thread local storage). The base address of the segment should be the address of the thread-local structure. This structure could contain the stack bounds for the CURRENT thread. Say I had the following structure for each thread:

typedef struct {
    unsigned nStackBottom;
    unsigned nStackTop;

    unsigned nUserThreadData;

// etc...
} ThreadLocalData;

So in a thread, I can look at fs:[0] to see the lowest valid stack pointer address, or fs:[4] to see the highest valid stack pointer address. Because each thread can assume offset zero on fs, the base of the segment really points to the data. This way, my stack checking code can compare against fs:[0] and fs:[4] instead of the hard-coded addresses 0x00200000 and 0x00300000.

This is not necessary right now, but will be once you start using multi-threading.

Did you test the code within Bochs or so? I get only stack overflows just before the code starts...

I tried signing up at your home page, but it says some German stuff. (translates to "Log in not correctly" or something in babelfish :). I don't know German.

Okay, I've looked up into my code. It looks like the memory code works not as well as it should do.

It hangs at:

memory: Allocating user memory space (XXX entries, XXX

Kernel panic! Stack overflow! (0

Kernel panic! Stack overflow! (0

etc.

I am not *booting* the kernel, only examing the disassembly to check that stack-check code.

Should I do a build and make a boot floppy? What filename for the kernel file? Does it just look for nucleus.bin on the floppy? How do I write the boot code to the boot sector? I could look but you could easily tell me... :)

When I try to signup I get

Konnte User nicht einfügen (Portal).

Since you are actually getting stack overflows, I could quickly extend the stack check to show the address of the overflow and do a printf for a stack panic...

Put this in support.asm

Shows more detailed stack panic information

Put this in the appropriate place:

extern _printf

And update:

; Stack checking code. (Same function used on func entry AND exit)
; Stack pointer must be between 0x00300000 and 0x00200000
; This is only used when you compile with -finstrument-functions
___cyg_profile_func_enter:
      nop
___cyg_profile_func_exit:
      pushfd
      cmp esp,0x00200000
      jbe _stack_overflowed
      cmp esp,0x00300000
      ja _stack_overflowed
      popfd
      ret

_stackoverflowmsg:
db 10,10,'Stack overflow! At %08X called from %08X',10
db 'Use map file to find what these addresses mean',10
db 0

_stack_overflowed:
      ; Grab the parameters, the overwrite already occurred,
      ; but read __cyg_profile_func_XXXX parameters anyways
      mov eax,[esp+8]
      mov ecx,[esp+12]
      ; Due to stack overflow, I can't push parameters for dopanic,
      ; so flush/reset stack, dopanic freezes execution anyways
      mov esp,0x00300000
      push ecx
      push eax
      push _stackoverflowmsg
      call _printf
.hang
      hlt
      jmp .hang

This is better.

Put the strings and instructions in the order shown! (short jumps):

; Stack checking code. (Same function used on func entry AND exit)
; Stack pointer must be between 0x00300000 and 0x00200000
; This is only used when you compile with -finstrument-functions

_stackoverflowmsg:
db 10,10,'Kernel panic! Stack overflow at %08X '
db 'called from %08X. '
db 'ESP was %08X',10
db 'Use map file to find what these addresses mean',10
db 0

___cyg_profile_func_enter:
      nop
___cyg_profile_func_exit:
      pushfd
      cmp esp,0x00200000
      jbe _stack_overflowed
      cmp esp,0x00300000
      ja _stack_overflowed
      popfd
      ret

_stack_overflowed:
      ; Grab the parameters, the overwrite already occurred,
      ; but read __cyg_profile_func_XXXX parameters anyways
      mov eax,[esp+8]
      mov ecx,[esp+12]
      lea edx,[esp+16]
      ; Due to stack overflow, I can't push parameters for dopanic,
      ; so flush/reset stack, I freeze execution anyways
      mov esp,0x00300000
      push edx
      push ecx
      push eax
      push _stackoverflowmsg
      call _printf
.hang
      hlt
      jmp .hang

You're not starting threads yet right? It is supposed to be always using the kernel stack so far, right?

I figured out how to build boot image myself... It's writing the image now. Will try actual run.

Okay, to have you informed about current things - here is a short summary:

booting kernel
init memory and co.
starting multitasking
init two threads in nucleus.c
starting cdsl.c (shell)

I'll send you the whole code files. You only have to use m.bat in the temp directory to compile everything (I hope) :-)

Yes, I noticed that it is going straight into multitasking. That requires the stack check to be thread-aware. There is no way to do thread-local-storage right now. When you add that, you can make the stack check use the stack bounds of the current thread.

I've tried to get somewhere on sourceforge but nobody is answering. Where the heck do I go from here https://sourceforge.net/projects/nucleuskl/

My name on sourceforge is doug65536

I checked out the newest revision of nucleus from sourceforge. Several differences between the one you sent and the one I checked out. How old is the CVS copy?

The CVS copy is still older than the current .rar I sent you. I think about 5 months or so.

I accepted you answer cause you pointed me the right way.