Solved

How can I block optimizations made be DJGPP?

Posted on 2004-08-12
59
1,216 Views
Last Modified: 2013-11-13
I've been angry for many days. I can't believe that GCC (DJGPP) is such a silly compiler. I turned on optimizations.. Okay everything went well. But after I removed my debug printf's the code doesn't work anymore. It seems like GCC optimizes my code too much so some variables are wrong. Here is a small piece of my code.

The whole code: http://www.tds-net.de/fat.c

If I remove the slashes before the printf's all is fine *grml*

dword get_fat_entry(dword start, byte type)
{
      dword usl;

      switch (type)
      {            
            case FAT12:
                  usl = (start >> 1) + start;
//                  printf("%X\n", (*((unsigned short *)(fat_buf+usl))) >> 4);
//                  printf("%X\n", (*((unsigned short *)(fat_buf+usl))) & 0x0FFF);
                  if (start & 1) {  // if odd, get high 12 bits
                        return (*((unsigned short *)(fat_buf+usl))) >> 4;
                  } else {        // if even, get low 12 bits
                        return (*((unsigned short *)(fat_buf+usl))) & 0x0FFF;
                  }
            case FAT16:
                  return fat_buf[start];
            case FAT32:
                  return (*((dword *)(fat_buf+(start<<2))));
            default:
                  return 0x00;
      }
}

my makefile:

CC = gcc
LD = ld
AS = nasm

LDFLAGS = -e 0x0000 --oformat binary
ASFLAGS = -f coff
CFLAGS = -I include -Wall -fno-builtin -DDEBUG

CPUOBJ = drivers/cpu/cpu.o drivers/cpu/cpu_k6.o drivers/cpu/cpu_mhz.o drivers/cpu/bugs.o
VIDEOOBJ = drivers/video/vga.o drivers/video/text.o drivers/video/vtext.o
INPUTOBJ = drivers/input/keyboard.o drivers/input/layout.o
BLOCKOBJ = drivers/block/floppy.o drivers/block/ide.o drivers/fs/fat.o

DRIVEROBJ = drivers/pic.o \
      drivers/mem/mem.o \
      drivers/video.o \
      drivers/timer.o \
      drivers/input.o \
      drivers/block.o \
      drivers/cmos.o \
      drivers/dma.o

MAINOBJ = header.o \
      nucleus.o \
      stdio.o \
      support.o \
      logo.o \
      strings.o \
      string.o \
      irq.o \
      multi.o \
      cdsl.o \
      stss.o

OBJ = $(MAINOBJ) $(DRIVEROBJ) $(CPUOBJ) $(VIDEOOBJ) $(INPUTOBJ) $(BLOCKOBJ)

all: nucleus.bin
      @rem echo Nucleus wurde erfolgreich compiliert!

clean:
      -rm $(MAINOBJ)
      -rm $(DRIVEROBJ)
      -rm $(CPUOBJ)
      -rm $(VIDEOOBJ)
      -rm $(BLOCKOBJ)
      -rm nucleus.bin
      @rem echo Nucleus wurde geloescht!

nucleus.bin: $(OBJ)
      $(LD) $(OBJ) $(LDFLAGS) -o $@ -T nucleus.ld
      
%o: %c
      $(CC) $(CFLAGS) -c $? -o $@

%o: %asm
      $(AS) $(ASFLAGS) $? -o $@


Plz help me.

Bye, TDS.
0
Comment
Question by:Mathias
  • 33
  • 20
  • 4
  • +1
59 Comments
 
LVL 11

Expert Comment

by:lbertacco
ID: 11784376
Howdid ou turn on optimization? And what exactly do you mean with "the code doesn't work anymore" Do you get wrong results from the reported function or in some other place?
Maybe this function continues to work properly but fat_bus is broken.
0
 
LVL 22

Expert Comment

by:earth man2
ID: 11784987
What version of gcc are you using ?

use gcc -S  -o fat.s fat.c

to generate assembler code.  Verify that is where problem is occuring.
0
 
LVL 22

Expert Comment

by:earth man2
ID: 11785043
Why use nasm ?
0
 
LVL 22

Expert Comment

by:earth man2
ID: 11785065
Pls disregard my last remark
0
 
LVL 3

Author Comment

by:Mathias
ID: 11788374
The thing I do is an own operating system for learning purposes. I can't compile the fat.c on my own, it must be within the makefile statements. All the kernel files are linked together. I use NASM because it's easier to code an own OS with startup code in assembler :-) The strange output is unbelievable and I don't know where it come from. But uncommenting the printf-lines and all the code works very well. Is there something wrong in my code or in the compiling setting or is there maybe an error in the declaration?

Bye, TDS.
0
 
LVL 22

Expert Comment

by:earth man2
ID: 11788868

sometimes stack overruns occur and debug / conditionally compiled code can hide the effects.

fat.c returns 0x00 is that a dword ?

you have multiple return points which is speed over style.

At the end of the day compilers are software and contain bugs like any other software.

Nuclear power stations and aircraft engines must have code verified at machine code level.



0
 
LVL 11

Expert Comment

by:lbertacco
ID: 11790604
as earthamn already suggested, use "gcc -S  -o fat.s fat.c" both with and without the printf and compare the code generated in the two cases. However it's possible that the origin of the problem is somewhere else (stack, fat_buf,...)
0
 
LVL 3

Author Comment

by:Mathias
ID: 11790824
Hmm, it can't be the stack. We have defined a stack of 1MB! That should be enough. Maybe I should search for type conversions or something else. The "-S" option I will check today.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11798554
I have, on occasion, blamed the compiler for such issues. It has *always* been my fault. Doing an optimized build makes a big difference in how the program will run. The code sizes are all different, the stack is used differently. For example:

void RunBug(int b)
{
    char *p;

    if (b)
        p = &something;

    if (*p == 2)
       DoSomething();
}

In the above code, you'll notice that sometimes p has a garbage value in it (when b is zero). If that garbage value just HAPPENED to be a valid memory address, I might not notice it. (Due to coincidence, the stack just HAPPENED to contain a value in that location that was a valid address.)

Now I switch over to an optimized build. It is more efficient and doesn't use the stack as much. So the stack pointer will be in a different position in this function, which may no longer HAPPEN to have a valid pointer where p is located.

Make sure you initialize EVERY variable in a function. Don't worry about "efficiency", storing zero to a variable is almost free. Especially, initialize EVERY pointer to zero before the first real line of code in the function. It will help you work out the bugs.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11798586
I don't see optimizations enabled anywhere in your makefile. Did I miss it? Should be -O or -O1 -O2, etc.

-O is completely safe (with valid code).
-O2 is aggressive
-O3 is maximum.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11798613
Since you are writing the operating system itself here, you can't trust printf to work. Who wrote it? Are you linking it in from some library? Deep inside printf it uses file handles to write out it's data - did you implement the file handle code? stdout is file 1, if you know what I mean....

printf also allocates memory. So it will use malloc, and malloc will go to the operating system for actual RAM. How does malloc actually get RAM from the operating system. If you don't know, that is probably your problem(s).

If that code (printf/malloc/etc) was written for some other existing environment it might be doing bad things to your program.

In this sort of environment, you have to write "freestanding" code. That means you can't just pull in whatever is convenient. For example, malloc uses something in the operating system to allocate physical pages, someone has to write the code to give memory to malloc. Same goes for printf. printf hands off the actual "printing" to some other code, deep within the operating system, by using write calls on the stdout handle.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11798861
We have already a working memory code with malloc and free. I works very well tested with several functions. The printf function also works very well with and without debugging.
But how can it be that after the two printf's all thing are working perfect? I also see that no optimizations are made.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11798915
What calling sequence are you using. Is it possible you are mixing stdcall/cdecl/fastcall calling conventions (with __attribute__ declarations)? This will misalign the stack resulting in a definite crash.

I see you already have maximum warnings enabled. Rebuild the project, and post the warnings you receive here.

When you said "with optimizations" in your original question, you must have meant "not DEBUG" - ie. you removed "-DDEBUG" to get the problem. Is that right? If so, see if you can find what's different when DEBUG is defined.

0
 
LVL 8

Expert Comment

by:adg080898
ID: 11798920
Ah, nevermind that calling sequence thing... you said printf working fine...
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11798937
Did you use assembly for something in printf? Maybe you are clobbering ebx, esi, edi, or ebp somewhere? That might hardly show up, and when it does, as really weird bugs.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11798949
It might help to compile that source file with -S (as already suggested by ibertacco), with and without the printf calls, and post the assembly for get_fat_entry here. I really really doubt gcc messed up though...
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799003
Okay, I've no DEBUG's in the fat.c code. All calls are normal functions calls in this code and data segment. There should be no collision.

My stdio.c:

#include <stdio.h>
#include <datatypes.h>
#include <stdarg.h>
#include <string.h>
#include <drivers/video.h>
#include <drivers/input.h>
#include <nucleus.h>

void gotoxy(word x, word y)
{
      video_set_cursor_pos(x, y);
}

void printc ( char c )
{
      word w1, w2;

      w1 = (video_get_cursor_col() << 8 | video_get_cursor_row());

      if ( c == '\r' )
      {
            w2 = video_get_cursor_row() ;
            gotoxy (0, w2) ;
            return ;
      }
      if ( c == '\n' )
      {
            if(nucleuskl_debug == 0) return;

            if(video_get_cursor_row() == 24)
            {
                  video_scrolldown(1);
            }
            else
            {
                  video_set_cursor_row(video_get_cursor_row() + 1);
                  video_set_cursor_col(0);
            }
            return ;
      }

      if ( c == '\r')
      {
            if(nucleuskl_debug > 0)
                  video_set_cursor_col(0);
            return;
      }
      video_putch(c);
}

void prints ( char* s )
{
      int i1 ;
      i1 = 0 ;
      while ( s[i1] != 0 )
      {
            printc ( s[i1] ) ;
            i1 += 1 ;
      }
}

#define S_SIZE  160

void newline(void)
{
      prints("\n\r");
}

long _formatout(char *outptr, char *fmt, char output, va_list argptr)
{
      char numstk[33], *ptr, justify, zero, minus, chr[2], us, chr2[2];
      unsigned long width, value, i, total, counter;
      int x;

      chr[1] = '\0';      x = 1;
      chr2[1] = '\0';      total = 0;
      while ((chr[0] = *fmt++) != 0)
      {
            if (chr[0] == '%')
            {                        /* format code */
                  chr[0] = *fmt++;
                  ptr = &numstk[32];
                  *ptr = justify = minus = 0;
                  width = value = i = 0;
                  zero = ' ';
                  if (chr[0] == '-')
                  {                  /* left justify */
                        --justify;
                        chr[0] = *fmt++;
                  }
                  if (chr[0] == '0')      /* leading zeros */
                        zero = '0';
                  while (chr[0] >= '0' && chr[0] <= '9')
                  {                  /* field width specifier */
                        width = (width * 10) + (chr[0] - '0');
                        chr[0] = *fmt++;
                  }
                  /* first switch allows for ld Ld ld ud Ud etc... */
                  switch(chr[0])
                  {
                        case 'U' :                                      /* unsigned number */
                        case 'u' :
                              i = 10;
                              us = 1;
                              chr[0] = *fmt++;
                              break;
                        case 'l' :                                      /* Long (it is anyway) */
                        case 'L' :
                              chr[0] = *fmt++;
                              break;
                        default:                                        /* all others */
                              us = 0;                                 /* not unsigned */
                              break;
                  }

                  switch(chr[0])
                  {
                        case 'd' :                                      /* decimal number */
                              value = va_arg(argptr, int);      /* get parameter value */
                              i = 10;
                              if (!us)
                                    if(value & 0x8000000)
                                    {
                                          value = -value;
                                          ++minus;
                                    }
                              break;
                        case 'X' :                                      /* hexidecimal number */
                        case 'x' :                                      /* hexidecimal number */
                              value = va_arg(argptr, long);
                              i = 16;
                              break;
                        case 'o' :                                      /* octal number */
                              value = va_arg(argptr, int);
                              i = 8;
                              break;
                        case 'c' :                                      /* character data */
                              value = (char)va_arg(argptr, int);      /* get parameter value */
                              *--ptr = value;
                              break;
                        case 's':
                                   ptr = va_arg(argptr, char *);
                              break;
                  }

                  if(i)      /* for all numbers, generate the ASCII string */
                  {
                        do {
                              if((chr[0] = (value % i) + '0') > '9')
                                    chr[0] += 7;
                              *--ptr = chr[0]; }
                        while(value /= i);
                  }
                  /* output sign if any */

                  if(minus)
                  {
                        chr2[0]='-';
                        *outptr++ = '-';
                        if (output)
                              prints(chr2);
                        ++total;
                        if(width)
                              --width;
                  }

                  /* pad with 'zero' value if right justify enabled  */

                  if(width && !justify)
                  {
                        chr2[0]=zero;
                        for(i = strlen(ptr); i < width; ++i)
                        {
                              if (output)
                                    prints(chr2);
                              *outptr++ = zero;
                        }
                        ++total;
                  }

                  /* move in data */

                  i = 0;
                  value = width - 1;

                  while((*ptr) && (i <= value))
                  {
                        chr2[0]=*ptr;
                        if (chr2[0] == '\n')
                        {
                              newline();
                        }
                        else
                        if(chr2[0] == '\t')
                        {
                              chr2[0]=' ';
                              if (output)
                              for(counter = 1;counter<=5;counter++)
                                    prints(chr2);
                        }
                        else
                        if (output)
                              prints(chr2);
                        *outptr++ = *ptr++;
                        ++total;
                        ++i;
                  }

                  /* pad with 'zero' value if left justify enabled */

                  if(width && justify)
                  {
                        while(i < width)
                        {
                              chr2[0]=zero;
                              if (output)
                                    prints(chr2);
                              *outptr++ = zero;
                              ++total;
                              ++i;
                        }
                  }
            }
            else
            if (chr[0] == '\n')
            {
                  if (output)
                        newline();
            }      
            else
            if (chr[0] == '\t')
            {
                  chr[0]=' ';
                  if (output)
                  for(counter = 1;counter<=5;counter++)
                          prints(chr);
            }
            else
            {
                  /* not format char, just move into string  */
                  *outptr++ = chr[0];
                  if (output)
                        prints(chr);
                  ++total;
            }
      }

      *outptr = 0;
      return total;
}

/************************************
      Formatted print to stdout
*************************************/
int printf(char *fmt, ...)
{
      va_list ap;
      long total;
      char buffer[S_SIZE];

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, fmt, 1, ap);
      
      va_end(ap);
      return total;
}

int dprintf(char *fmt, ...)
{
#ifdef DEBUG
      va_list ap;
      long total;
      char buffer[S_SIZE];

      if (nucleuskl_debug == 0) return 0;

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, fmt, 1, ap);
      
      va_end(ap);
      return total;
#else
      return 0;
#endif
}

int sprintf(char* str, const char *fmt, ...)
{
      va_list ap;
      long total;
      char buffer[S_SIZE];

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, (char *)fmt, 0, ap);
      strcpy(str, buffer);
      
      va_end(ap);
      return total;
}

int scanf(const char *format, ...)
{
      va_list ap;
      char *result, input[120];
      int i = 0, j = 0;

      va_start(ap, format);

      while(!(format[i] == '\0'))
      {
            switch(format[i++])
            {
            case '%':
                  switch(format[i++])
                  {
                  case 's':
                        while(j < 120)
                        {
                              input[j++] = input_getch();
                              if(input[j - 1] == 0x08)
                              {
                                    if(j == 1)
                                    {
                                          j = 0;
                                    }
                                    else
                                    {
                                          j -= 2;
                                          video_set_cursor_col(video_get_cursor_col() - 1);
                                          printf(" ");
                                          video_set_cursor_col(video_get_cursor_col() - 1);
                                    }
                              }
                              else if(input[j - 1] == 0x0A)
                              {
                                    input[--j] = '\0';
                                    break;
                              }
                              else printf("%c", input[j - 1]);
                        }
                        j = 0;
                        result = va_arg(ap, char *);
                        
                        strcpy(result, input);
                        break;

                  default:
                        break;
                  }
                  break;

            default:
                  video_putch(format[i] - 1);
            }
      }

      va_end(ap);

      return 1;
}

As you will see there is no assembly... If I want to compile fat.c with -S options I get errors because it uses functions from other files. So I can't use it.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11799110
I notice printf uses a fixed size buffer of S_SIZE. What is S_SIZE. Are you overrunning it with a large message?

For every place where you use _formatout, do this:

if (total >= S_SIZE) {
    panic();
}

where panic is your "that was really bad" function. :)

If you have no panic, just do "*(char*)0 = 0;" to catch these extremely fatal overruns.

Consider implementing _snprintf:

int _snprintf(char *buf, size_t bufsize, char *format, ...)

where it returns count of output characters excluding null terminator, always terminates output string, returns -1 if it would overflow and truncates string at max buf size (terminated).

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore98/HTML/_crt__snprintf.2c_._snwprintf.asp
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799334
Notice that in the middle of the code:

#define S_SIZE  160

An the var "total" is set up with "_formatout":

total = _formatout(buffer, fmt, 1, ap);

So I can't use the panic statement.

Where should I past the "*(char*)0 = 0;" statement. Maybe here?
int printf(char *fmt, ...)
{
      va_list ap;
      long total;
      char buffer[S_SIZE];

      va_start(ap, fmt);              /* set up ap pointer */
      
      *(char*)0 = 0;
      total = _formatout(buffer, fmt, 1, ap);
      
      va_end(ap);
      return total;
}

Can you explain me the advantages of _snprintf ?
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799338
It's also ugly that if I use printf in the get_fat_entry function all things work fine, if not I have 0-values.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11799523
I'm saying that if S_SIZE is 160 bytes, then any printf that generates more than 160 bytes will spill over and corrupt other variables located after buffer. The advantage of _snprintf is it won't allow a buffer overflow, you can set a limit to the output so it won't spill over the buffer and corrupt memory. This overrun might happen in some other area of the program entirely, and cause a problem later in somewhere seemingly unrelated.

Consider this:

#define panic() dopanic(__FILE__, __LINE__)

void dopanic(char *pFile, int nLine)
{
    clear_the_screen_somehow();
    printf("panic! %s(%u)", pFile, nLine);

    // freeze the program here. This never returns
    while (1) {
    }
}

int printf(char *fmt, ...)
{
     va_list ap;
     long total;
     char buffer[S_SIZE];

     va_start(ap, fmt);              /* set up ap pointer */
     
     total = _formatout(buffer, fmt, 1, ap);

     if (total >= S_SIZE)          
          panic();    // I just overwrote memory, die here, where it happened
     
     va_end(ap);
     return total;
}

You'd use the panic macro in places in the program where something really bad happened and you can't trust program execution to continue. In this case, an overrun of buffer will cause stack corruption. printf can no longer be trusted to return. It may have had registers saved on the stack that will not be restored correctly.

See what I mean? In memory, this is how the local variables are laid out:

[buffer]
[total]
[ap]
[saved registers]
return address
[parameters]

If you overrun buffer, you will spill over and corrupt the "total", "ap", and finally, the really bad part, you will corrupt registers saved on the stack. If you overrun a bit further you will destroy the return address, and printf will return to some invalid address and control is lost.

My suggestion is to catch the overrun (after each call to _formatout). When the panic() macro is executed, it will pass the source file name and line number to dopanic. dopanic clears the screen shows the panic location and freezes the program.

If you had _snprintf, you could block the overrun from occuring at all, it would simply truncate the string at 159 characters, saving one for null terminator.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799621
Hmm, it's difficult to understand why this thing should happen if the printf function isn't called by get_fat_entry. All seems to work. I inserted the panic code, but nothing happens. All things are working pretty well.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11799637
Here's a rule of thumb I keep in my head when working with local variables - a way to mentally picture buffer overflows on the stack:

 - as you fill forward in memory, you are going backwards in source lines. If you overflow a buffer, you overwrite the variables declared before the buffer (before as in in-the-source, not memory). if you overwrite further - into the opening brace of the function, you destroyed saved registers. If you overwrite all the way up to the function name, you overwrote the return address. Overwriting the return address is completely fatal. You can't return. When you do, it's to some invalid address. Overwriting just saved registers will cause very very weird problems because stuff that seems impossible can happen.

I suspect that slight overflows are corrupting the stack somewhere else (maybe another source file entirely), causing weird stuff.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799682
You mean I shouldn't call so many procs and funcs from main()? Should it be a problem that I call a function on 5th position?

main -> block_init -> fat_init -> fat_read -> get_fat
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11799698
You put...

     if (total >= S_SIZE)          
          panic();

...in dprintf and printf, right?
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799702
Okay, I've tested it -> no success.
I screw up now, it's unbelievable.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799798
int printf(char *fmt, ...)
{
      va_list ap;
      long total;
      char buffer[S_SIZE];

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, fmt, 1, ap);
      
    if (total >= S_SIZE)          
          panic();    // I just overwrote memory, die here, where it happened
    va_end(ap);
         
      return total;
}

int dprintf(char *fmt, ...)
{
#ifdef DEBUG
      va_list ap;
      long total;
      char buffer[S_SIZE];

      if (nucleuskl_debug == 0) return 0;

      va_start(ap, fmt);              /* set up ap pointer */
      
      total = _formatout(buffer, fmt, 1, ap);
    if (total >= S_SIZE)          
          panic();    // I just overwrote memory, die here, where it happened
      
      va_end(ap);
      return total;
#else
      return 0;
#endif
}

BUT IS IT REALLY THE PRINTF FUNCTION?
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11799808
Maybe not. It could be lots of things. It just looked like it was something global that had potential to overrun when called from somewhere in the program. Some day, you are going to call printf with more than 160 bytes to print, and the panic will catch that before the program crashes.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799821
That might be right. I looked into severyl printf source but I couldn't find anything else that could make troubles.
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 8

Expert Comment

by:adg080898
ID: 11799846
How do you boot it right now? You are using a custom linker script (nucleus.ld). Can you show it? Also, did you write the loader? Is it clearing uninitialized memory?
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11799901
I was doing a project very much like yours (freestanding kernel written with gcc). I ran into a bug in ld that caused it to miscalculate addresses because of a custom linker script.

I had a problem in an earlier project, where I attempted to use separate sections for locked code and locked data (interrupt handlers and their data). I used the __attribute__((section, ...))  declarations and a custom linker script, but it miscalculated the address of some stuff. Would have been cool if it worked because I could lock all data touched in interrupt handlers in one nice linear block of memory.

That was a long time ago, on DJGPP though. I would assume they fixed the linker script bugs by now.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799914
I wanna answer all your questions but look at the code first. There are difficult questions about each load oder run process.

http://www.tds-net.de/nucleuskl.rar
0
 
LVL 3

Author Comment

by:Mathias
ID: 11799926
Sure we use a linker script, because another chance to compile we don't have :-(
I also played with __attribute__ ((packed)) option, without success. The bead thing is that I can't trace back the calling funcs in a kernel which isn't ready.
0
 
LVL 8

Accepted Solution

by:
adg080898 earned 500 total points
ID: 11803338
Ok, I modified the makefile and made a new target "disassemble". It builds a coff executable with debug info, and then does a "disassemble with source" . In order to hold the debug info, a couple of extra sections are added to the linker script. For this reason, it needs a special linker script to build the image with debug info. The normal build should be unaffected.

The following files will be built when you "make disassemble":

 nucleus.cof - coff image with debug info (used to disassemble from)
 nucleus.map - linker map, shows address of each symbol
 nucleus.dmp - disassembly of entire image, includes source so it is easy to work with.

Here is the modified makefile. Note that the indented lines MUST begin with tabs so you'll have to fix them:

-----

CC = gcc
LD = ld
AS = nasm

LDFLAGS = -e 0x0000 --oformat binary
ASFLAGS = -f coff
CFLAGS = -I include -Wall -fno-builtin -g -DDEBUG

CPUOBJ = drivers/cpu/cpu.o drivers/cpu/cpu_k6.o drivers/cpu/cpu_mhz.o drivers/cpu/bugs.o
VIDEOOBJ = drivers/video/vga.o drivers/video/text.o drivers/video/vtext.o
INPUTOBJ = drivers/input/keyboard.o drivers/input/layout.o
BLOCKOBJ = drivers/block/floppy.o drivers/block/ide.o drivers/fs/fat.o

DRIVEROBJ = drivers/pic.o \
      drivers/mem/mem.o \
      drivers/video.o \
      drivers/timer.o \
      drivers/input.o \
      drivers/block.o \
      drivers/cmos.o \
      drivers/dma.o

MAINOBJ = header.o \
      nucleus.o \
      stdio.o \
      support.o \
      logo.o \
      strings.o \
      string.o \
      irq.o \
      multi.o \
      cdsl.o \
      stss.o

OBJ = $(MAINOBJ) $(DRIVEROBJ) $(CPUOBJ) $(VIDEOOBJ) $(INPUTOBJ) $(BLOCKOBJ)

all: nucleus.bin
      @rem echo Nucleus wurde erfolgreich compiliert!

clean:
      -rm $(MAINOBJ)
      -rm $(DRIVEROBJ)
      -rm $(CPUOBJ)
      -rm $(VIDEOOBJ)
      -rm $(BLOCKOBJ)
      -rm nucleus.bin
      -rm nucleus.dmp
      -rm nucleus.map
      -rm nucleus.cof
      @rem echo Nucleus wurde geloescht!

nucleus.bin: $(OBJ)
      $(LD) $(OBJ) $(LDFLAGS) -o $@ -T nucleus.ld
      
%o: %c
      $(CC) $(CFLAGS) -c $? -o $@

%o: %asm
      $(AS) $(ASFLAGS) $? -o $@

disassemble: cofflinkdisasm
      @echo
      @echo Dumping nucleus.bin to nucleus.dmp. Use nucleus.map to find addresses.
      @echo Good luck

cofflinkdisasm: $(OBJ)
      $(LD) $(OBJ) $(LDFLAGS) --oformat coff-go32 -M -g -o nucleus.cof -T nucldbg.ld > nucleus.map
      objdump -d -S nucleus.cof > nucleus.dmp

-----

And create a new file named nucldbg.ld for the debug linker script:

-----

SECTIONS
{
  . = 0x00100000;
  .text : { *(.text) }
  .data : { *(.data) }
  .bss : { *(.bss) }

  /* Stabs debugging sections.  */
  .stab 0 : { *(.stab) }
  .stabstr 0 : { *(.stabstr) }
  /* DWARF 2 */
  .debug_aranges  0 : { *(.debug_aranges) }
  .debug_pubnames 0 : { *(.debug_pubnames) }
  .debug_info     0 : { *(.debug_info) *(.gnu.linkonce.wi.*) }
  .debug_abbrev   0 : { *(.debug_abbrev) }
  .debug_line     0 : { *(.debug_line) }
  .debug_frame    0 : { *(.debug_frame) }
  .debug_str      0 : { *(.debug_str) }
  .debug_loc      0 : { *(.debug_loc) }
  .debug_macinfo  0 : { *(.debug_macinfo) }
}

-----
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803361
Did you notice that there are two source files with the same name? fat.c
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803363
I see... not using LFN one.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803453
I noticed a HUGE error in header.asm. It initializes esp *misaligned*!!! That means the cpu is struggling with every stack access. Not that it would cause an actual bug, but it slows the cpu right down.. Fix this line in header.asm:

  mov esp,0x002fffff

It should be

  mov esp,0x00300000

Before you jump and say I am wrong, consider this: The cpu subtracts 4 from esp BEFORE writing a value to the stack. So if it starts at 0x300000 as I suggest, then the first push will be to 0x002ffffc. Since the push is 4 bytes, the first push will be to

0x002ffffc
0x002ffffd
0x002ffffe
0x002fffff

I am looking into a technique using gcc's function instrumentation capbility, to add a stack overflow check at every function entry. See the gcc docs for the -finstrument-functions switch.

0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803493
Ok, I have completed the changes necessary to automatically add stack checking to every function.

Add the following lines in the appropriate places in support.asm:

----

  global ___cyg_profile_func_enter
  global ___cyg_profile_func_exit

  extern _dopanic

----

And add this near the bottom, before "section .data"

----

; Stack checking code. (Same function used on func entry AND exit)
; Stack pointer must be between 0x00300000 and 0x00200000
; This is only used when you compile with -finstrument-functions
; Raises panic if stack pointer is out-of-bounds
; Completely transparent. Even preserves flags.
___cyg_profile_func_enter:
      nop              ; This is so disassembly shows proper different symbol for entry/exit
___cyg_profile_func_exit:
      pushfd
      cmp esp,0x00200000
      jbe _stack_overflowed
      cmp esp,0x00300000
      ja _stack_overflowed
      popfd
      ret

_stackoverflowmsg db 'Stack overflow!',0

_stack_overflowed:
      ; Due to stack overflow, I can't push parameters for dopanic,
      ; so flush/reset stack, dopanic freezes execution anyways
      mov esp,0x00300000
      push dword 0
      push _stackoverflowmsg
      jmp _dopanic

----

And, add change the CFLAGS in your makefile to this:

----

CFLAGS = -I include -Wall -fno-builtin -g -DDEBUG -finstrument-functions

----
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803497
OOPS! Change

jmp _dopanic

to

call _dopanic

I forgot that the called function needs the return address there to get the parameters properly
0
 
LVL 3

Author Comment

by:Mathias
ID: 11803508
@adg: Thanx for helping me so much. You see errors I can't see because I don't know (e.g. system optimizations). But the error isn't solved. I wrote lines and I removed lines in the code without success. Where do you came from, maybe I can have your email? My is: webmaster@tds-net.de

I increased the value of points. Maybe it helps.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11803511
Wow, you are too fast for me :-)
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803516
I have not completely looked through the threading code, but I suggest you add some sort of "thread local storage". What I mean is, for each thread, there should be a special storage area for information related to that thread.

For each thread created, allocate a small structure for data related to that thread. Make a selector in the GDT for that thread's fs register (allocate a range of selectors for thread local storage). The base address of the segment should be the address of the thread-local structure. This structure could contain the stack bounds for the CURRENT thread. Say I had the following structure for each thread:

typedef struct {
    unsigned nStackBottom;
    unsigned nStackTop;

    unsigned nUserThreadData;

// etc...
} ThreadLocalData;

So in a thread, I can look at fs:[0] to see the lowest valid stack pointer address, or fs:[4] to see the highest valid stack pointer address. Because each thread can assume offset zero on fs, the base of the segment really points to the data. This way, my stack checking code can compare against fs:[0] and fs:[4] instead of the hard-coded addresses 0x00200000 and 0x00300000.

This is not necessary right now, but will be once you start using multi-threading.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11803529
Did you test the code within Bochs or so? I get only stack overflows just before the code starts...
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803553
I tried signing up at your home page, but it says some German stuff. (translates to "Log in not correctly" or something in babelfish :). I don't know German.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11803554
Okay, I've looked up into my code. It looks like the memory code works not as well as it should do.

It hangs at:

memory: Allocating user memory space (XXX entries, XXX

Kernel panic! Stack overflow! (0

Kernel panic! Stack overflow! (0

etc.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803555
I am not *booting* the kernel, only examing the disassembly to check that stack-check code.

Should I do a build and make a boot floppy? What filename for the kernel file? Does it just look for nucleus.bin on the floppy? How do I write the boot code to the boot sector? I could look but you could easily tell me... :)
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803558
When I try to signup I get

Konnte User nicht einfügen (Portal).
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803560
Since you are actually getting stack overflows, I could quickly extend the stack check to show the address of the overflow and do a printf for a stack panic...
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803576
Put this in support.asm

Shows more detailed stack panic information

Put this in the appropriate place:

extern _printf

And update:

; Stack checking code. (Same function used on func entry AND exit)
; Stack pointer must be between 0x00300000 and 0x00200000
; This is only used when you compile with -finstrument-functions
___cyg_profile_func_enter:
      nop
___cyg_profile_func_exit:
      pushfd
      cmp esp,0x00200000
      jbe _stack_overflowed
      cmp esp,0x00300000
      ja _stack_overflowed
      popfd
      ret

_stackoverflowmsg:
db 10,10,'Stack overflow! At %08X called from %08X',10
db 'Use map file to find what these addresses mean',10
db 0

_stack_overflowed:
      ; Grab the parameters, the overwrite already occurred,
      ; but read __cyg_profile_func_XXXX parameters anyways
      mov eax,[esp+8]
      mov ecx,[esp+12]
      ; Due to stack overflow, I can't push parameters for dopanic,
      ; so flush/reset stack, dopanic freezes execution anyways
      mov esp,0x00300000
      push ecx
      push eax
      push _stackoverflowmsg
      call _printf
.hang
      hlt
      jmp .hang

0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803589
This is better.

Put the strings and instructions in the order shown! (short jumps):

; Stack checking code. (Same function used on func entry AND exit)
; Stack pointer must be between 0x00300000 and 0x00200000
; This is only used when you compile with -finstrument-functions

_stackoverflowmsg:
db 10,10,'Kernel panic! Stack overflow at %08X '
db 'called from %08X. '
db 'ESP was %08X',10
db 'Use map file to find what these addresses mean',10
db 0

___cyg_profile_func_enter:
      nop
___cyg_profile_func_exit:
      pushfd
      cmp esp,0x00200000
      jbe _stack_overflowed
      cmp esp,0x00300000
      ja _stack_overflowed
      popfd
      ret

_stack_overflowed:
      ; Grab the parameters, the overwrite already occurred,
      ; but read __cyg_profile_func_XXXX parameters anyways
      mov eax,[esp+8]
      mov ecx,[esp+12]
      lea edx,[esp+16]
      ; Due to stack overflow, I can't push parameters for dopanic,
      ; so flush/reset stack, I freeze execution anyways
      mov esp,0x00300000
      push edx
      push ecx
      push eax
      push _stackoverflowmsg
      call _printf
.hang
      hlt
      jmp .hang

0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803703
You're not starting threads yet right? It is supposed to be always using the kernel stack so far, right?
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11803810
I figured out how to build boot image myself... It's writing the image now. Will try actual run.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11804465
Okay, to have you informed about current things - here is a short summary:

booting kernel
init memory and co.
starting multitasking
init two threads in nucleus.c
starting cdsl.c (shell)

I'll send you the whole code files. You only have to use m.bat in the temp directory to compile everything (I hope) :-)
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11804800
Yes, I noticed that it is going straight into multitasking. That requires the stack check to be thread-aware. There is no way to do thread-local-storage right now. When you add that, you can make the stack check use the stack bounds of the current thread.
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11815693
I've tried to get somewhere on sourceforge but nobody is answering. Where the heck do I go from here https://sourceforge.net/projects/nucleuskl/
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11815830
My name on sourceforge is doug65536
0
 
LVL 8

Expert Comment

by:adg080898
ID: 11816205
I checked out the newest revision of nucleus from sourceforge. Several differences between the one you sent and the one I checked out. How old is the CVS copy?
0
 
LVL 3

Author Comment

by:Mathias
ID: 11818111
The CVS copy is still older than the current .rar I sent you. I think about 5 months or so.
0
 
LVL 3

Author Comment

by:Mathias
ID: 11833245
I accepted you answer cause you pointed me the right way.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Article by: Nadia
Linear search (searching each index in an array one by one) works almost everywhere but it is not optimal in many cases. Let's assume, we have a book which has 42949672960 pages. We also have a table of contents. Now we want to read the content on p…
This article will show, step by step, how to integrate R code into a R Sweave document
An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …
In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now