Solved

Kernel

Posted on 2004-09-25
23
830 Views
Last Modified: 2008-02-01
Hi all,

I welcome everyone to join this discussion about kernels. Points will be split equally between all good comments.

This question has been posted in the MS-DOS section so that we start from the most basic kernel of DOS, io.sys and msdos.sys.

My understanding of DOS and its kernel is as follows (please correct me if I am wrong):
---------------------------------------------------------------------------------------------------------------------------------------------------
All communication with DOS (in and out) is controlled by 2 files: io.sys and msdos.sys. They cannot however speak by themselves and therefore require a command interpreter, command.com to speak to the user and recieve the user's input. All output command are issued by the kernel to the command interpreter which displays it on the screen. We too, cannot directly talk to the kernel and therefore need the command interprter.

There is a set of limited/predefined commands defined in command.com like copy, del, dir, cd etc. (known as DOS INTERNAL commands) that are built for the convience of the user. The DOS internal commands are limited to the command interpreter. Therefore, there is a way to communicate with the kernel directly by building real exe files (like console C/C++ applications) that are limited only to the capability of the kernel (instead of being limited to the command interpreter). They are called DOS EXTERNAL commands like format, label, xcopy etc.

An example of something that cannot be with the command interpreter but within the reach of the DOS kernel is taking in a string from the user (a typical cin >> in C/C++). C++ is simplified for our use so that we dont have to communicate with the kernel directly using assembly language.

However, something out of the reach of the kernel itself is graphics and external speaker sound which leads to the evolution of new kernels and operating systems like Windows and Linux.
---------------------------------------------------------------------------------------------------------------------------------------------------


This is where my understanding starts the collapse:
In C++, you can have a DOS graphics BGI driver. Now what is a driver? Is it an add-on to the kernel. Was the kernel never meant to draw colour on the screen and the driver teaching it how to do that or was the kernel always able to do that and the driver just cosmetic? Why cant a driver act like an add-on to the kernel and make it draw like Windows?

How does a kernel load itself in memory? I understand that the BIOS searches the first device in the boot sequence for a boot sector and loads whatever is mentioned in it into memory (RAM). Now, I understand that to make a floppy bootable with the most basic kernel, it has to contain a minimum of three files: io.sys, msdos.sys, command.com. What I dont understand is how more advanced operating systems start: What is the actual Winows (XP or should we begin with 98?) kernel? Windows XP kernel is listed as 'ntoskrnl'. Does that one small file (1.92 MB) handle all the functions in Windows!? I dont think so. I think the Windows kernel is made up of more than one file for one purpose. And what really is explorer.exe? Can it be considered a command interpreter?

Why is Windows "DOS based"? It is quite prominent from the presence of io.sys and msdos.sys in every Windows system. It performs all common functions with the help of io.sys and msdos.sys. (If a copy fails, a messagebox with 'invalid MS-DOS command' pos up) Why does it have to depend on small dos programs to do everything for it (like telnet to communicate with other computers)? Why isnt it independent? Why does it use a DOS loader? Isnt DOS old enogh? Why cant it load its own kernel straight into memory and show a nice graphical startup?

And the big question: How is a kernel written? I assume its done in assembly language (again, please correct me if I am wrong)


Thanking you for your time,

Regards,

ram_einstein
0
Comment
Question by:ram_einstein
  • 12
  • 11
23 Comments
 
LVL 10

Accepted Solution

by:
For-Soft earned 350 total points
ID: 12153802
DOS kernel is in fact a set of system services, like memory management, file management, keyboard and display management. To work with user it is necesary to use some shell program.
The kernel is designed to provide services for programs to run. Shell is a user compatible part of DOS. So COMMAND.COM is a program, and it can be replaced by other shell program.

About DOS session in Windows GUI.
Windows 9x is using DOS kernel just to start it's own kernel. Then all services are passed by Windows kernel bypassing old DOS kernel. DOS programs started in DOS window, are not working with IO.SYS kernel. Windows kernel emulates DOS services, and adds some more (like LFN support, mouse support).

About drivers in DOS.
Driver adds new services in DOS. The whole DOS kernel is based on interrupt chain system. If you add a new interrupt service routine, you can add a new services to programs. DOS is just a BIOS extension. It adds more services than BIOS does. But both work in the same interrupt based sytem.
To use graphics mode, or sound, programmer has to acess devices directly, or to run a driver, and then to use driver provided interrupt interface.
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12153860
Windows XP seems to use cmd.exe as a "DOS shell". Is this a kind of replacement for command.com and does it have a larger collection of predefined commands? So I can extend the capabilities of an ordinary user by creating a better shell for that kernel, right?

Tell me more about the so-called "Windows kernel". And why can't the Windows 9x kernel start by itself if an old DOS kernel can?

Regards,

ram_einstein
0
 
LVL 10

Assisted Solution

by:For-Soft
For-Soft earned 350 total points
ID: 12153927
Windows 9x was made that way. I don't know why. Ask Microsoft.

I believe cmd.exe is simply a DOS emulator.
In original DOS it is possible to use other shell. It is possible to use a regular program as a shell.
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12199022
Since nobody else seems to be interested, I think you can answer my questions:
1. Is my understanding (enclosed seperately in my question) correct?
2. What is the Windows kernel? Is it ntoskrnl or a huge collection of files?
3. How does one write a kernel or an Operating system?
4. Is the Windows GUI our means of talking to the kernel? Is it a "Command Interpreter" or "Shell"?
5. Only the Windows kernel can do some things that io.sys cannot do like reading NTFS partitions right? So how come command.com or cmd.exe, the command interprters for DOS read NTFS partitions while in Windows? (If you dont understand, Start>Run>Command and you can navigate through you hard disk. Try doing that with a boot disk) Is command.com/cmd.exe reading/recieving commands from the "Windows kernel"?

Thanking you in this regard,

ram_einstein
0
 
LVL 10

Assisted Solution

by:For-Soft
For-Soft earned 350 total points
ID: 12199572
I believe, the kernel is a part of operating system wchich assigns procesor time, memory and system resources (IO ports).

The dos kernel would be io.sys and msdos.sys (version dependant). But there is no procesor time assign function in DOS. Only conventional memory management.

Other memory management is made in himem.sys and emm386.exe.

Resorces management is done by drivers themselves.

So, it's difficult to talk about kernel in DOS.
Kernel tasks are spreaded through many components in DOS.

"1. Is my understanding (enclosed seperately in my question) correct?"
Your understanding of BGI driver as add-on to the kernel is correct.

But, it would be good to find a definition of the kernel, because my beliefs are not the definition.

"How does a kernel load itself in memory?"

The bios loads from MBR a program which starts operating system. Everything bios has to do is find a program in first sector of some media.

"3. How does one write a kernel or an Operating system?"

The most common low level language is C. And many kernels were written in C language.
0
 
LVL 10

Assisted Solution

by:For-Soft
For-Soft earned 350 total points
ID: 12200743
I just realized, kernel could be the part of OS, which is absolutely necesary for a system to work.

io.sys and msdos.sys for DOS. It is possible to run program without command.com.

Don't know which files in Windows, but definitely not EXPLORER.EXE.

This leads to a conclusion. Kernel is a part of OS wchich can not be thrown away, no matter what it does. And without it you can not start a system.
0
 
LVL 10

Assisted Solution

by:For-Soft
For-Soft earned 350 total points
ID: 12201176
"Why is Windows "DOS based"?"

Maybe Microsoft did not want to create separate boot process, and windows kernel starts like normal DOS program, then it takes controll over all resources. This situation has some advantages. If DOS emulation in windows is not perfect enough for some DOS programs to run correctly, you can still run them before windows kernel is started. Greater backward compatibility. And I must admit I like it a lot, and I'm using it a lot.

"And what really is explorer.exe? Can it be considered a command interpreter?"

I think so. It would be best to call it a system shell. You can change a system shell to an other program. Just like it is possible to replace command.com, or explorer.exe.

"So how come command.com or cmd.exe, the command interprters for DOS read NTFS partitions while in Windows?"
System shell or program can do anything system can do. If you add a driver to work with some other file system, you will be able to use a new service in all programs.

I believe it is better to think of a OS as a set of services provided to user program. System shell is just a standard user program with a function to start other programs. You can add new drivers to the system providing additional services.

At the beginning there was no OS. Every program had to do everything. Then some common used functions were packed in a program (OS) providing services to other programs. So new programs were simpler, but they needed the service provider (OS) to run. Newer OS'es took more and more job from user programs, and user programs were easier to make.

I'm curious. Where is the place for kernel in this story? I believe it is just a set of most important functions in the OS. Other OS functions depend on kernel functions. Kernel is able to work without system functions, and OS can not work without kernel.

I'll better be ending. Who knows how many other kernel definitions I could create.
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12206362
Thanks a lot for your comments. I really liked this paragraph:
"At the beginning there was no OS. Every program had to do everything. Then some common used functions were packed in a program (OS) providing services to other programs. So new programs were simpler, but they needed the service provider (OS) to run. Newer OS'es took more and more job from user programs, and user programs were easier to make."

And I was surprised by this: "The most common low level language is C. And many kernels were written in C language."

Information on kernels on the net is very thin indeed. Not many can really explain what a kernel is and what it does.
-------------------------------------------------------------------------------------------------------------------------------------------------
Anyway, I have one final request (Its a little out of the topic, area etc.):
I know a bit of C++ and I can't even begin to imagine how to build a kernel. There are basically two types of programs you can build on it: One that is based on the DOS kernel and another on the Windows kernel. I don't know if you know any C++ but if you do continue:
A typical DOS program would include <iostream.h>,<conio.h> etc... and look like this:

void main(){
                   ...
}

A windows program would require <windows.h> and would look like this:

int WINAPI WinMain(HINSTANCE hInst, HINSTANCE hPrev, LPSTR pszCmdLine, int iCmdShow){ .. }
LResult CallBack WindProc(...){...}
################################################################################
Now how do you build a kernel itself? I assume you have to start from the basics, use your own header files and start by allocating and deallocating memory but that would mean assembly language. By any remote chance, could we lay our hands on the MOST BASIC kernel and examine it like something thats just able to load itself into memory?

I am sorry if I am asking for too much,
Thanking you for your time,

ram_einstein
0
 
LVL 10

Assisted Solution

by:For-Soft
For-Soft earned 350 total points
ID: 12206518
Every programming language has a built in support for an operating system. In C++ you can chose a support for DOS kernel or Windows kernel, by a proper library.

The final system compatibility depends on linker job. The linker will merge all program components with system libraries in sytem compatible way.

If new OS kernel is created you can not use any system support library. You have to do everything yourself. You have to create your own policy of memory managemt. The simplest memory management I can think of is: everything from some adress up is not your OS business, and you do not care what user program will do with it.

I believe Microsoft went the easiest way to use built in language functions to start the kernel in Windows 9x. The kernel loader had file read support from DOS OS. After Windows kernel is loaded all interrupt vectors are switched to windows kernel, and computer is now controlled by Windows kernel.

One thing for sure. It is simpler to do some things in assembly language. C++ is easy to mix with assembly code. It is easy to mix pascal with assembly code also. Anyway programmer must understand how the language works, or how the compiler uses procesor and memory, in order to mix the language with assembly.
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12210576
"At the beginning there was no OS. Every program had to do everything." Can you give me an example of a program that requires no OS to start and can boot itself into memory?

The only type of BOOTABLE floppy I know of is one that contains the DOS kernel and shell (io.sys, msdos.sys, command.com). What does the BIOS search for in a media to verify whether it is "bootable"?: Okay, I know it reads head 0, cylinder 0 or something like that but what does it look for there? A kernel? I really want to create a bootable floppy without the DOS kernel on it.

Thanking you yet again,

ram_einstein
0
 
LVL 10

Expert Comment

by:For-Soft
ID: 12210664
Memtest 3.0 can boot directly from FDD without DOS. Every boot manager does this also.

First sector of a bootable media must apply to few standards. Bios reads the sector into memory. Checks if it is bootable, and starts a small program from the sector. Bios task is then completed. The program is actualy loading a kernel.

In Memtest 3.0 the program from boot record starts Memtest instead of OS.
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 2

Author Comment

by:ram_einstein
ID: 12210700
I found exactly that using Google just now!

The problem is that I don't really understand what a boot floppy is. What are image, bin and iso files> (*.img,*.bin,*.iso)
Are they used to make a floppy/cd (media) bootable?

I read this:
Once a floppy or cd is "bootable", it has something called a "boot image" that we can't see physically: Examples of programs that can extract that: MKBT (Make bootable) and BBIE (Bart's boot image extractor). They can extract the boot image and compile it to a bin file. They can also put the bin file into the floppy or cd (Burning sotware can do that too). I read that the boot image tells BIOS where to jump to loacte the MBR (Master boot recors which is a table of the partition, file systems on the disk). Does that mean that every boot image is different or is there a standard boot image for a floppy? After that its the "boot loader's" (like NTLDR) job to load the kernel, right?

Can you create a boot floppy by dumping io.sys, msdos.sys and command.com on a floppy? What about the boot sector? What is the boot image? Where is the boot loader? Thats a DOS boot floppy. What is a Windows XP boot floppy? It doesn't seem to require any of these files!

Thanking you for your time,

ram_einstein
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12210702
Question value increased by 50 points.
0
 
LVL 10

Assisted Solution

by:For-Soft
For-Soft earned 350 total points
ID: 12210965
There are some differences betwenn bootable floppy disk, and nonbootable one.
- The bootable disk has a program in boot record. The program purpose is to start OS.
- the bootable disk has system files with OS kernel.

If you copy system files to disk it will not boot because there is no "OS loader program" in boot sector. It is necesary to use SYS command in MS-DOS to put the "loader program" in boot sector. Boot image software makes an exact copy of a floppy disk, from the first sector to the last one. So "loader program" is stored in the image also. If you want to copy a boot disk, you have to use boot imaging software, because it must be an exact sector to sector copy of the disk.

"I read that the boot image tells BIOS where to jump to loacte the MBR (Master boot recors which is a table of the partition, file systems on the disk)."

Disk does not contain the boot image. Boot image is just an image of a bootable disk.

Master boot record is always stored in the first sector of the hard disk drive. Bios does not have to look for it. Its always in the first sector. Every partition has it's own boot record, but there is only one MBR, and it is always in the first sector of the disk. There are no MBR on the floppy disk also. Hard disk structure is more complicated than floppy disk structure, every partition can have different "loader program".
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12265598
Guess I got that wrong: "I read that the boot image tells BIOS where to jump to loacte the MBR". Thanks for the corrected statement.

What exactly is a boot record and a Master boot Record? Boot record, def. "The program recorded in the Boot Sector. All floppies have a boot record, whether or not the disk is actually bootable." Okay so how many things are there in the "boot sector" (Is the boot sector the first sector also?): The boot record, the "OS loader program", the "Master boot record" what else? And what is the real difference between the Boot Record and the MBR?

What exactly does the sys command in dos do? It is supposed to put system files on to a media. I assume that the command also copies the "OS loader program" otherwise the media wouldn't be bootable in accordance with your previous comment. "- ition can have different "loader program"". So there are different loader programs for different type of media, OSes etc right? Is this loader program represented physically as a bin file?

Regards,

ram_einstein

p.s- Motherboard burnt! I am currently working on my laptop. So don't expect a very quick response! I hope to get back my desktop computer in three days.
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12265653
I found an excellent site containing valuble information on floppies. Check it out: http://members.iweb.net.au/~pstorr/pcbook/book4/floppyd.htm

Regards,

ram_einstein
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12265668
The commonly accepted term is "Bootstrap Loader": We'll talk in terms of the bootstrap loader. The bootsrap loader resides in the computer's ROM!?

"Bootable and non-bootable floppies have identical boot sectors. When you try to boot a PC Computer from a non system disk, the error message is coming from the simple Boot Strap Loader, loaded into RAM, from the Boot Sector"

I don't understand very much. Plus, what is this "OS loader" then?
0
 
LVL 10

Assisted Solution

by:For-Soft
For-Soft earned 350 total points
ID: 12265698
Bootstrap loader is a program. It can be the OS loader, but it do not have to be the OS loader.

Formating program normaly puts in to boot record program that displays "no system disk" message instead of booting OS.

If disk was formatted as boot disk bootstrap loader is booting OS.
If disk was formated as not bootable, bootstrap loader is a simple program to display "no system disk" message.
0
 
LVL 10

Expert Comment

by:For-Soft
ID: 12265721
Bios has a boot strap function too. When you set the boot priority in bios, you set which bootstrap function to run first.

In fact everything works as a chain.
- Bios bootstrap function loads bootstrap loader from MBR.
- MBR bootstrap loader calls next loader from bootrecord.

In case of FDD there is no partition table, and the chain is shorter.
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12269582
Thanks! Now I understand a lot more.

I understand that there are 2 ways in which a floppy can be formatted:
- A non-bootable floppy in which the bootstrap loader written to the first sector of the disk displays "no system disk" and
- A bootable floppy in which the bootsrap loader written to the first sector loades the kernel of the desired OS.

Please correct me if I am wrong.

So how do I create a bootstrap loader program to load a different kernel? For a start, how do I even find it? Since it is written to the boot sector, I cannot see it when I read a floppy normally because I am reading the data section, right?

Thanking you yet again,

ram_einstein

p.s- Still on laptop so don't expect very quick responses!
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12269675
Hey, check this out: http://www.geocities.com/mvea/bootstrap.htm

It tells you how to write a bootsrap loader and put it onto a disk!
0
 
LVL 10

Expert Comment

by:For-Soft
ID: 12269931
Yes, You are right.
To see the bootstrap sector you have to use a disk editor.

Bootstrap loader is a program working in absolute adress mode. It has to be BIOS services dependant only. It can not use any OS services.
0
 
LVL 2

Author Comment

by:ram_einstein
ID: 12274592
Thanks for everything, For-Soft.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

One of my most closely kept secrets is revealed in this discussion How to output text on the same line This question was recently posted in EE by Simon336697 (http://www.experts-exchange.com/Programming/Languages/Scripting/Shell/Batch/Q_2459…
TOMORROW TOMORROW.BAT is inspired by a question I get asked over and over again; that is, "How can I use batch file commands to obtain tomorrow's date?" The crux of this batch file revolves around the XCOPY command - a technique I discovered w…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now