We Are Busy

OK, so Bruce has done an amazing amount of work on the SharpOS kernel's demo shell, which provides an interface for users to inspect and play with the SharpOS kernel as it currently stands. It is still rather raw, but the remaining bugs are being fixed and once that is done we can begin looking at the first milestone release of SharpOS. Today we've been busy organizing our bug tracker, closing old bugs and opening new ones, assigning them to the milestones etc. I wrote a draft article for introducing SharpOS to the community, we came up with an idea of using a chicken and a cracked egg shell for our mascot, we've got a few new faces around the list and IRC channel, and the usual work on mscorlib stubs continues. The IRC channel has been ablaze with discussion, some important, some not, and our list has been equally busy.

The bottom line is, after a year and two months we are almost ready to show our proof of concept to the world! I don't know about the rest of us but I am at least mega-excited.

PS: I did nothing on Nash today so far.

Pointers, and Structures, and Strings... Oh My...

Ye machine language is an unforgiving beast, aye, she is...



Its true folks, the demo-shell is finally getting off the ground. As you can see, I've added a couple commands. There are still a couple of very small bugs - but the majority have been hammered out.

Some things to remember when you are coding an OS:

1) Your code is what you code. An infinite loop is always an infinite loop. Even if you don't know you coded one, and can't figure out why your kernel "locks up" - and then revert all changes and try again - if you code an infinite loop twice, then you code an infinite loop twice.

2) Pointers are not toys. They are pointy. If you play with some (instead of others), (like writing allocation node consolidation code to prevent memory fragmentation), the ones you aren't paying attention to, will twist together and stab you from behind. You cannot have tunnel vision when playing with pointers, they are unforgiving.

But basically, I've journeyed along the the managed-brick road, and come out with a demo-shell that people can start playing with. In fact, if you download SharpOS from our Subversion repository's trunk right now, and compile and boot it, you will see the demo-shell in action.

There is one bug I'm still trying to figure out. The first command or two, in the linked list of commands, always seems a bit corrupt. Sometimes the first command in the list is completely swallowed into the void. And other times, the list will have two seemingly corrupt commands off the top, with the complete list of expected commands tacked on the end. I've double checked all of my initialization code, and linked-list management code, and I haven't figured it out yet.

More commands, (and obviously a bug-fix or two), are still on the way...

SharpOS Kernel + UNIX

For those of you who don't know me, my name is William Lahti. I am one of the original developers of the SharpOS project, and have contributed various odd parts of the kernel, mostly amounting to organization. At some inspired point I also wrote the foundations of a modern, network-transparent C# window system dubbed SharpWS.

Due to my inexperience with kernels and rather shoddy performance at programming one, I decided that I could help the project without doing too much design work on my own if I were to create a UNIX foundation (libraries and commands/utilities) that could be used natively on SharpOS (written in C#).

So, I started with the shell. I started the Nash shell, which by now has actually gotten quite workable, except for a few bugs in mono that are making things difficult. Currently, Nash supports variables (dynamically and statically typed for future interoperability with scripts), aliases, complete line editing and autocomplete, a set of builtin commands, support for interweaving script calls with Nash commands, the foundations of pipeline support, correct handling of most common special variables/environment variables, and more. It strives to be at least POSIX-compatible, if not BASH-compatible. Oh, and it runs programs alright too. It's still got it's issues, though. Currently, Console.Cursor[Top/Left] are not updated accordingly by Mono if other programs write to the terminal, but a workaround is in progress.

Now, what's the fun of having a managed UNIX shell if we don't have managed UNIX utilities? Sure, I could use Nash with my local GNU utilities, but that won't be the case when SharpOS is ready to run this bad boy. So, I started implementing UNIX utilities. So far I have started two packages of UNIX utilities. 'coreutils', like GNU coreutils, contains the most essential tools... I've implemented cat, cp, cut, date, dirname, hostname, kill, killall, ldd, less, ls, mkdir, pwd, rm, sleep, touch, tr, uname, and which.

The second is netutils, which so far only has one program, 'ping'. Unfortunately, Mono decided not to implement the System.Net.NetworkInformation.Ping class, because it would require super user powers. I disagree, but for now I can't test ping :-\.

I've started three libraries: SharpOS.Unix, SharpOS.Unix.Tar, and SharpOS.Unix.Curses. SharpOS.Unix implements common code for handling UNIX permissions, command-line options rules, timestamps, etc. SharpOS.Unix.Tar is a managed interface for reading and writing tar files. Yes, I am aware that SharpZipLib supports Tar, but I only discovered this fact after I finished the first commit of SharpOS.Unix.Tar, and SharpZipLib is not as powerful or accessible as this API is. Finally, SharpOS.Unix.Curses will be a managed terminal interaction API inspired by ncurses, and using terminfo terminal descriptions from Linux. Right now, only the terminfo code is completed, and the terminal code is not yet committed.

That just leaves ntar, which is a (quite-functional and compatible) implementation of tar, using the SharpOS.Unix.Tar library.

I've tentatively been referring to the whole set of tools as 'nunix' and/or SharpOS.Unix (the namespace of libraries and program classes).

Okay, this blog post got really long but I talked about all the new stuff I'm doing, so there! I'll try not to make my next ones so long :)

Getting the Juices Flowing

Well folks, if you've made it here, you can see that SharpOS now has a development blog. On the other side of things, if you have no idea what SharpOS is, then I should point you at our website – http://sharpos.sourceforge.net/ However, the website is under construction as part of a move (back) to http://www.sharpos.org/ . So, I'll introduce you to the project.

The SharpOS project aims to create an operating system using 100% C# code. A lot of programmers will respond to that thinking, "How is that even possible?" C# is a managed, JITed language. It compiles to CIL, which creates a safety net by being completely compiled into native code, on the spot by the .NET (or Mono) runtime, only when the program gets executed. Unlike C/++ or Assembler, it has no constructs that make it immediately useful on a low level. But, it does have pointers. And that's all we really need.

The founding members of SharpOS decided to tackle the conundrum at hand by writing an Ahead-Of-Time compiler, (in 100% C#), that takes our kernel .NET assembly, and generate a bootable kernel image from it. That is, it compiles the CIL of the kernel DLL into native code. To get around not being able to do low-level machine language stuff, the AOT contains a class with static method stubs on it, (which in and of themselves do nothing). There is a stub for every x86 assembler instruction. And when the AOT compiles the kernel, and encounters calls to those method stubs, it converts them into the correct native instructions. This, along with using method stubs to declaratively label static memory allocations, allows us to leverage C# against the entire CPU without leaving the comfort of our C# IDE.

The project is definitely still in an infantile state. We need to write an entire runtime support layer so that we can embed a JIT. Remember: think assembler – unless we write memory management, there is no: object foo = new object(); And unless we write type information handling, there is no: if(foo is Int32) {…}

So, in order to stick to our C#-only motif, we have to use a restrained form of C# (that is, no heap types, no type information, etc.), to write a runtime, that the rest of our kernel can be AOTed against. This means we also have to emulate parts of the Mono/.NET corlib. Eventually, our goal is to use our AOT as the JIT that will get embedded in the kernel.

People tend to notice that SharpOS developers are not scared of chicken-and-the-egg problems. If you understood the last couple of paragraphs, you'll understand why.

Currently, the three most active SharpOS members are all doing very different things. xfury is working on porting/emulating common Unix utilities in C# - because we will definitely need them when the time comes. DarxKies, being the lead on the AOT part of the project, is working on writing runtime support so that we can start embedding fully managed code into the kernel (but at that point, still AOTed). And yours truly, has been working on a demo command-line shell to grab the attention of passer-bys, like you.

Here is a screen-shot of the beginnings of fruits of my labors (see the yellow text at the bottom):

You'll notice I had to type the "version" command in twice. Frankly, it's very much full of bugs. But considering our memory management code was just a temporary satisfaction of a complex need, and I had to write a ton of dynamic string handling code from scratch, it's pretty impressive, right?

Within the next few weeks, I'll hopefully iron out the rest of the bugs and code a handful of commands so that we can put out a first release.

Each of the commands is stored as a structure consisting basically of a string pointer for the command's alias, and a function pointer that points to the function that does the work for the command. I know, who thinks of string pointers and function pointers when they think of C#? But when you don't have class instances, and you don't have Runtime Type Information, what is an OO programmer to do?

Like I said, we are in baby stages. (And in need of lots of help!) But hopefully, the screenshot will help get those juices flowing…