DrunkenBlog

Reader Mail: 'Porting' OS X to the Linux kernel?

Regarding Funneled Performance, Clinton P writes:

What are the possibilities of 'porting' OS X to the linux kernel? Linux kernel has the real multi threading that is required and seems to be good enough for the IBM 710 multiprocessor server you just mentioned. It seems the BSD decision is holding back OS X. With IBM standardizing on Linux and Apple on the IBM processors it may be worth the effort to migrate.

Lots of questions in this one, so we're going to have to break them out individually in order to keep things coherent.

What are the possibilities of 'porting' OS X to the linux kernel?

Part of the question is going to depend on how you define what Mac OS X is in relation to its Darwin underpinnings, but I'd have to say the probability is very low. Very, very low. As for possibility... doing what you're asking would be one of two things:

• Reworking the Linux kernel/userland to map back to Mach
• Letting go of the Mach deal (XNU) and simply porting the technologies that matter over to a Linux kernel

It's all technically possible, but highly unlikely and is the kind of thing they could probably have running in a rudimentary form in a month but would end up taking years to complete. It's just not really on the table.

I'd say that out of the two options the latter has a higher likelihood, which means a snowballs chance in hell. At this point, I think you'll see their energies directed towards accommodating the differences between other Unix's (and they do do things differently!) and OS X in order to try to catch some of Linux's wake, rather than becoming Linux.

As an aside, Apple actually looked at using Linux for OS X in some form, and there's a story about Linus meeting with Apple and coming away very disappointed that Apple chose not to go with Linux for the underpinnings of OS X. Before the NeXT acquisition, Apple even started a project called MKLinux, which an effort to port Linux to PowerPC machines on top of a Mach microkernel. Remember that, at the time, Linux was not where it is today.

Linux kernel has the real multi threading that is required and seems to be good enough for the IBM 710 multiprocessor server you just mentioned.

IBM is pushing Linux heavily across the board, and that's cool. And yes, one of the big things they've done for Linux (as have others) is work heavily on its scaling for its big iron. And it's true that throwing OS X, as it currently exists, on a 4-way, 8-way or 32-way system would be far from ideal. However, currently exists is the main phrase here.

However, remember that right now, OS X is basically based on Mach + FreeBSD 4.x with a small smattering of stuff from FreeBSD 5. My understanding is that Mac OS 10.4 will now be moving to a full v5 userland tacked onto Mach, and in terms of SMP, the differences between v4.x and v5.x are like night and day. v5.x was a massive push, and in fact there might end up being some bickering, as some feel that v5.x isn't quite up to where v4 was in terms of quality. And of course now FreeBSD is pushing on v6...

Either way, while there are things in FreeBSD v5 SMP support that might not be ideal, it stacks up decently with what's out there now. And while I don't know exactly how Apple will be handling things, I believe they'll be improved to such a degree that at least in terms of the funnel situation, it won't be nearly the issue it is now. In other words, while this was one massive Hack, its a hack that will, at least for the most part, hopefully be going away.

And in all honesty, it's not just a drop-in fix, there's much to do in Apple's own software. Remember, it wasn't until 10.3 that Quicktime was supposed to be 'fully reentrant' in terms of being thread-safe, and you still can't do two exports within Quicktime Pro concurrently, no matter how many CPUs one has.

It's also worth noting that Linux isn't quite 'there' either and has its own problems it's going to have to deal with, although it's currently much better than Mac OS 10.3 and below. The 2.6 kernel was one hell of an achievement both in terms of how the scheduler handles desktop usage and scaling up to obscene amounts of RAM and storage, but there are major things being looked at.

Google on Linux + Kernel + SpinLock and you should come up with some goodies. Be wary though, you'll probably start diving into the differences between SMP models of FreeBSD, Linux, and DragonFLY... and that way madness lies.

It seems the BSD decision is holding back OS X.

Here's the thing about Mac OS X: its architecture from a bird's eye view is absolutely elegant. It really is. How everything is laid out, and compartmentalized in terms of the system is truly beautiful in a way.

However, when you start peeking inside those individual compartments, you can find yourself gagging at what's there. In some cases because there are obvious hacks, and in some cases because while OS X is fairly new, much of what's in it isn't. Apple has plenty of work cut out for it just improving and optimizing many of the Cocoa APIs. You would not believe how much optimization can still be done in OS X: everything from being smarter about what is updated on the screen, to just how badly an TSTableView can harm your CPU.

NeXT, for a good while of its later life, was not focusing a hell of a lot on the underpinnings, it was focusing on other aspects of its business model. Apple bought ten years ago, and then had to deal with porting over core Apple technologies, then Quartz, then creating Carbon. Large amounts of the APIs, both Cocoa and Carbon aren't exactly really thread-safe.

Remember, if you will, that while Apple has 64-bit hardware, and the kernel can address gobs of memory, GUI-based apps can't in OS X and won't be able to in Tiger, either. This means that if you want a GUI app to be able to use all that memory in your G5, you'd have to use a form of inter-process communication with a CLI-only background app.

There's plenty to do on their end, even while more becomes possible.

With IBM standardizing on Linux and Apple on the IBM processors it may be worth the effort to migrate.

This part could easily turn into one hell of a long post, so I'll try to keep it simple for now: IBM focusing on Linux is good for Apple indirectly, but don't expect OS X to start showing up on IBM hardware, nor for Apple to be shipping quad-CPU+ boxes any time soon.

Also, a small note about HyperThreading: it doesn't exactly come for free. While the kernel needs to be being fed multiple threads at once, the kernel also needs to be aware of HyperThreading. Basically, while you'd have serious problems getting anything good out of it as it currently stands, the kernel does need its own work done.

Although I did note that Apple introduced support for HyperThreading into Darwin for x86 machines over a year ago...

 Posted by drunkenbatman
    January 28, 2005, at 05:02 PM