Apple and x86 Questions, Part 2

I was able to chew through a few more questions tonight. Endians (aka, one of the most boring subjects known to man), the coexistence of 32-bit and 64-bit, and cheap macs. It's approaching 4am now, so consider this post to come with zero warranty, but if something is off let me know and I'll check it out once I've slept and my functional IQ is back above 30.

I thought the switch to x86 was unlikely because of endian problems, and it seems we're going to be paying for that for a long time to come because the OS must now be endian-agnostic which means LOTS of byte order swaps. And of course the developers have to make extra system calls and do more debugging. Nice going, MacTel. Didn't it occur to Intel to support endian switching at some point the last 30 years?

But the thing that really worries me is what all of this means for the 64-bit road map. The low power Intels that Apple is obviously after don't support 64-bit at all, AFAIK, and the Intels that do or will, can they run 32-bit processes and 64-bit processes side by side like the G5 can?

Iljitsch (whose last name is also insane)

I never completely understood this whole endian-killer-of-platforms business, and wherever it got started I think it got blown way out of proportion. Yeah, it's there, and it's an issue, but it's nowhere near insurmountable or even that big of a deal for the majority of applications out there assuming they will need to be recompiled anyways.

Now, brace yourself, because Endians are possibly one of the most boring subjects known to man, and I'd wager its used as a catch-all just because not enough people get what's going on. For the purposes of this, we'll try to alleviate the latter.

According to a book I read once, 'Endian' comes from the old novel "Gulliver's Travels". I'm going to have to trust my memory of that book, because I never read the referenced book, but do have a distinct memory of sitting in front of the TV with a bowl of Lucky Charms watching a cartoon of it as a kid. If you'll recall, our protagonist, Gulliver, gets in a shipwreck and ends up on an island called Lilliput.

Lilliput is full of Leprechaun-sized people (For the record, I have no idea how big a Leprechaun is supposed to be, nor whether it should be capitalized) who tie him down with little ropes while he's out of it. I know, we've all been there. Anywho, Gulliver comes to find out that the Lilliputians have been at war with their neighbors over how an egg should get cracked open.

Specifically, which end should get cracked -- the little or the big -- and at some point in the early 80s while the byte-order wars were starting to rear up, a hacker looked at the situation, thought of the war over the ends of the eggs, uttered this, and it stuck.

cow icon i need to replaceThe Cow says this is actually a little weird when it comes to the PowerPC, which is considered to be bi-sexual, or in this case byte-sexual, meaning it is 'endian-agnostic'.

The PowerPC, at runtime, can deal with a process in either little-endian or big-endian mode. Theoretically this could get fairly weird, and I'm not aware of it being used in any real way, but theoretically someone could run the OS as big-endian and a process as littl-endian. Just because I'm not aware of it doesn't mean it's not used pervasively by the mole people, so who knows.

Also, this is going from memory, and I don't know why the PowerPC has this ability or the original thought behind it, just as I don't know why the PDP-1's and such were middle-endian. Don't even worry about what middle-endian looks like: Suffice to say it looks really weird and I have no idea what possessed them to create it.

Now, what gets a little weird is that the PowerPC up to the PowerPC 970 is open-endian. All 64-bit PowerPC chips currently in existence are big-endian-only, and my guess the 970's lack of open-endian has to due with its POWER pedigree and the fact that few knew the feature existed in the first place.

Basically, this 'endian' business has to do with how 'words' of data (the basic unit of storage on a computer) are stored and accessed: RAM, files, etc. Macs are 'big endian' and PCs are 'little endian'.

This means if you write data using a Mac, you'll end up with a file that's storing its data in a specific sequence that expects you to start at one end of the file. If you magically dropped it on a PC, it would try to open the file but at the other end of the sequence and see what looks for all the world to it to be gibberish.

Think of it as how English reads left to right, but other languages read right-to-left. To be really clear, on a Mac (And SPARC, MIPS, etc.) you have this:

  • aa bb cc dd ee ff

And on a PC (And Alpha, I think), you have this:

  • ff ee dd cc bb aa

On a PDP-1, you'd see what looks like a jumbled mess.

So on a big-endian platform, you start off with what is most significant, and on a little endian platform you end with what is most significant. To put it another way, consider the number '1024', or '1,024'.

The 1, denoting 'thousands' is the most significant and its on the left on a Mac and is more intuitive to people. If you were telling someone the number out loud, you wouldn't say "24 and one hundred and one thousand", you'd say "one thousand and one hundred and twenty-four".

On a PC, it would be '4201', or '4,201', which means the exact same as the above as long as you know it's stored in little-endian format. If you are working off that number as though it's big-endian, you are in for a world of pain.

To give you an example, let's take the file format .BMP, which was birthed on x86 and insists you write it out as little endian. If you're writing out this format on the Mac, you need to first pass it through a function to reverse the byte order. If you're going to be reading a .BMP file on the Mac, some bit of code is going to have to reverse the byte order yet again when you try to view it. At the same time, you can/could write out .TIFF files as 'Mac TIFFs' or 'PC TIFFs', the difference being the byte-sex of the file. This is an example, as not all file types store data this way, but with .BMP and .TIFFs it's an issue.

If you've worked with graphics awhile on the Mac, you've probably opened .TIFF files that were formatted for the PC and -- you guessed it -- those files were sucked in through a bit of filter code. 'Big endian' is also known as 'network order', because it's the standard for sending data via sockets. I.E., if you're a Windows developer dealing with the network at a lower level you've had to deal with TCP/IP, which sends its data out with the most significant byte going first, which is 'big endian').

If one didn't take it into account, one'd run the danger of the equipment you're connecting to not having a clue as to what you were trying to send it. At a higher level, the APIs will generally convert the IP address on say, a Windows machine to 'big endian' before sending off the machine-readable value so that the router or gateway -- which are generally big-endian -- wouldn't get it backwards.

What's strange about this whole endian business is how people have gotten almost attached to their platforms byte-sex. There are arguments to be made for both that I'm aware of, where one might say big-endian is more 'human readable' and little-endian is more 'machine readable'. The idea being that the CPU crunches numbers starting with the least-significant digit, so you can shave off a small bit of work with little-endian. However, in practice it's not that big of a deal.

The one thing that comes to mind is that little-endian machines have to do some converting before passing things off protocol stuff to the network, but the impact of this is now so minor most aren't even aware it has to occur.

My point in dragging you through this is that Endians are just a fact of life in the computer world, and aren't some magical road block between platforms. While I'm simplifying some things, it's not rocket science, and for the most part it's a completely trivial, barring a few exceptions, because there are always a few exceptions.

Now, by trivial, I mean it's not major brain work, it's just annoying grunt work. However, the great thing about annoying grunt work is that once you do it once, software generally makes it fairly trivial to automate, and yes this is the type of thing that can generally be easily automated or filtered via software.

If you're having to break out a major binary format on your own it's a chore, but once you've done it doing it again is easy. If you know lots of people are going to have to be doing it, going down another layer in the software and making it 'automatic' is a chore to do but transparent once it's done.

In fact, OPENSTEP had software in it that swapped the order of datatypes to keep things running smoothly across the platforms it supported (i386, RISC, etc.), which didn't all use the same byte order.

Rosetta will have similar auto-swapping built in, so if you are running a PowerPC app in emulation and copy some text to the clipboard, then switch to say Apple's x86 version of TextEdit, Rosetta will filter the data first, reverse the byte order, and give TextEdit what it is expecting.

Now, it doesn't mean there still won't be issues. If you have an app that is doing things in a custom way, you could well see problems. For example, if you are just writing out a file, Rosetta is smart enough to know you're a PowerPC app and handle the Endian stuff. However, if you have your own custom virtual memory solution for some reason, Rosetta will be pretty confused. If you have rolled your own instead of relying on the built-in clipboard, you're going to have some work to do too.

If you're using Java, most of this stuff is abstracted for you already, and there are actually similar issues when dealing with 32-bit and 64-bit operating together; I.E., things don't play well in this area unless each knows what the other is and you've added in some glue. Games are one of the most-ported types of software for the Mac, so that could well be where some of this got started.

Sure, having to deal with porting over the binary formats can be an issue, but if someone has been doing it for long enough they've got some libraries for it, and its more about nailing down the specifics of the particular format. With modern games, a much bigger deal is how dominant Microsoft's DirectX set of APIs has become, both because they're very good and they're very pervasive.

If you happen to be using something XML for storage, which uses keyed values, none of this will be an issue for you since you will have to recompile anyways. If you're using something like Core Data, I'm sure it's abstracted also and is probably another reason for developers to give it another look.

Now, while I am saying that in the grande scheme of things endianness is not that big of a deal for a transition like this, it's worth noting that for some people it will be a huge deal as we don't know some of the specifics involved. Things like Microsoft Office are coming to mind, but that would be misguided because those formats are little-endian-native, and Microsoft is having to already filter them on the Mac. Going to x86 just means removing a filter for some things, etc. As a small bit of trivia, I beleve WordPerfect's format is big-endian.

Still, there are developers out there who are going to be at the center of a cluster fuck because of this, and while I hate to be using Adobe's name so much lately, but I can imagine they're in for a world of pain specifically because of how abstracted they are from much of Apple's 'native' APIs. This can be confusing, but Adobe has some really custom stuff going on -- which generally works great for them as they've often pushed the envelope -- until things like this come along.

I don't know how much of this will be handled for people using things like Metrowerks, and while you can say those people should be using XCode anyways, those people are using what they're using for a reason. If they could compile for Windows using XCode, I doubt they'd have much of an issue, but there you go.

It's probably also worth noting that due to some of how I've been portraying things so one could grok it, you could well just be thinking of this in terms of files on a hard drive, but it really goes a little beyond that. I.E., if you're storing things in OpenGL while you're working with them, many of the data types are endian-specific, so a developer has to comb through and work some of that out by filtering them through a function.

Still, of those developers who are going to have major problems, endianness is probably one of their smaller worries. Moving on, the guy whose name starts with an I also asked in an adamant way via email and comments:

But the thing that really worries me is what all of this means for the 64-bit road map. The low power Intels that Apple is obviously after don't support 64-bit at all, AFAIK, and the Intels that do or will, can they run 32-bit processes and 64-bit processes side by side like the G5 can?

The short answer to your question is that yes, it's entirely possible for one to run 32-bit and 64-bit processes side-by-side on both AMD's and Intel's 64-bit chips. There's nothing precluding Apple from having a fully 64-bit version of Mac OS X running on Intel's EMT64 while running 32-bit processes (apps).

cow icon i need to replaceThe Cow says if all these new names can be confusing, so here's a quickie review.

The PowerPC has is own set of instructions, or instruction set architecture (ISA), which are like building blocks of instructions built into the CPU for the developer to put together to do what they need.

While you have the PowerPC ISA, Intel had IA-32, which is what we normally think of when we talk about x86. Intel also created IA-64, aka EPIC which is completely different and used on the notorious Itanium (as a side note, while EPIC has fallen on its face many times, its still highly cool). EPIC had a sort of emulation layer for 32-bit code, but it was extremely slow.

Instead of creating something completely new, AMD went the other route and tacked 64-bit goodness onto its existing IA-32 cores, known as AMD64 or x64. It shipped it originally in boxes destined for servers with the Opteron, and was able to get Linux support fairly quickly. It was doing really well, and Microsoft joined the bandwagon, and somehow Intel was convinced to adopt AMD's extensions... EMT64.

For the record, while Microsoft may not be shipping a new OS every one or two years, any geek has to be impressed that there are versions of Windows running right now on completely different architectures: PowerPC, IA-64, IA-32, and EMT64.

They could theoretically have a system very similar to what they're doing right now. We just don't know what option they'll choose or what they're looking at, but most things are possible, although it's good to note that 'possible' does not always mean it's elegant. Right now, there is little documentation for a lot of this stuff, and at least while I was looking into driver issues, you get the impression that much of it is in a state of flux right now.

To my knowledge anyone with an 'Developer Transition System' trying to build 64-bit versions of their apps is finding things are still wonky in that area. So, we have a small problem here. The answer to your question by anyone except someone 'in the know' at Apple is going to be: "Fuck if I know, but..." because what is technically possible isn't necessarily what Apple will end up doing.

Going back to your first question about what Intel is shipping right now, well, Apple isn't shipping x86 boxes right now, so what Intel is shipping right now doesn't matter all that much. By the time Apple is starting to push product, Intel's more power-conscious lines will have 64-bit goodness baked in.

Much of their product line already has it, it's just not really being taken advantage of by consumers because WindowsXP x64 doesn't have fantastic driver support right now because it requires kernel-mode drivers to be 64-bit. A lot of the 'big name' drivers are there, but many that make up the all-important long tail of the Windows base aren't. Some of the things Microsoft is doing with WindowsXP x64 will probably be similar to what Apple does, as some of the same limitations are there: like libraries and plugins being the same bitness.

To give you an example of how you have a limitation, and some of the above, you could look at plugins. Plugins are a way to modularize your app, and if you so choose, let others extend it.

A lot of developers make heavy of use of them, but in Mac OS X as it stands, an app and its plugins and such need to have the same bitness or things go wonky. While a developer who is using plugins to have more modular code can update all of his, if there are third-party plugins things could get bad.

So if you want a 64-bit app, you're going to require 64-bit plugins, and that's generally the approach Microsoft is taking for x64, but there are ways around it. You can write glue in your code to allow for it, or you can spin out a little helper app to interact with the older plugins. There are generally workarounds for all this stuff (like there are for say, endian issues) where the onus is on the developer, but you never know where Apple might step in and put most of the onus on themselves.

For the record, I wouldn't conclude from earlier that Mac OS X on Intel (I'm still not ready to say MacTel or MacIntel) will also require 64-bit drivers, or really conclude anything. Again, we just don't know what Apple has up it's sleeve or how it's going to proceed. Or more specifically, I don't, and I don't mind admitting that. I could lay out a bunch of guesses and possible options, and even weight them in order of probability, but that'd be wanking to the point of chafing.

Besides, my track record here isn't very hot. If asked, I would have said the possibility that there would be virtually no way to navigate around Dashboard via the keyboard to be about 5%. Instead, we got:

  • 'F12' - Open/close the Dashboard layer
  • 'Command + =' - Show/hide widget bar while in Dashboard
  • 'Option + hovering' - Allows you to close a widget without opening the widget bar first

In my defense, I was working off the idea that any usability testing would have shown requiring someone to mouse down to a tiny little arrow on each side of the screen to see your widgets would have shown it was stupid, and adding keyboard shortcuts that require you to go move your hand back and forth between the mouse and keyboard in succession to get anything done would be close to rounding the horn towards fubar.

If asked whether an environment like Dashboard, which was so obviously mouse-centric, wouldn't let you scroll the widget bar with the mouse, I would have said 1%, because one has to allow for the will of Jobs God in these things.

Which brings us to our last question for today...

People say PowerPC chips are cheaper than Intels. But just going to Best Buy the computers are MUCH cheaper than Apples. Dell has a dual core pentium for a third less than Apples single G5 Powermac. Will Macs be more or less expensive now that they are using Intel?

D.G.F

Don't expect cheap Macs from this. The price might fudge down a little, or it might go up a little, but much of it will probably be around the same. Apple might pay a little more here, but it'll save some there, and it'll probably end up fudging out to a wash. However, I would expect that in many respects (especially the low and mid-range) what you are getting in that wash will be more comparable to the technology offered by its competitors than it currently is.

Historically, Apple just hasn't done very well when it is playing by the same rules as everyone else, and if there is one thing to note about the iPod it's that it consistently tries to change the rules to its advantage. However, there isn't going to much it can really change here.

In x86 land, you don't update your hardware only once a year (or less), so that aspect is going to get very interesting. During the clone years you had a sense that Apple was almost befuddled by these crazy cloners rolling out the latest and greatest as soon as they could get it into a box and using their nimble nature to undercut Apple in price, performance and quality.

This is something I'm curious to see play out, as things are going to come to a head when the specs of Apple's hardware and that of its competitors will be almost exactly the same. Right now, a lot of the 'Apple Tax' is fudged in a mistique of its non-standard hardware. That'll be gone, and if Apple has slower components than its competitors but is more expensive, well, it's hardware will be "Slow and expensive" but "Pretty and runs OS X".

You might say "But wait drunkenbatman, Apple machines already use many of the same components as x86?" and you'd have a point, but that's psychology for you. Mark my words, if the 'PowerMac G6' has half the drive bays of machines half the price, the derision will be much worse than it is now.

Let's hope the machines are very pretty, very functional, and Mac OS X is very, very good.

And with that, I go sleep.

yummy alcohol posted button Posted by drunkenbatman
    June 27, 2005, at 04:09 AM


Comments (25)




Post a comment



Anonymous comments are allowed, but please enter something for a name.

And do endeavor to appear sane.









Remember personal info?