DrunkenBlog

The High-PPI Problem (Redux (AKA, notes on CoreImage))

Back in "The-High PPI Problem" I went into what resolution independance was, why it was important, and what was needed to get it that we don't have.

The biggest thing I went on about was accelerated drawing, namely that it didn't look we'd get it for 10.4, and so it didn't look like we'd be anywhere close to resolution independence for quite awhile. I've been meaning to revisit that...

Awhile back, one of the guys at Apple started leaking out that "After this, we're going to have our releases more spread out... this isn't sustainable", or something like that. Most people took that to mean after 10.4, but as the WWDC rolled around, it came out that 10.4 was going to be pushed back quite a bit.

We're not looking at yearly updates now, rather two years, and I wouldn't be surprised if 10.5 has an even longer development cycle. I'm not necessarily against this, as it has its own inherent advantages for developers and consumers, but it does mean that cool bits of candy are going to come further and further apart in the Mac world.

This means that if there is something you really, really want, like resolution independence, if it doesn't make it into the next version it's going to be a long time till you do see it. It also means that developers will (hopefully) have a less fragmented user base to target, and have longer to take better advantage of what is there. Trade offs.

I still don't think we'll see real resolution independent displays in Mac OS X until 2007-2008 now. However I was kindly informed, and in some cases not so kindly informed, that Apple was working on the problem and sewing the seeds.

That was intriguing, and I started shaking lots of trees and picking up lots of cocktail napkins. I found some really interesting tidbits, and it looks like may indeed get HW-ACCL (hardware accelerated) drawing for 10.4. I haven't been specifically told this, and haven't touched a Tiger build, I'm just connecting some dots.

To go back a bit, at the 2004 Worldwide Developers Conference (WWDC), Apple kind of muddied the waters a bit. Most people have forgotten all about the expo specifics, as the keynote kind of sucked. My notes on this are here, as well as some further bits on Quartz and Longhorn.

To recap, Apple showed off two new things: CoreImage and CoreVideo. The CoreVideo demos showed off real-time video filters being used, with the processing being offloaded to the GPU. The CoreImage demos showed off what were dubbed 'Image Units', which were sorta like hyper-accelerated Photoshop filters, only they were System-wide.

Basically, I think two things went wrong here:

• The CoreImage demos just sorta sucked; they were nice, but how things were explained and what was shown off was... lacking compared to where we know where other things are heading. Admittedly, you only have so much time in a Keynote, but going back I get the impression the part in the Keynote devoted to things like CoreImage seem tuned to trying to create pressure on Adobe and others to get in line, rather than to really give you an idea of what's going on.
• Going by the marketing terms CoreImage and CoreVideo, I think I got a little mixed up by contrasting them to how the other service works that has the same moniker: CoreAudio.

Trying to extrapolate out CoreImage and CoreVideo by their likeness to CoreAudio was probably a mistake (if I'm right about some things). Trying to extrapolate out what CoreImage really was from the keynote was probably another recipe for disaster.

I've gone into more detail about this before, so we'll keep this fairly basic. Most people have an idea of what Aqua is, it's the 'look and feel' of Mac OS X, except for that whole metal thing we try not to talk about. Quartz is the moniker for technology that allows that to happen, and replaced QuickDraw from the classic Mac OS.

Basically, with the way things worked before, an application would just spit whatever it wanted the user to see onto the screen. QuickDraw was a series of routines built into the Mac OS for doing this, and over time the companies making graphics cards built in routines to make things much, much faster.

If you were going through them, that is. Most things did use the QuickDraw routines for getting stuff onto the screen for just about everything, while spitting things up onto the screen was fine, and very fast, but there were limitations: applications didn't really know what other applications were doing. You were just spitting your stuff to the screen, and when another application that was in the foreground spit its stuff to the screen whatever came before just got covered up.

If you wanted to go further, like making your window to be transparent, you'd have to go through a whole mess of trouble to figure out what the screen was supposed to look like, decide what you wanted it to look like, and then spit that onto the screen. This is entirely possible, it's just not very elegant.

If you were a company like Adobe, and had products like Photoshop, Illustrator, etc., this was a bit of a problem. In order to do what those products need to do, they need a much better idea of what is what so they can work their mojo.

What they ended up doing was creating a separate, internal engine for keeping track of what is what and then tying it to QuickDraw to spit the final output onto the screen. This let Photoshop know that layer one was a butterfly with an alpha channel, and layer two was a flower while it was working on them. What you're seeing on the screen is a representation of the groups of data Photoshop is holding onto, and the options you set determine what algorithms get run between them to produce the final image.

If the user turns off the display of the flower, Photoshop then takes its basic checkerboard pattern for a layer with nothing in it and runs a routine to composite them together. When it hands that image off to the display routines, in this case QuickDraw, the user benefited from having those routines accelerated when they moved the image around.

Up, down, left, right... the routines were basically moving a final image across the screen. However if you changed what that final image was, like say running a filter or turning on the background image again, things would have to be composited into one image again.

There was hardware acceleration, as anyone who ever tries to move things around the screen on a machine without it will attest to, even today. If you've ever set up a Linux box and it happens to have some weirdness with the graphics card, hardware acceleration is usually the first thing to go and things on the screen are slooooow until that magic setting is corrected. Then they're yippy skippy.

However, in the context of an engine like Photoshop's, that HW-ACCL was limited to moving final images around and things like scrolling; the real heavy lifting of putting everything together was being done by the CPU.

Quartz, coming out of its NeXT heritage which had something called Display Postscript, was an effort to bring this type of capability to the entire OS as a whole.

Applications no longer drew directly onto the screen, but would instead draw to a window you never actually saw - a buffer. Then the Window Server application would take those various windows you haven't seen and composite them into one cohesive image that would then be displayed.

Given the Photoshop example, this should sound familiar and the benefits were very much the same - the Window Server knew what window was what, and could just apply some algorithms on the buffers to get the final output it was after. Examples of these might be making one window transparent, or even drop shadows. In order to create a drop shadow around a window, you need to know not only where one window ends and one begins, but what overlaps what, etc.

Again, all of this is possible by 'brute-forcing' things, and yes an OS like Windows has add-ons (and even some built-in stuff) that allows one to achieve the same effects on that OS. So does OS9. However, something like Quartz becomes transparent to the developer and means they don't have to reinvent the wheel (among other things).

It is also much, much more elegant and cohesive, with things like 'tearing' (where you see the remnants of updates as things move around left on the screen) becoming a thing of the past. The big thing to keep in mind is that no longer is the application itself responsible for drawing what it needs to to the screen, but rather it just draws to its window when it needs to with the appropriate meta-data and the Window Server takes care of the rest.

Unfortunately, if we go back to some of what our brains picked up from the Photoshop example, it becomes clear that there are some downsides to all of this elegance. Namely, you lose all of the acceleration, and everything becomes CPU-bound.

I'm not going to go into the whole MHz Myth crap, but suffice to say that while there is truth to it, it often works both ways and it doesn't tell the whole story. And there is no getting around the fact that Quartz put a massive load on the system, especially at a time when Apple wasn't shipping machines that had a lot of CPU power to spare.

Most users, if they even check in at all, see what's going on with their Mac via the 'Activity Monitor' program. Nothing wrong with that, it's a great tool, but you get used to thinking of your computer in terms of percentages. I.E., that iTunes is using 10% of your CPU.

If you run the CLI app top you can see that information, but you're also presented with 'load' statistics, such as 0.1, 1.7, 2.0, etc. While 100% is a little easier to imagine, it's worth wrapping your head around this.

In a multi-tasking OS like OS X, there's a master scheduler that keeps track, and decides, what gets run at the CPU at any given time. These routines are fairly fine-tuned, but basically work by creating an arbitrary unit of 'CPU time', and then slicing it into slots. Processes that want CPU time request it, and the scheduler gives them each a number and tells them to stand in line.

It isn't quite a first-come, first-serve deal, and all processes aren't considered to be equal. The scheduler decides what processes get chunks of CPU time based on a variety of factors, like how long it's been since they last got some CPU love and what their priority is.

I.E., all of these processes have been drinking heavily all night, and are all lined up in a queue to go to use the loo. Those that have been tagged as higher-priority processes (VIPs) start farther ahead in line than those who just walked in off the street, but the scheduler does keep track of who has gone how many times and makes sure everyone gets there eventually.

Unfortunately they're only allowed a certain amount of time in front of the urinal, at which point they have to pinch it off and head back and request another number until they're completely satisfied (I.E., the process/app has quit or is idling).

The scheduler is remarkably smart at what it does, but its options are limited when it's faced with a marked shortage of urinals for those standing in line. If the number of processes needing to do their business vastly outnumbers what's available, things are going to get backed up and that line is going to get really long (sadly, computers don't have alleys). You experience this as the computer feeling 'slow'.

Going back to those numbers you see in 'top', a 1 is sort of like 100% for that queue. If your Mac says 0.5, it means your CPU is only hanging out at 50% capacity. If it goes over 1.0, like say 1.2, it means your computer has pushed beyond capacity and it's having problems getting what's asked of it done in real-time. If it says 60 or 100, your machine is groaning like Paris Hilton in a firehouse.

Things are getting done, and will get done, but to the end user they might seem to take quite a long time. You see this whenever you're waiting for a Photoshop filter to complete, or whenever it seems like it's taking forever for iPhoto to resize.

Quartz really did introduce a ton of load onto the System, much of which you can see directly by watching the Window Server application I mentioned earlier via top or any GUI tool. If you start doing anything on the screen, like say, moving a window it's not unusual to see it spiking up to 10%, 20%, or even 50%+ of your CPU depending on your hardware (for gawd's sake, don't shake a window back and forth, it's murder on the WM).

This has a direct impact on the user, in that if the queue for the CPU is filled with stuff the Window Server is wanting to do there's less time for anything else, and other processes start to lose out. Things slow down. Sometimes way down, to where every process is losing out to each other that the machine is 'chugging'. And and all drunkers know, chugging just sucks unless it actually involves liquids.

Occassional spikes in CPU here and there shouldn't seem like that big of a deal, but there are a lot of them. The damnable bouncing dock icon is an example, but so are things like spinning barbershop bars. As an example, it freaks me out how much CPU load Safari can cause by having its 'downloads' window open.

While there are things you can do to make 'Quartz' more optimized, such as making it smarter about what it updates and when (which has started, but isn't nearly where it needs to be), there's just no getting around the fact that everything was pushed onto the CPU.

I should quantify exactly the type of hit we're talking about... millions upon millions of calculations to do trivial things, like moving a window. Each pixel on the screen is basically four values (Red, Green, Blue, and Alpha which is needed for transparency) and any time a window is moved not only does that window's pixels have to be dealt with, but anything in front of or behind has to be taken into account to.

This isn't trivial, and while beefier hardware makes a difference, you'll still wonder what the hell iPhoto is doing that allows it to be capable of spanking a dual G5. Altivec optimizations help also, and you can really tell the difference between a G4 and G3 system when it comes to the 'snappiness' of the UI. But it's still nowhere where it needs to be.

However, getting it back off the CPU and onto the graphics card is a little more difficult than it was the first time around...

There are other aspects that impact the user, like memory usage which I'm not going to go into that great of detail because at some point I'll probably do a PCI-Express post, and will cover a lot of things then in a more technical way. However, since it did impact the user in a big way, it's worth covering in a cursory way.

In a system like Quartz, every window is using up quite a large chunk of memory. I mentioned how every buffer on your screen is composed of pixels, each RGB+Alpha. Quartz has to keep all of those pixels in memory, and generally keeps two copies: one to show, and one that is being drawn to. That way the user sees the drawing 'when it's done' and not as it's happening.

Each pixel is 32 bits of data, because each channel is 8 bits. I.E.:

• Red: 8
• Green: 8
• Blue: 8
• Alpha: 8

If we use a typical iMac as an example, the 17" model has a resolution of 1440x900. Multiply those together, and you get the amount of pixels on the screen: 1,296,000.

If we multiply that by 32 bits, we get about 4.95 Megabytes. But wait, remember we said we said the Window Server is often dealing with two copies of each window, the final one it keeps around to do its work and a background one that is being drawn to to keep things nice and tidy.

And yes, even if you are just showing the 'desktop', that's a window. So we need to double it, for 9.9 Megabytes. This isn't even taking into account actual application windows you may have open. This is rough, and there are other things going on, but you should have the idea.

Start opening up a bunch of windows on an OS X system, and watch the Window Server's memory usage. It can get kind of frightening... It doesn't really matter what it is, how much memory is sucked up is directly proportional to the size of the window.

Now, as of today, 13 out of 17 computer models Apple current sells (seriously, they do still sell computers) ship with 256 Megabytes of RAM standard. It wasn't that long ago that it was 128. In fact, when it comes to Apple hardware, you have to spend at least $2,500 to get a machine with more than 256 Megs of RAM.

The examples we've gone through in terms of the scheduler getting backed up pale in comparison to what happens to a unix box when it's entered 'swap hell', but it's not really appropriate for this and I'll save that for my big post on 'funnels'.

There's been a lot of talk lately about hardware-accelerated 3D interfaces and the like, everything from what Microsoft is doing in Longhorn to what Sun is doing with their latest on-again-off-again trists with Linux.

Usually a Mac user will chime in with something along the lines of, "Yeah, the Mac already has that", which isn't really true, at least not in the way the person who usually says it is thinking of it. With Mac OS 10.2 (AKA, Jaguar), Apple introduced 'Quartz Extreme', which brought the beginnings of hardware-acceleration... but only to one part of the process: compositing.

If you go back to the examples we went through before in your head, this is where the Window Server takes the buffers that have been drawn by various apps and slaps them together to get a final image. That image then gets thrown out frame buffer on the video card and is displayed on the screen.

With Quartz Extreme, applications draw as they normally do, to the buffer. However, when it is time to composite everything together the WM is able to offload most of that step to the GPU, which then sends it off to the screen. Sort of. It basically slaps the window onto a polygon, and sends it off to the graphics card with some OpenGL commands telling it what to do.

This is provided it's on a machine with enough VRAM, a fast enough AGP interface to handle the bandwidth, and a card that has some special features. Sort of, and we'll come back to this a bit.

This is really, really cool. However, going back to those examples, you'll see that there are certain aspects that are benefit from QE more than others... basically the stuff in the last step. Like transparency.

There's a lot of transparency in Mac OS X. Even in an opaque window, you still have the drop shadow surrounding it that interacts with everything behind. If you drop down a menu, it's subtle, but there's some transparency... and a drop shadow. Personally, I remember when Apple released the version of Mac OS that finally allowed you to drag a window by its borders, and had a bunch of UI info derived from studies they'd done showing how much more efficient it was versus only being able to move a window via the title bar.

Before, when you'd take a bunch of transparent windows and pile them on top of each other, the Window Server would, as we mentioned, have to do a bunch of calculations between all the pixels. You could reach the breaking point really, really easily. God forbid you had anything moving that meant it was constantly having to run those calculations.

Offloading that compositing step means transparency just isn't as big of a deal as it was before. It reduces load on the machine by allowing those CPU cycles to be used for other things.

No, it's not the whole story, and you see that in a big way if you start moving windows around on a QE-enabled box and watch the CPU usage. But it was a huge step forward. There were a few other things that came in 10.2, like window compression, which allowed the Window Server to categorize a window as dormant/inactive and run a compression algorithm.

It might seem a little wiggy to do this, but on the whole it was a win because it wasn't that complex of a calculation, and it allowed memory to be saved... especially memory going back and forth over the bus. And the G3 and G4 weren't exactly known for their bus speeds.

Names on the Mac have gotten a little confusing lately. Quartz became Quartz 2D, not to be confused with Quartz Extreme. CoreGraphics is now sort of CoreImage, Rendezvous is now OpenTalk, Titanium is out, Aluminium is in, and everything cheap is plastic. Well, except for those displays, which while they were plastic for a long time, are now metal.

Marketing can get a bit of a bad wrap, and it's not undeserved. The technical definition is making someone aware your product is even available and suits their needs, but a more 'modern' definition has become saying anything you can get away with to get someone to part with their cash.

I.E., marketing does have its place. If I've just created the coolest widget in the world, and someone down the street is pining for a widget that does something they need to do but can't find it, no one is winning. But when people are faced with the problem of trying to convince someone that is something they have to have, things start to get a little more fast and loose.

When the Titanium Powerbook came out, everyone ooh'd and aaah'd about it... it was titanium and wow that must be durable... after all, it's used for airplanes, race cars and golf clubs (golf is pretty upscale, so if you want to see what monikers companies will be snapping up, watch golf).

And it was a Titanium Powerbook. Sort of. Just not in the way most people thought of it. Most laptops have a metal frame, often magnesium alloys and other such things, which reinforces a plastic enclosure. The Titanium Powerbooks were basically the same, except instead of plastic, it was a super, super thin sheet of titanium which was then painted to give it a color most people expected Titanium to look like.

And by thin, I mean really thin. The first revision had so much flex in the case that if you picked it up with one hand, chances are the battery would disconnect and the computer would turn off. The fix for this, after you sent it in, was to put a strip of rubber alongside the battery compartment. I'd get all sorts of weird looks when I'd recommend the lowly plastic iBook for someone who cared about durability, because, well, the Powerbook was titanium... but I digress.

If you come out with a new version of something, ideally you want someone to buy it. If they own 10.1, and now the company is offering 10.2... it might not seem to be that big of a deal, because you already have 10.1. But do you have jaguar? Or the Panther edition? That's easier to lust after than just next year's model, which is likely to be incremental.

This gets exacerbated when you have already created a moniker because it's really hard to get people excited about a technology named IEEE.1492.14.

Like, say, Airport. If someone already has 'Airport', do you call it Airport 2? The idea is you want something that promotes the idea of it being so much better than what came before, and to create an image in someone's head, and you often try to piggy-back on other people's marketing.

I.E., if people are hearing lots of commercials for high-end things that say titanium, you can call your product the titanium edition and because of our collective pop-culture unconscious you have an idea that it's the cat's pajamas. If there is 'Airport' and 'Airport Extreme', you have an instant idea of what is better than the other... and it's not just better, it's extreme.

For awhile, just about everything had an 'X' in it, because it was just so... Xtreme. Then you couldn't close your eyes and pick a random page of a computer catalogue without finding something with 'Titanium' in the name. And then there's 'Ultra'. Sometimes they even combine them. I know I've seen "blah blah Ultra Extreme", but not yet the dare-we-try-it "Titanium Ultra Extreme" edition.

This usually continues down the chain of products until $12 skateboards with absolutely horrid graphics have bastardized versions of them slapped on in garrish fonts, at which point something new has to be found and the cycle repeats anew. When no one can think of anything catchy, they go back to numbers and technical terms.

There are some unwritten rules that follow the collective unconscious, and one is that you can't grab back a term before it's gone through it's cycle without looking lame... because people will associate them with $12 skate boards.

And gawd-damnit, do I hate the word extreme in products at this point, and I hold a personal little black nub of resentment towards Apple for making me type, let alone say it, in actual company. It's just so... lame. Someone, somewhere should have been able to come up with a better moniker than 'Extreme' for Quartz and Airport. For another example of how this can go bad, please see '.NET'.

*shivers*

Luckily, this last shift to 'CoreImage' as the enclusive marketing term should hopefully mean that the term 'Quartz Extreme' will soon be wiped from the general Mac vocabulary... although it does make one wonder what they'll do with Airport when Wi-Max starts hitting.

Mac OS 10.3 slinked into release in October of 2003 (my old, and late, Panther impressions are here), but in terms of what we're discussing today, there wasn't anything major to come with Panther.

There were some refinements to Quartz Extreme, and some very limited refinements to how Quartz behaved on slower machines, such as what was transparent and what wasn't... but you'd be stretching really hard to call it scaling. Honestly, I was disappointed in this area.

Cards supporting pixel shaders and such were already on the market, even for the Mac, and figured we'd see HW-ACCL drawing at this point. We didn't, but I'm now hopeful we will see it with 10.4. If we don't, I can tell you I'll be unpleasant about it.

Anyways, since I've brought up drawing again...

We talked about what Quartz (*Gag*) Extreme takes care of, but we didn't specifically say what it doesn't take care of: drawing.

Drawing was a bit of different thing in the Mac OS, as if you used QuickDraw for graphics, acceleration for those types of operations were built into the card, and the driver just had to do some magic.

Again, with Quartz, all this changed. As a quick review, since the term 'drawing' can be a little vague, but it's basically the process of the app getting its internal data onto the screen.

They can use QuickDraw for this (yes, still), or they can use CoreGraphics, or even straight into OpenGL. Once they are done pumping their data into one of those three APIs, those technologies (QuickDraw, CoreGraphics, or OpenGL) have a final image which is then passed onto the GPU, which then composites them into one big image, which is then dumped to the frame buffer, which is memory on the graphics card set aside to hold what's going to be slapped onto the screen.

Whew. (pause for breath)

The idea with HW-ACCL for drawing is basically the same as before, where the app passes off its data to a set of routines (QuickDraw or CoreGraphics or OpenGL). Except instead of those routines crunching through what they need to do to get their image of the window via the CPU, those APIs would then also hand it off to the graphics card to do it.

Load on the system goes way, way down and ideally the performance of drawing would increase too - it's dependent upon the video card now.

There are some complications, like the fact that when it comes to 2D drawing, Open-GL just isn't very precise. This isn't something you encounter while playing a game, etc., but for a lot of general purpose stuff, the fractional precision just isn't there.

This can get worse when you're dealing with different implimentations and different cards... a pixel is a pixel is a pixel, until you're needing to deal in fractions of a pixel. Then things get messy.

The big break in this came several years ago, with GPUs (graphics processing unit, aka the engine of your video card, aka sometimes VPU, but VPUs are more programmable) decided they were going to evolve and be less hard-wired.

In a nutshell, CPUs (central processing units) are designed to be very general beasts in the grand scheme of things. They can never really tell what's going to be thrown at them. Which isn't bad, as no matter what you want to do you can break it down into pieces and feed it through the CPU.

This changes a bit with a dedicated device: it's intended to do very specific tasks, and is entirely oriented around those tasks to the exclusion others. Again, sort of like Paris Hilton. GPUs are a great example of this, in that they take the instructions that'll be generated by say, certain types of 3D apps and completely orient the design around them.

Now a 3D card is basically designed to push polygons through it like no one's business, and as GPUs included acceleration for different types of things game developers would build them into their games... but as I said, they did have a habit of looking a lot alike, because they were all doing similar things.

I.E., cards incorporated 'hardware transform and lighting', and the last generation of games all looked very similar when it came to lighting. These generations of cards were built around fixed-function pipelines: they could spit out a limited number of functions very rapidly over a bunch of pixels.

A new generation of cards hit the scene that offered programmable shader pipelines: developers could feed small programs along with their data to be executed on the pixels. Huge, huge deal. Huge.

These little shader units can be used for realistic bump maps, relfective surfaces, programmatically created textures... all sorts of things.

All wasn't exactly rosy here. As we know, large competitors often don't like to play nice, and there was a lot of back and forth between nVidia and ATI, and it didn't help that the companies seemd to be racing the people creating the specs to getting something out.

We've ended up in a situation where there are often multiple ways to go about programmers for these, and some are vendor-specific (I.E., will only work on an nVidia card, or only work on an ATI card)... in short, we came close to a cluster-fuck, but do to a bunch of work on a bunch of sides, and some glue, it's mostly all coming together... especially for other platforms.

Apple hasn't exactly been racing to get a lot of this stuff into their drivers, and OpenGL people I'll talk to will still rant about specific things. It's hard to be angry at a game maker when Apple is still adding major things to their OpenGL drivers as late as 10.3.6, and there'll be even more support added in Tiger... but again I'm digressing.

In short: with pixel shaders, and some other aspects of the new breeds of cards, the pieces are all there for HW-ACCL drawing to be implimented via an abstraction layer.

There were a whole bunch of other benefits to be brought to bear with this generation, like floating-point color. In the past, the GPU dealt with whole numbers for reasons of speed and other reasons, and exacting color wasn't all that important when it came to playing Unreal.

Long story short, floating-point means values can be specified like 5.68 instead of 5, which means everything is richer, more accurate, and there are less rendering errors.

So we can have an idea that Apple should be able to write an abstraction layer with glue to handle the rough spots, and then pipe out their drawing stuff to the graphics card. The benefits should be obvious if you've read this far.

The problem is, as we mentioned, there are a few different ways an application can draw to the screen: QuickDraw, CoreGraphics, and OpenGL. For the purposes of this, OpenGL doesn't really matter. And, unfortunately, neither does QuickDraw.

Everything I'm hearing is telling me that there will probably be HW-ACCL drawing in Mac OS 10.4, but that it will be CoreGraphics, now CoreImage, only. If you're drawing to the screen using those routines you'll get the benefits, but if you're not, you won't.

This might not seem to be that big of a deal at first, as QuickDraw is kinda ancient. But a lot of apps still pipe their stuff through it to get stuff on the screen in Mac OS X. Web browsers, Adobe Applications... basically any large Carbon app as well as a whole slew of freeware and shareware.

If you'll recall, when Apple was building Mac OS X, Carbon wasn't really on the map. Third-party developers, like Adobe and others, basically said, "No thank you", they just weren't interested in rebuilding their big apps from scratch.

Carbon was created, with the idea that they could just rewrite 10-20% of their apps (and as usual, it was never that simple) and have a 'native' OS X version running. In some cases this might not be that big of a deal, and an app can just spend some time changing some calls it's using for text.

In others, like programs like Photoshop or Dreamweaver this could be a very big deal indeed. Of course, I could be entirely wrong.

Personally I think these are a bit of a red herring, or at least how they were presented, and have just come along for the ride with CoreImage.

I.E., Apple went and created their engine for piping out the CoreImage stuff to the GPU, and things like ImageUnits are a way for developers to do the same thing without having to delve straight into OpenGL and one of the shader languages themselves.

There were rumors that they were created for one of the 'pro' apps, and someone just decided "Hey, why not just put this into the whole OS", but who knows, my napkins are fairly dry in this area.

Oh, yeah, much faster. However, I wouldn't expect everything to just become real-time. What I'm hearing says a 1.5 to 3x speedup for drawing and such, depending on your hardware.

If that doesn't sound like a miracle to you, then don't expect them... most Apple GPUs just aren't lightening fast, and there is overhead in the whole process... but it'll be significant.

Which isn't quite the whole story, because load should be reduced so dramatically that you'll get much more use out of what you already have because those cycles are freed up.

There's a reason why I harp on how crappy Apple's GPUs are... besides being embarrassing, many of their machines aren't even remotely 'future-proof' because of them.

If your machine doesn't have a GPU capable of pixel-shaders, sometimes called 'DirectX 9 compatible', then you are going to be stone-cold out of luck. This is unfortunately going to mean a lot of Macs, especially a lot of portable Macs, are going to get screwed.

We've gone through this before, where Apple ships the crappiest stuff they can sort of get away with for as long as possible... and then something like Quartz (gag) Extreme comes along and people who had just bought a Mac were left out in the cold.

Right now, out of their current lineup, only the iBook doesn't have a card that will handle CoreImage. However, this doesn't mean that the others will be crazy fast at it... Apple is basically shipping the cheapest video card still on the shelves that will do shaders, the 5200 Ultra.

The 5200, while it will do pixel shaders, is literally so weak that it might as well not be able to. You can't give these things away. The (gag) Ultra version that Apple is using now is a little more powerful than the standard, but still very, very weak and Apple isn't exactly pimping it out in terms of VRAM.

There will be a pronounced difference in performance between these el-Cheapo cards and something more powerful. HW-ACCL drawing will be a break in capabilities akin to Quartz (gag) Extreme, so it's hard for me to recommend anything at this point that has an intengrated graphics card that can't handle it, and handle it well.

I get asked about PCI-X and PCI-Express quite a bit, and it basically works like this:

• PCI-X is a stop-gap revision to the standard PCI that is faster, and also backwards compatible.
• AGP was a new type of slot between the CPU and the graphics card designed to deliver a bunch more bandwidth, among other things. It's gone through a bunch of revisions, increasing its bandwidth and such each time.
• PCI-Express is a new standard, multi-purpose slot which is not only cheaper to put into the computer, is as fast as AGP and can also take the place of PCI. Where it really shines though is that while AGP gives a lot of bandwidth going from the CPU to the graphics card, only a trickle is allowed back. PCI-Express allows it to flow both ways at high-speed.

That last part is going to get more and more important as we offload more and more to the GPU. You're seeing these start to ship faster and faster on the PC side, and I'd have to expect Apple is looking at it.

It's not a prerequisite for the stuff we've talked about, but that bandwidth being able to go both ways at high speed will become more important.

It's probably a lousy time to be buying an iBook.

 Posted by drunkenbatman
    December 17, 2004, at 12:46 AM