At other times, Apple is tight-lipped about stuff not because of resource constraints, but rather intentionally, though it's not always clear why. Do you want to know how fast the processor in your iPhone is, or how much memory is available to your application? Both of these are obviously relevant to developing for the iPhone, but don't look to Apple for that information. The information is available (processor | memory), just not from Apple.
Here's another area where Apple is less than fully forthcoming: When it comes to the iPhone's use of PNG images, Apple tells you that Xcode "optimizes" PNGs as part of the build process and, as a result, you should always use PNG images in your iPhone projects. PNG images will usually be bigger than JPEGs, causing your application to be a little larger, but you'll be rewarded by some mysterious kind of improvement.
Since this information is not documented, what follows is something of an educated guess based on research and observations. I will gladly be corrected on any inaccuracies, so don't hesitate to ping me if you see something wrong.
So, just what is this magical, mystery "optimization"?
During the build, Xcode basically screws up your PNG image, turning it into something that's technically no longer a PNG image. Xcode doesn't change your original file, but when it copies it to the Resources folder of your application bundle, what it does is it byte-swaps the red and blue octets and also "premultiplies the alpha" (if that doesn't make sense, don't worry, we'll talk about it in a moment). The PNG specification does not support either of these changes, which is why the image is no longer technically a PNG and can't be loaded by most image editing software. Not that it really matters, since the new version only exists inside your application bundle, but it does mean that any PNG file that you include in your app bundle should not be, for example, FTPed or e-mailed from your application directly, because the file will be corrupt as far as most software is concerned.
Byte Swapping
Byte swapping is exactly what it sounds like. Uncompressed computer graphic images are most-often stored as sequences of three or four bytes or "octets" representing each pixel in the image. Each byte in the pixel represents one of the three additive primaries (red, blue, and green), plus there's often another byte called "alpha", which we'll look at in a moment. There are other ways to store images, but this is the most common technique. The most common byte-ordering used in this technique is RGB (or RGBA) which stands for Red-Green-Blue(-Alpha). This means that a single pixel is represented in memory by four bytes, the first representing the intensity of red, the second representing the intensity of green, and the third representing the intensity of blue (we'll ignore alpha for now). The PNG specification uses this byte-ordering, as do many other image formats.
However, the iPhone's video memory uses a non-standard byte-ordering of BGR or Blue-Green-Red (I don't believe there's an alpha component in the iPhone's vRAM). In order to copy from an image in memory using RGBA to the video memory using GRB is more computationally expensive than just copying from, for example, BGR to BGR, which can be done with a single, fast memory move (or "blit") operation. By doing this byte-swap in the Copy Files Build Phase, your application is (in some situations) able to ignore the alpha component when loading the file into memory so it can use the faster memory move operation to get the image into video memory (and hence onto the screen).
Premultiplied Alpha
Okay, that makes sense, but what about this "premultiplying the alpha" thing? That one sounds kind of mysterious, doesn't it? As mentioned in the previous section, the iPhone's vRAM has no alpha component, so if we're going to ignore that component, we still need to take it into account somehow. Remember that PNG Images that have been optimized for the iPhone are stored as four values, representing the Blue, Green, Red, and Alpha components (in that order). Although it's called a "component", Alpha isn't really a color component at all. It's more like a modifier that acts on the other three values, and it represents how much of whatever is beneath the color will show through. So, a color that looked like this:
B:1.0
G:0.0
R:0.0
A:0.5
Would represent the color blue, but at 50% opacity. In order for the computer to use the alpha component, it has to multiply the alpha times the other three components (and possibly by other values) before putting them into video memory. This multiplication process is more computationally expensive than just copying the value into video memory. So, Xcode also premultiplies the alpha by the three components and stores the multiplied value. As a result, the color above would look like this after the Copy Files Build Phase:
B:0.5
G:0.0
R:0.0
A:0.5
Notice that the Blue component in this new pixel is equal to the Blue component of the previous version multiplied by the alpha. The result of this premultiplication, is that the alpha component can be ignored when loading the image into memory and the image can then be blitted directly to video memory without having to do the expensive floating-point multiplications to calculate the alpha. Well, at least, sometimes. This pixel is what the pixel would look like when drawn over white.
So, what happens if there's a color beneath that needs to show through? This is where things can get a little confusing. If your application is drawing the image on top of something else, or if you're using the alpha channel in the image as a mask, then the iPhone can't use this optimization. It has to do the alpha calculations (which is why the alpha channel is left intact in the pre-multiplication process), the the byte-swapping still offers some improvement. For the most part, you don't have to worry about this - it all happens under the hood. The thing you should take away from it though, is that you can help your iPhone know when it is safe to use the optimization. The way to do that is to make sure you check the "opaque" checkbox in Interface Builder for your image views, or other views that contain images, or set the opaque property of your UIImageView to YES in code. This tells the iPhone that it's safe to do the faster blit operation and ignore the alpha channel because nothing below shows through an opaque object and there's no need to do any expensive floating point alpha calculations for that object. Of course, if you are using the alpha channel in your image, or if you are doing complex image compositions in your app at run-time, you shouldn't make the view opaque. But, if you're just displaying an image in an UIImageView, checking that opaque checkbox is a really, really good idea.
An awful lot of the time, PNG images have 1.0 for the Alpha value in every pixel. As a result, more often than not, program runs faster and use less memory because of the PNG "optimizations" done by Xcode.
What happens when you use different file types
When you use any other file type (or if you load a non-optimized PNG files), your iPhone has to do the byte-swapping and alpha premultiplication at load-time (and possibly re-do the alpha multiplication at display time). Your application basically has to do the same processing that Xcode does, but it's doing it at run-time instead of at build-time. This is going to cost you both in terms of processor cycles and memory overhead. One of the reasons why Mobile Safari is the biggest memory hog of the built-in iPhone applications is because the images it has to load in order to display web-pages are all non-optimized images, mostly JPEGs. Since JPEG is a compressed format, it has the added extra step of having to decompress the image into memory before it can do the premultiplication and byte-swapping.
No comments:
Post a Comment