waffle

A good bit of what I learned about Grand Central Dispatch, I learned from Mike Ash’s excellent Friday Q&A series. The latest installment highlights some real world problems you may come across when trying to apply task parallelization in the wrong way. He demonstrates with a fairly representative example that won’t be so easily mechanically transformed into going faster because the obvious way of doing so moves the bottleneck to a place where it, along with the good intentions of every other part of the system, actually floods it in a horrible way.

The problem with parallelization is fundamentally that we have to think beyond what we did before because of the discrepancy between new facts and old assumptions, and that extends not only to keeping locks and concurrency in check but to figure out what sort of load you’re putting the rest of the system under when you’re being more efficient with execution. Even a good solution with a wide scope won’t insulate you from the pre-existing architecture, pre-existing code and your pre-existing assumptions and approach. There are formulas that you can use (I’m seeing where I can change a @synchronized to pushing the work into a dedicated dispatch queue, asynchronously or not, for example, because that code often runs in its own thread) but you shouldn’t ever let them excuse you from finding out the full picture so that you may see everything in context.

We may finally be headed towards the right tools to tame the right hardware. Let’s put ourselves in the right mindset, too.

(Follows: House season opener spoilz!)

Verdict: Good. I have no idea what this brings about for (dreaded) continuity, but in retrospect, like all revolutions, it seems inevitable. House will always be himself, but the series has had to work increasingly harder, including more frequent speculative trips to “get out of pain free”-ville, to avoid veering off into outrageousness for pure effect.

There was a point about two thirds in where the cards were stacked in such a way that House the doctor would have swept in, saved the day and won over his opinionated captor using techniques that embody House and that are established safe set pieces. That that didn’t happen stands to the writers’ enormous credit. Every episode this far has been about a case; this one was about House himself, and I’m happy that they allowed themselves to take it to its conclusion just like any other.

It wasn’t all perfect; the whole mute girl speaking thread was a bit predictable, but I’m willing to endure it if only for the way it was woven into every other thread at one point or another. And yes, the whole thing started uphill by feeling like an unconnected TV movie where almost everyone else except for the main character are gone and probably won’t return for later installments. It saved itself by not pretending that those people had always been there; this was a one episode special that’s actually integrated with the rest of the series. In contrast to most things you might compare it to, it’s pivotal, not negligible. And it proves that House is portable. I don’t want to seem him run back and forth between genres, of course. But the actor, direction and script is good enough that he never stops feeling like House.

I love 37signals.

Before we begin: For the purposes of this post, freedom refers to this exact definition: the freedom to mess around with the apps you have installed on your own system the same way nothing prevents you from putting a saw, sledgehammer or glitter, glue and macaroni to a chair in your possession.

Mac OS X ships with support to sign executables with cryptological certificates. Generally, thanks to cryptology, a private key in addition with a hash of its content can produce a “footprint” of the executable that: a) can be verified by the public key and the content hash alone, and b) can be assumed with very narrow error margins to originate from the certificate owner. The uncertainty of b) is usually double checked with a Certificate Authority which can be inquired about revoked certificates, as well as with a limited certificate lifetime. Mac OS X’s code signing does not work exactly like this for reasons that I’ll get back to.

Mac OS X relies heavily on the concept of bundles; specially tagged folders that contain executables, resources and metadata. Applications are bundles that end in .app, and like many bundles they contain a special folder structure and a “property list”-format metadata file (Info.plist) containing primary information, such as the document types the application is capable of handling as well as metadata related to the point of entry into the executable (the principal class).

When bundles mix with code signing, a new dimension is added. An extra file provides for the cryptological hashes of many of the individual files in the bundle as well as a set of rules detailing which files are included, omitted or optionally included (included if they’re there, not included if they’re not there). This file provides the explanation to how Mac OS X’s signing works: a code signature consists of a unique identifier (either generated and provided by the signer or derived from Info.plist), the seal (which is the extra file just mentioned) and a digital signature that signs the seal.

As of 10.6 (Snow Leopard), Mac OS X does not particularly enforce signed executables. Using the codesign command line tool (“code sign”, not “co-design”), you can verify the signature of an executable, but the verification is weak. Checking a signature made by signing with an expired certificate passes, and for codesign to verify that a certificate was not revoked, the certificate must be set to ask that revocation is enforced and a user specific setting to enforce revocation must be enabled. (If the certificate asks for revocation checking but the user setting is “Best Attempt”, and the revocation checking fails, the signature will be regarded as valid.)

Despite all this, certain subsystems in Mac OS X still use code signing in ways that could be regarded as safe. The built-in firewall regards updates to policy by applications that were previously signed but are now signed under a different certificate or unsigned as highly suspicious, and the Keychain allows access to application-specific items by default to applications proven to have the same cryptological lineage.

Relatedly, an important point to make is that these subsystems, as well as the code signature itself, can be designed to care about different facets of integrity. For Keychain, which at any point in time regards an application’s identity in one specific way, just checking for a change of certificate can help determine whether the upgraded application should have access or not. The firewall will willingly wrap any unsigned application in a makeshift code signature just to be able to verify it using the same method as it does signed code.

So am I getting to the freedom part yet? Yes, yes I am.

The core issue is that many of the popular self-applied hacks today involve editing, removing or replacing the files on which the seal or even other parts of the code signature is based. The reason to tackle these files are obvious; they are where the orientation of the contrarily oriented split views are declared, where the decision is made to hide the dock icon and where that tyrannical know-it-all developer hard codes her tasteful themes which you could vastly improve in a heartbeat.

I would surely bet on Apple including more code signing capabilities in future versions of Mac OS X, and being less lax on signature validation. This is not itself a catastrophe; actually, it’s rather nice. Code signing just does one thing, which is to provide a way of checking that the executable and the files the application author determines are important aren’t modified or corrupted. Apple is doing a good deed by making sure this technology is there because it’s something that could directly benefit you, and they’re the only ones who can add it. The danger is in the ways in which this technology could be misused.

Info.plist by default plays a special role in code signing, and for good reason since it to some extent controls the startup of the application. Resources are opt-in, but the default set of rules used in the seal will include at least some of them. (I couldn’t find a definite word on this, which probably means that the exact set is implementation-specific and prone to change. I don’t like the lack of documentation on this key detail.) Even so, applications could definitely be signed in such a way that these resources are excluded and hackability is maintained. I believe it would be irresponsible of Apple to actually make this the default, though. Resources are often scripts of varying capacities and could dramatically change any guarantees you care to make about the app.

Is your freedom to do to your apps what you believe is right constrained by code signing? That answer will vary depending on the implementation of code signing, the certificate and what data is inspected. But in a worst case scenario where everything in the bundle is considered holy, removal of the certificate from a signed app is prevented and the integrity of the code signature is checked on launch, the answer is yes. We’re still far from this point in any of these three dimensions (in order: opt-in, not that I can see, nope), so there’s no immediate danger. And it’s definitely possible to adopt a code signing policy where select files in the bundle can be modified without repercussions (beyond the normal dangers of such an operation, of course).

Read up on the details directly from Apple: Code Signing Guide and Technical Note TN2206: Mac OS X Code Signing In Depth. I am definitely not an expert at cryptology, certificates or code signing, but effort has been taken to make sure the information in this post is correct.