waffle

Waffle is a weblog.
The author of Waffle, some guy in Sweden, also occasionally writes stmts.net.

Lately on Waffle

Dyna-might

Brent Simmons:

When I talk about dynamic programming on iOS and Mac, I mean this:

  1. An app can learn about its structure at runtime.

  2. An app can do things based on what it knows about its structure.

  3. An app can make changes to its structure.

Brent is explaining this in an effort to see that the worthwhile qualities of Objective-C are not sacrificed to the gods of method call devirtualization. But his criteria are interesting. Almost everything about it is possible in nearly every language – some things are just major pains in the ass.

For example, in C#, my day time language (and increasingly night time), all of them are possible. 1 is easy due to reflection, 2 is easy due to reflection and the type system and 3 is easy if you share a predilection for hand-writing IL, where IL is the ungodly Intermediary Language used by the CLR virtual machine. (Okay, it’s a little bit easier with the modern Roslyn compiler which has an API for compiling C# code and loading code in memory, but the process for unloading it or replacing it is an emphatic “uh, let me get back to you on that”.)

Swift started out not very dynamic at all, but is at least now largely introspectable through “mirrors”, which I’m told is a sane(r) form of reflection. In Objective-C, the Objective-C runtime API’s information is willfully obtuse, because guess what, it’s the same API used to implement all the behind the scenes shenanigans to make message passing and classes and metaclasses and instance creation and method adding a thing in the first place, and it’s been optimized out the wazoo. Hello, Type Encoding and goodbye, scrutability.

In Swift, mirrors are comically loosely-typed-feeling, but still hard to decipher and still technically an API in flux. I have tried to write a dynamic JSON deserializer-and-class-populator and it was not self-describing. In C#, the reflection metadata is almost annoyingly precise and a task like that is just about as easy as that sort of thing will ever be likely to get.

The question isn’t really whether dynamism is needed. Many sharp brains have pointed out that the frameworks underlying Cocoa today would not be readily implementable in the pure Swift. Brent does a good job explaining why “just writing the code to solve the problem while using Swift idiomatically below” is a step backwards. Drew Crawford explains why there sometimes really really is no other solution.

What I’m wondering is whether there’s some amount of room for a third way. All of this, or at least parts, may be completely insane. I have not done any engineering of compilers or runtimes before.

What no one wants or likes is to have to wire up everything yourself. Code generation is a partial solution for this, but ginormous switch statements and lookup tables are also considered wildly inelegant. Brent quotes Guy English: “If you see a switch statement or dispatch table they blew it. Boilerplate that needs to be managed is a stagnant pool for nasty bugs.”

But lookup tables are exactly how message passing works. The problem isn’t that they exist, it’s any solution where, as Guy reminds us, you have to manage and see boilerplate.

So what if Swift let you code classes that looked like Swift but behaved dynamically? What if Swift let the virtualness of methods bleed out into runtime, and it could gradually be bolted down as the application ran, depending on the actual types, actual objects, actual methods, actual messages passed, call site by call site? What if that metadata was allowed to remain, could be used later and maybe even be extended?

I know that Swift creates “witness” intermediaries to make sure the right usage or definition of a method is captured. What if sending the foo:bar: message could be translated into a special failable method call on a compiler-construed catch-all protocol extension, for example, ensuring that sending it down a responder chain wouldn’t cause everything to come tumbling down?

I know that the Java HotSpot VM was capable of doing specialization optimizations that it could back out of whenever the optimizations stopped making sense – does that really take a JIT to do and not “just” (LLVM committers are given free permission to sneer knowingly now) some self-healing trampolines? What if suddenly sending that message to a proxy object would still end up working fine, without selling out the potentially much faster optimized version that is already being run 500 times per second? Objective-C was compiled because C was compiled. Swift is compiled by choice. What’s to say that it has to continue to follow the one-big-compilation model, or compile all the code, or compile only the code apparent when the build starts?

What if dynamic things could be dynamic, and Swift could be good at them too?

What if I’m full of shit and/or out of my depth here? The things I mention are things that to my intuition seem like puzzle pieces that could snap together. But I’m just following a few things I have read a bit about and mentioning them in the vicinity of each other. If it turns out that I’m wrong, well, I fully expect to be.

What I also expect is there to be a dialogue right now about these kinds of details by people who know what they’re talking about, about how Swift could actually become more dynamic without copying the flaws of its progenitors and without selling out the feel, mechanics and (most of the) performance of the language. I’m not seeing too much of it, and even if nothing came of it I’d love to read those sorts of thoughts too – even just a description about how and why my spitballing is logically inconsistent grasping at straws and the language design equivalent of alphabet soup, if that’s what it boiled down to.

The Forest

Don’t be encumbered by history. Go off and do something wonderful.
–Robert Noyce

The App Store is about to turn 8 years old and everything related to it is still immature. Apps are sold for pittances and mostly developed with the naive hope of a precocious youngster aiming to strike it rich with his combined marble distributor/lemonade stand. (Annoyingly often named following the same lexicon – oh, and backed by some investor’s war chest.) They are still reviewed under the purview of the parodically inconsistent App Review mechanism, renowned for its ability to take issue with minor details that have been there for versions. And they are still restricted to the subjects established by the recent meeting with, and concordant edict from, the True Love Waits club in Kaysville, Utah (nondestructively summarized as “Tits? GTFO!”… where GTFO presumably stands for “God, Thy Father, Obey”).

In short, we’re where we were. Concessions have been made along the way such that people can now build and deploy an application to their own device for free, even with officially sanctioned apparatuses. The mind boggles.

This absurd situation may work itself out in time, but I will spill no more pixels on it here.

The App Store is one half of the New Model, the one that recurring commentator Chucky will shortly refer to derisively as “turning things into consoles”. The other half is the technical restrictions used to uphold this New Model. Sandboxing and isolation. Restrictions out the wazoo. For all we know, ACLs on individual bits of memory.

Sandboxing on the Mac App Store has been an exercise in annoying frustration. By and large, existing apps haven’t been able to cope. Apps have gone into the App Store but had to back out because of all the bugs. I wrote a color picker that stored its preferences in the only place it could, and that broke causing it to ask whether you want to check for updates the first time the color panel’s invoked. This has been a mess from day one.

And yet… I don’t see it as the problem any more. Protected memory and process isolation has been with us for a long time. The problem with these sandbox mechanisms aren’t inherent problems in the idea of sandboxing. Name a security exploit that is not made less severe by inducing isolation around its boundaries. Post-App Store, post-Docker, what is to say that isolation and sandboxing isn’t just treating security with the requisite care that it has deserved for all this time, and that the right thing to do is to keep the goldfish in a bowl instead of a plastic bag?

What is the problem? It is that users haven’t changed. Needs haven’t changed. And too little has been done to make the new solution solve the old problems.

For all the noises made about respecting privacy and integrity, respecting someone’s right to their own data also comes down to the their right to take that data to another app. Yet with sandboxing today, you can’t build an import function from someone else’s app because it’s seen as invasive.

It would be supremely convenient to be able to host other extensions within your own app, or even within your own extension. If Apple wants to so desperately raise their Services revenue, allowing developers to develop more stuff that customers actually want might be worth kicking around.

For better or for worse, we are living in the era of the contained app. It has happened. It may still be happening. What will decide if I will be happy on tomorrow’s OS X/macOS won’t be whether my apps have writing permissions to a path or not, it will be whether I can still do the things I want to do. I don’t want the focus of this discussion to be on whether the locks should exist or not, I want it to be whether there is enough forward motion so that the things that are worth saving can be retained, and for vendors of every closed platform to be pressed on these issues.

(See also: The Tension of Swift.)

Mace Slipup

Mace Slipup is what you get if you rearrange the letters of Apple Music, and that’s just about what it feels like to use it too, sometimes. Burning, itching and meant to save your life but instead making it messier.

I will admit to being a late convert to the benefits of listen-to-anything-at-anytime-without-playing-the-DMCA-“no-Copyright-intended“-YouTube-lottery. Spotify is headquartered not too far from where I live, but I have tried to avoid streaming services in general and ones owned by labels in particular.

It is fun, convenient and still gives me a bit of the sense of living in the future (or, as some would have it, in 2011) where nearly everything is available in a celestial jukebox. And then it turns out that this is still something 1. somewhat hastily thrown together by people who 2. build iTunes, 3. have forgotten most user interface usability principles and 4. under the auspices of people who are artists but not kept in check. Apologies in advance to anyone in the Apple Music team who reads this and had to butt heads just to get it to the point it is right now.

People normally sold on more or less everything Apple – people, indeed, through which Apple leak their event preannouncements – have tried and simply given up. The thing eats people’s files. I keep backups and I avoid turning on iCloud Music Library, so I thought I was safe, but in the year I’ve been using it, it’s been pulling down duplicates of things I bought on iTunes ages ago and inserting them as new entries at the top of my library (sorted descended by Date Added). On iOS, it has “very helpfully” restored some ugly-as-fuck album art that I “Got Info” on ages ago and deleted the album art from. And just a few days ago, it seems to have re-checked a bunch of albums and songs that I for various reasons have unchecked through the ages. (Every song in iTunes has a checkbox, and if you uncheck it it won’t be played unless you manually start playing it. If iTunes is to Winamp/VLC what Excel is to TextEdit/Notepad, imagine silent data loss in the form of formatting, value or formula changes in spreadsheets. Heads would roll.)

Volumes have been written on iTunes’ steady decline. The only reason I persist in using it is because it’s still mostly the good jukebox app and acceptable podcast player it’s been for years. But if it’s news to anyone inside Apple at this point that the moss of complexity and mounting demands to become a fridge—freezer—toaster—shower—18–wheeler–balance—ice–dispenser has eaten the app and shit out the slow-loading, rotund 302 MB memory hog currently in my Applications folder, a pink slip should be contemplated, and the phrase Reality Distortion Field could find new relevance.

Someone needs to sit down and note the 40 or so good ideas about iTunes and build it over again from the ground up. Someone needs to smoothly extend it to where it can also play music from Apple Music and the iTunes Store (with “Buy to keep” buttons on all songs where licensing so allows, and it is ridiculous that it sometimes won’t), and someone needs to take to heart the firm principle of Just Keep My Fucking Music Library Intact or I’ll Fucking Cut You.

I think the odds of this happening are slim to none. It would make me happy and both preserve and restore the utility of a once well-made app, so it seems like the usual asking Apple for a unicorn. Watch as this year’s WWDC is about doing the exact opposite of these things instead.

« Newer posts · Older posts »