waffle

Waffle is a weblog.
The author of Waffle, some guy in Sweden, also occasionally writes stmts.net.

Lately on Waffle

WWDC 2016 Predictions

  • macOS.
  • watchOS 3 – simpler overall, custom watch faces and something else on the side button. (No new watch until later this year.)
  • tvOS 2 with pluggable Universal Search and integrated support/allowance for live TV.
  • Siri:
    • A passive Siri API, in the same way as the Spotlight API is passive – you get to put things in the right places or define static (code-signed property lists) phrases but not respond directly to Siri commands.
    • A way to embed Siri functionality, like speech recognition/synthesis or Spotlight.
  • X11 for iPhone. Just kidding.
  • iOS 10 with a new home screen.
  • App Store:
    • Mea culpa about unfocused, uneven history (followed no doubt by the unanimous awarding of a pony to me, with condolences); focus on recent developments.
    • Letting some developers have privileged or trusted access.
    • Some sort of widening in ad hoc/enterprise distribution for iOS, similar to Developer ID/Gatekeeper.
    • In-app purchases on Mac.
  • iMessage for Android.
  • Thunderbolt 3:
    • A new Retina MacBook Pro with Thunderbolt 3.
    • A bumped Mac Pro with Thunderbolt 3.
    • A Thunderbolt 3 5K Display with an embedded GPU.
  • International:
    • Touting the 60 countries that developers come from.
    • …followed by references to sports that no one outside of the US plays and to clubs that no one outside of the US has heard of.
    • At least one service or capability or upgrade that is only usable in the US.
    • Apple Pay to Germany and France.
  • Grab bag:
    • A major upgrade to Photos.
    • Tag support in iCloud Drive.
    • Eddy Cue’s outfit louder than auditorium PA system.

Dyna-might

Brent Simmons:

When I talk about dynamic programming on iOS and Mac, I mean this:

  1. An app can learn about its structure at runtime.

  2. An app can do things based on what it knows about its structure.

  3. An app can make changes to its structure.

Brent is explaining this in an effort to see that the worthwhile qualities of Objective-C are not sacrificed to the gods of method call devirtualization. But his criteria are interesting. Almost everything about it is possible in nearly every language – some things are just major pains in the ass.

For example, in C#, my day time language (and increasingly night time), all of them are possible. 1 is easy due to reflection, 2 is easy due to reflection and the type system and 3 is easy if you share a predilection for hand-writing IL, where IL is the ungodly Intermediary Language used by the CLR virtual machine. (Okay, it’s a little bit easier with the modern Roslyn compiler which has an API for compiling C# code and loading code in memory, but the process for unloading it or replacing it is an emphatic “uh, let me get back to you on that”.)

Swift started out not very dynamic at all, but is at least now largely introspectable through “mirrors”, which I’m told is a sane(r) form of reflection. In Objective-C, the Objective-C runtime API’s information is willfully obtuse, because guess what, it’s the same API used to implement all the behind the scenes shenanigans to make message passing and classes and metaclasses and instance creation and method adding a thing in the first place, and it’s been optimized out the wazoo. Hello, Type Encoding and goodbye, scrutability.

In Swift, mirrors are comically loosely-typed-feeling, but still hard to decipher and still technically an API in flux. I have tried to write a dynamic JSON deserializer-and-class-populator and it was not self-describing. In C#, the reflection metadata is almost annoyingly precise and a task like that is just about as easy as that sort of thing will ever be likely to get.

The question isn’t really whether dynamism is needed. Many sharp brains have pointed out that the frameworks underlying Cocoa today would not be readily implementable in the pure Swift. Brent does a good job explaining why “just writing the code to solve the problem while using Swift idiomatically below” is a step backwards. Drew Crawford explains why there sometimes really really is no other solution.

What I’m wondering is whether there’s some amount of room for a third way. All of this, or at least parts, may be completely insane. I have not done any engineering of compilers or runtimes before.

What no one wants or likes is to have to wire up everything yourself. Code generation is a partial solution for this, but ginormous switch statements and lookup tables are also considered wildly inelegant. Brent quotes Guy English: “If you see a switch statement or dispatch table they blew it. Boilerplate that needs to be managed is a stagnant pool for nasty bugs.”

But lookup tables are exactly how message passing works. The problem isn’t that they exist, it’s any solution where, as Guy reminds us, you have to manage and see boilerplate.

So what if Swift let you code classes that looked like Swift but behaved dynamically? What if Swift let the virtualness of methods bleed out into runtime, and it could gradually be bolted down as the application ran, depending on the actual types, actual objects, actual methods, actual messages passed, call site by call site? What if that metadata was allowed to remain, could be used later and maybe even be extended?

I know that Swift creates “witness” intermediaries to make sure the right usage or definition of a method is captured. What if sending the foo:bar: message could be translated into a special failable method call on a compiler-construed catch-all protocol extension, for example, ensuring that sending it down a responder chain wouldn’t cause everything to come tumbling down?

I know that the Java HotSpot VM was capable of doing specialization optimizations that it could back out of whenever the optimizations stopped making sense – does that really take a JIT to do and not “just” (LLVM committers are given free permission to sneer knowingly now) some self-healing trampolines? What if suddenly sending that message to a proxy object would still end up working fine, without selling out the potentially much faster optimized version that is already being run 500 times per second? Objective-C was compiled because C was compiled. Swift is compiled by choice. What’s to say that it has to continue to follow the one-big-compilation model, or compile all the code, or compile only the code apparent when the build starts?

What if dynamic things could be dynamic, and Swift could be good at them too?

What if I’m full of shit and/or out of my depth here? The things I mention are things that to my intuition seem like puzzle pieces that could snap together. But I’m just following a few things I have read a bit about and mentioning them in the vicinity of each other. If it turns out that I’m wrong, well, I fully expect to be.

What I also expect is there to be a dialogue right now about these kinds of details by people who know what they’re talking about, about how Swift could actually become more dynamic without copying the flaws of its progenitors and without selling out the feel, mechanics and (most of the) performance of the language. I’m not seeing too much of it, and even if nothing came of it I’d love to read those sorts of thoughts too – even just a description about how and why my spitballing is logically inconsistent grasping at straws and the language design equivalent of alphabet soup, if that’s what it boiled down to.

The Forest

Don’t be encumbered by history. Go off and do something wonderful.
–Robert Noyce

The App Store is about to turn 8 years old and everything related to it is still immature. Apps are sold for pittances and mostly developed with the naive hope of a precocious youngster aiming to strike it rich with his combined marble distributor/lemonade stand. (Annoyingly often named following the same lexicon – oh, and backed by some investor’s war chest.) They are still reviewed under the purview of the parodically inconsistent App Review mechanism, renowned for its ability to take issue with minor details that have been there for versions. And they are still restricted to the subjects established by the recent meeting with, and concordant edict from, the True Love Waits club in Kaysville, Utah (nondestructively summarized as “Tits? GTFO!”… where GTFO presumably stands for “God, Thy Father, Obey”).

In short, we’re where we were. Concessions have been made along the way such that people can now build and deploy an application to their own device for free, even with officially sanctioned apparatuses. The mind boggles.

This absurd situation may work itself out in time, but I will spill no more pixels on it here.

The App Store is one half of the New Model, the one that recurring commentator Chucky will shortly refer to derisively as “turning things into consoles”. The other half is the technical restrictions used to uphold this New Model. Sandboxing and isolation. Restrictions out the wazoo. For all we know, ACLs on individual bits of memory.

Sandboxing on the Mac App Store has been an exercise in annoying frustration. By and large, existing apps haven’t been able to cope. Apps have gone into the App Store but had to back out because of all the bugs. I wrote a color picker that stored its preferences in the only place it could, and that broke causing it to ask whether you want to check for updates the first time the color panel’s invoked. This has been a mess from day one.

And yet… I don’t see it as the problem any more. Protected memory and process isolation has been with us for a long time. The problem with these sandbox mechanisms aren’t inherent problems in the idea of sandboxing. Name a security exploit that is not made less severe by inducing isolation around its boundaries. Post-App Store, post-Docker, what is to say that isolation and sandboxing isn’t just treating security with the requisite care that it has deserved for all this time, and that the right thing to do is to keep the goldfish in a bowl instead of a plastic bag?

What is the problem? It is that users haven’t changed. Needs haven’t changed. And too little has been done to make the new solution solve the old problems.

For all the noises made about respecting privacy and integrity, respecting someone’s right to their own data also comes down to the their right to take that data to another app. Yet with sandboxing today, you can’t build an import function from someone else’s app because it’s seen as invasive.

It would be supremely convenient to be able to host other extensions within your own app, or even within your own extension. If Apple wants to so desperately raise their Services revenue, allowing developers to develop more stuff that customers actually want might be worth kicking around.

For better or for worse, we are living in the era of the contained app. It has happened. It may still be happening. What will decide if I will be happy on tomorrow’s OS X/macOS won’t be whether my apps have writing permissions to a path or not, it will be whether I can still do the things I want to do. I don’t want the focus of this discussion to be on whether the locks should exist or not, I want it to be whether there is enough forward motion so that the things that are worth saving can be retained, and for vendors of every closed platform to be pressed on these issues.

(See also: The Tension of Swift.)

Older posts »