Sir Isaac or: How I Learned to Stop Worrying and Love the Newton Virus

By Stephen Jayna, 1^st June 2009

841,464 YouTube views later and Troika's Newton Virus is released into the wild. And here's how it was done. Oh, by the way, it's not really a virus. Silly.

The Newton Virus In Action

Update 4^th Sept 2009: I'm working on a version for Snow Leopard almost as I type. Read about it here.

Update 11^th June 2009: Added section Auto-running The 'virus' From A USB Stick

Background

Named after the father of modern mechanics, the 'Newton Virus' introduces the concept of gravity causing your desktop to behave exactly as it would in the real physical environment. Desktop icons become susceptible to Newton’s invisible force and fall, roll and tumble in whatever direction gravity pulls them delivering a little bit of reality to your virtual environment. Read more and watch the videos here at Troika's website.

How It Was Done: In Brief

As you've surely guessed your desktop hasn't suddenly taken heed of Newton and decided that its current incumbants should behave accordingly. Nor is it really your icons that are taking a tumble ending in a crumpled heap, looking up in horror as a menu bar bears down on them from above. More's the pity.

— Spoiler Alert —

It turns out that all Mac laptops since October 2006 have included a Sudden Motion Sensor, perhaps better known to you and me as an accelerometer. You know, the sort of thing you'd find in an iPhone or Wii controller. Ostensibly it's to protect your hard disk drive from meeting an untimely demise when dropped. Butterfingers. But thanks to Unimotion that sensor can be harnessed for more, in our case at least, nefarious uses.

So that takes care of how we know that you're turning your laptop round and round trying to knock seven bells out of an otherwise innocent desktop icon. But how do we manipulate your desktop so? We begin by taking screenshots that's how. Lots of them! It's just like hitting Command-Shift-4 then spacebar and saving every object on the desktop indivdually, including your wallpaper. These images are then used in conjunction with Box2D, a piece of software that simulates real world physics. And voilà!

You'll be pleased to hear your desktop is never in any real danger: the 'virus' is just a façade. It's still there - your desktop that is - intact, oblivious to the chaos being meted out above it by the laws of nature. And so there you have it I suppose. I must confess that I hope some of you had your heart in your mouths the first time you saw it.

Credit and thanks to Troika for coming up with such a wonderful concept and for giving me the fun of implementing it.

Watch the original video here

How It Was Done: Not So Brief

A Disclaimer

Some of the techniques employed within are dubious and I wouldn't condone using them but for their artistic effect. A means to an end if you will. Bear that in mind before you recoil in horror at the methods that have been used. Of course if you've any better ideas on how to go about surmounting the challenges within they'd be most gratefully received.

The code was written in a mixture of Objective-C, C++ and a dollop of OpenGL.

Finally before you proceed I assume you've watched this video at the very least!

Auto-running The 'virus' From A USB Stick

Having the 'virus' run the moment the USB stick is inserted into the computer proved impossible. Once upon a time Apple allowed you to auto-run applications from a USB stick, much like you can on Windows. However — in their infinite wisdom — they decided it was potentially dangerous and it was last seen back in Mac System 9. Really some time ago. If we were on Windows, as far as I'm aware, we could have a U3 enabled flash drive emulate a CD-ROM, and configure its autorun.inf to launch the 'virus'.

Ejecting The USB Stick And Keeping The 'virus' Running

Key to the USB version is that the 'virus' continues to lie in wait even after the USB device has been ejected from the laptop. Sure, we could have copied it to your hard disk, but it seemed sensible not to start replicating ourself. It's such terrible PR you understand. Instead the 'virus' continues to run in memory only a reboot (or if I'm honest a page fault) away from oblivion.

First off you can't eject a device which is in use, be it a USB stick, a CD-ROM, or any other kind of removable media. It's naughty, can lead to 'unexpected behaviour', and you generally get a nasty looking warning if you try. Hardly what you want when you're trying to be sneaky.

However, if you purport to know what you're doing, there is a way around this. After discovering what device we're running from (and ensuring it is actually a USB device I hasten to add) we forcibly eject said device.

First find all the mounted removable media:

NSArray *volPaths = [[NSWorkspace sharedWorkspace] mountedRemovableMedia];

The 'virus' actually goes one step further and ensures that the removable media is in fact a USB device. Almost all USB devices have a serial number that is unique to them so I'm led to believe. It's possible to use IOKitLib to give us the value for the key 'USB Serial Number'. If you're interested you can see all sorts of properties attached to your hardware by running IORegistryExplorer, bundled with Xcode. It was quite the insight.

Then find out where our bundle is being run from:

NSString *ourPath = [[NSBundle mainBundle] bundlePath];

Then loop over [volPaths] until we find [ourPath]. This is the volume we need to unmount, using force:

  #include <DiskArbitration/DiskArbitration.h>

  DASessionRef session = NULL;
  DADiskRef disk = NULL;
                
  session = DASessionCreate(kCFAllocatorDefault);
                
  disk = DADiskCreateFromBSDName(kCFAllocatorDefault,
                                             session,
                                             [[volPath] UTF8String]);
                
  DADiskUnmount(disk,kDADiskUnmountOptionForce,NULL,NULL);
                
  CFRelease(disk);
  CFRelease(session);

Or a somewhat simpler alternative:

/usr/sbin/diskutil unmount force [volume]

The device will now have been quietly ejected regardless of whether it was in use or not. Yuck! Nausea aside we are now presented with a problem. Since the 'virus' lies in gestation, waiting for an unfortunate victim to return to their beloved Mac, much of its work takes place long after the USB stick has been removed. It still needs access to Unimotion, Box2D, a handful of sprites and of course its own code. All of which are long gone.

To get around this I borrowed from a neat trick of one of my colleagues. He'd discovered it while trying to speed up a freshly started MySQL on Linux. His idea was to force the database's indexes from a comparitively slow disk into memory immediatly rather than have MySQL do it piecemeal as and when the queries came in. This seemingly reduced the disks' thrashing and was very effective. And so to the magic in question:

  /bin/cat [file-destined-for-disk-cache] >/dev/null

Nifty eh? Upon execution the 'virus' iterates over the entire contents of its bundle on the USB device sending it to /dev/null prior to ejecting it. That includes the directories; despite the error you get when you do so. By doing this everything the 'virus' needs to run is placed in memory and the OS never needs to go back to the physical device for the files (well almost). Icky, yes. But ingenious, no?

I did wonder about using CSResourcesFileMapped in Info.plist for a while but not too hard and not for too long either. It's worth checking out if you don't know about it I guess.

Now, even as I write, I find myself wondering about the possibility of creating a RAM disk on the fly, and having the 'virus' run from there with only a bootstrapping program running directly from the USB device.

That seems eminently more sensible.

The Newton Virus appears on the MacLife.de cover disc

Window Capture

Now towards the final effect itself. Even if you don't initially realise that you can manipulate your desktop by tilting your laptop you'll be hard pushed to miss the mess at the bottom of your screen.

Of course it isn't actually your menu bar, icons and windows that are being flung around, rather a screenshot of each and every object - overlayed by way of OpenGL - in exactly the same place, initially at least.

Leopard provides an API that enables you to do this: you can capture the contents of any window. Icons, your menu bar and even the dock are just special types of window. So far so good I thought. That was until I realised Tiger doesn't have a public API for doing this.

Clearly you should not use private APIs, that goes without saying, they're private after all. You've no idea what might happen to them in the future or how they might be changed. But since I had no alternative...

Actually, heh, I have an inkling I could have used the Accessiblity API to capture the contents of the windows. Documentation for it, on the web at least, was difficult to find. Anyway in the end I had to resort to making undocumented calls via CGSInternal.h. Credit of which goes to Alacatia Labs.

The devil is in the detail of course. Working out what window belongs to which application, whether the window contains a transparent element and so on and so forth isn't easy.

Physics With Box2D

Having looked at a couple of 2D physics engine including Chipmunk I decided on Box2D for reasons that escape me as I write this. In part I suppose it was because I discovered quickly how easy it was to integrate C++ with Objective-C. Simply alter your source file extension from .m to .mm and you're free to mix Objective-C with C++ as you please. Well, nearly. Of course there are caveats but for the most part you don't have to think too hard about it.

Secondly and lest I forget, my predecessor Mauritius Seeger - who wrote the prototype of the Newton Virus in Processing - had used it too. And therefore it was patently up to the task.

Lastly the API is so simple it's easy to achieve a pleasing result with relatively little code. You define some shapes, attach them to a body, tell Box2D their starting positions, and watch as it tells you exactly where to paint those bodies around 30 times a second. Oh, I had to tell it in which direction gravity was acting too, but it was the least I could do given it'd relieved me of the task of doing any serious math. Besides I was saving myself for understanding projections and transformations in OpenGL, translating Box2D's output into something on screen.

And To Sum Up

So there you have it in all its glory. It's a little hard to believe that anyone will ever call on these so called techniques. But it was interesting to dig out my favourite snippets if only for the sake of posterity.

Having purchased my first Mac for the express purpose of building the 'virus' I can highly recommend it as a platform on which to develop. But you knew that already. If you're running a Linux desktop here's something that perhaps you don't know. Having run Fluxbox on Gentoo for as long as I can remember I was pained at the prospect of moving my everyday workings to OS X. In the event it was a breeze and I've never looked back.

Alright - so it's not open - and yes, that has tripped me up on more than one occassion. But fire up Terminal, install something like MacPorts and it's a home from home. What's more you've a beautiful desktop environment to develop end-user apps for.

Bonus Material: Having A ScreenSaver Loadable Bundle Link To A Dynamic Library

Should you ever write a screensaver, or any loadable bundle for that matter, that makes calls to an external library this is for you. It took me some time to work this out so it seems only fair I share it.

The Newton Virus ScreenSaver In Action

The problem is that by default the bundle will look for its libraries in the @executable_path. That is to say the path of the current executable that is running which in my case is the OSX screensaver engine. Which is where my libraries aren't. Instead you need it to look in the @loader_path, the path of the bundle requesting the load (your screensaver).

Below you've a script that fixes this. It's part of my build process in Xcode and runs after the screensaver has been successfully compiled and the binary linked with its libraries. In this instance I'm dynamically linking to libUniMotion.

#!/bin/bash

/usr/bin/install_name_tool -change "@executable_path/libUniMotion.dylib"
"@loader_path/../Resources/libUniMotion.dylib"
"$TARGET_BUILD_DIR/$TARGET_NAME/Contents/MacOS/$PRODUCT_NAME"

You can list the shared libraries linked to by your binary using the command below. Run it before and after the above and you'll see the fruits of your labour.

/usr/bin/otool -L [your-binary]

And that's your lot!

Snow Leopard

With the release of Snow Leopard Apple in their wisdom are forcing all screensavers to 64-bit. And here — in part — is why:

The screen saver engine in Snow Leopard has gone from 32 to 64-bit. As screensavers' are .dylib (dynamic libraries) they need to be ported to 64-bit in order to run under the new engine. Hence a 64-bit version of the Newton Virus Screen Saver is in the pipeline.

Desktop Screenshots

Nor can we take screenshots of desktop icons as we did in Tiger and Leopard. That facility has gone. Boo. This is presumably as Finder now paints desktop icons using an entirely different method, each icon is no longer a window. A method the Dock has employed since Leopard. But fret not Newton Virus fans: a work-around is on its way.