Saturday, May 4, 2013

Fast Android Iteration, Part I

The basic programming iteration cycle is:

  1. Make a change.
  2. See if it works.
That's pretty simple in theory, but hidden between those steps is an intermediate step often overlooked by developers: "build your changes, redeploy, and rerun the application".  Sometimes, that hidden step is longer than the other two, and it can seriously impair productivity.

For years, I did Windows development on Windows, and that hidden step was negligible.  But in the last couple of years, I've started doing a lot more mobile development, and that hidden step has brought so much pain into my programming life.

I've used Flash, Unity, and Marmalade for mobile development, and they all handle it the same way: develop and test your game on PC, then deploy it to device with the push of a button, and it's supposed to just magically work the same as on PC.  They're all pretty good about that, and it's true that most development work can be done and iterated on PC.  But none of those tools does it perfectly, because a mobile device is just too different in at least 4 ways:
  • Touch vs mouse input (touch is less accurate, and your hand obscures the screen)
  • Multitouch (just no way to do it on PC)
  • Performance (your game might run great on PC, but not on device)
  • The device's screen is way smaller and higher-resolution (so your game looks different on device)
So while most of your work can be done quickly on PC, any time you have to work on an issue related to one of those, you're stuck in device iteration hell:
  1. Make a change.
  2. Wait forever while your package is rebuilt and redeployed to device.
  3. See if it works.
The last Flash game I made for mobile took 5 minutes to package and deploy.  The last Unity game I made for mobile took over 2 minutes to package and deploy.

Using my new engine, iteration time is back down to just a few seconds.  That's all it takes to make a change on my PC and see it run on my Android tablet.  This is how I do it:
  • All engine and game code are compiled into dynamic library (.so) files.
  • The .so bundled with the .apk simply loads the engine and game libraries using dlopen, and then calls into their entry points.
  • I created a folder in my application directory on device called "raw".  I put all my .so files and assets into that folder.
  • Whenever I make a change, I run a script on my PC that that copies all changed files to the "raw" directory on device (this includes code and data changes), using adb push.
  • When the game runs, it looks first for the "raw" directory, and if it finds one, it loads all .so files and assets from there.  Otherwise it loads from the .apk like a normal Android app.
That's it!  This works great on Android, just as long as you do these things:
  • Pass in the full path to the .so file to dlopen.
  • Call dlclose on all your libraries.  If you don't, then Android caches them, and calling dlopen when you restart the app will cause you to get your old, cached version, instead of the version you just copied.
  • Load all your libraries with dlopen before allocating any dynamic memory.
  • Load your libraries in order of dependency.  If A depends on B, which depends on C, then load C, B, A, in that order.  You can't have circular dependencies at all.
  • Make sure the entry point for your application in your .apk has no references to anything in your engine or game.
None of those things have imposed any real problems for me.  I have my engine and game split into about 8 different .so files, including 3rd party libraries like libcrypto, libssl, and libpng.  By splitting them up, I don't have to build and copy all of the code every time I make a change.

I'll add here too that if you want a commercial engine that does this kind of thing already, check out the Loom Game Engine.  It has rapid device iteration that's even better than mine, because they have a scripting language, so you can see code changes without restarting the app.  Plus, it also works on iOS. I have yet to find out how the method I described above will work for iOS development.

Edit: there were a couple things I forgot here in part I, so check out part II.

No comments:

Post a Comment