Saturday, June 15, 2013

iOS Port

This last week, I ported my engine and current game to iOS. It was much easier than I expected! I thought it would take me longer, because I didn't realize that iOS actually supports so many of the APIs I used in Linux and Android (such as pthreads and sockets).

Most of the frustrations I experienced were due to these things:

  • Macs do things differently than Windows and Linux, such as text file navigation, using command instead of control, and the Finder having no shortcut to my home folder.
  • I had never used Xcode before.
  • Objective-C is an abomination.
I have no particular love for Windows, but I used it for many years, so I do appreciate that Linux does things like Windows when it comes to hotkeys and the home/end keys, now that I've switched. I also acknowledge that Mac and Objective-C aren't "wrong", just different. Objective-C has some nice features, but I do think the syntax is unfortunate.

Now that I've done Android and iOS development, here's what I think....

Things iOS does better
  • C++ support. The compiler seamlessly compiles Objective-C++ and C++ together, making it trivially easy to integrate my code with the OS. Google has significantly improved C++ support in the last couple years, but I've still had to use JNI for some things, and that is terrible--much worse than Objective-C.
  • Apple has an IDE that integrates seamlessly with the device for native development. I haven't had success getting Eclipse to work with the Android NDK, but I've seen some people have success with it. But Xcode took no effort at all. It detected the device when I plugged it in, and after I provisioned the device, everything worked great.
  • Consistency across devices. I don't have to worry about some obscure device that I haven't tested on.
  • Much simpler application life-cycle.
  • Simpler display device management.
Things Android does better
  • The platform is more open, and so there are fewer hoops to jump through to use a device for development. No account to sign up for, no provisioning portal to monkey with, no cost to start running my code on device.
  • More options for development tools. By far the worst part of iOS development, IMO, is that I have to use a Mac. I have a really nice PC running Linux, but my Mac is a Mac-Mini. To have a Mac as nice as my PC would cost twice what I paid for the PC. Maybe someday I could justify the cost, but right now, I'm glad I can do most of my work on Linux.
Conclusion

Even though the iOS list is longer, the Android list "weighs more". Mostly it's because I prefer Linux over Mac, and my Linux development environment is much nicer. I'm not trying to persuade anyone either way. Overall, I'm quite pleased with how easy it was to port my C++ code over to iOS, because Android and iOS have much more in common than not.

-----------------------------------------------------------------------------
Some problems I encountered...
  • I would recommend staying away from the iOS simulator. If it's all you got, then I guess it's better than nothing. But I kept running into issues where my code worked fine in the simulator, but not on device. So I finally quit using the simulator. Iteration on device with Xcode is just as fast (at least for my project), so I'm not being slowed down at all.
  • In my engine, I override new and delete. My operator delete was being called before anything was loaded, even before static object constructors. When I attempted to free the pointer passed in to operator delete, the app would crash, saying the pointer was never allocated. I implemented a work-around that is working fine. In my operator new functions, I allocate some extra memory and put a sentinel value in front of the pointer I return from operator new. Then in operator delete, I check the 4 bytes in front of the pointer I received to see if it equals the sentinel value. If it does, I free the pointer, otherwise, I just ignore the call completely.
  • Mac has a case-insensitive (but preserving) file system, like Windows. I did not know that, and that lead to some unexpected issues where I had header files named things like "String.h" and "Time.h". I had a namespace to protect the names of my classes, but I ended up having to rename the files.

Friday, May 24, 2013

Class Pointers and uintptr_t

In the last few months that I've been doing C++ programming again, I've run into some bugs in my code that were quite tricky to figure out. There are lots of opportunities for bugs in C++, like buffer over-runs and accessing an object that's been deleted, but this bug below was trickier than usual....

uintptr_t is a handy type, because it's an unsigned integer that's the same size as a pointer on whatever platform you're compiling for.  Therefore, a variable of type uintptr_t can hold an integer or a pointer. How I'm using uintptr_t in my engine is a good subject for a different post, but for now, let's consider a tricky bug that comes up when using them to store memory addresses.

class Foo { ... };
class Bar : public Foo { ... };

Bar b;
uintptr_t address = &b;

Foo *f = reinterpret_cast<Foo*>(address);
// at this point, *f is full of garbage
// even though f points to b

Of course, this works fine:

Foo *f = &b;
// *f is correct

I was having problems with the contents of objects being garbage even though I confirmed in the debugger that the memory address was exactly the same as when the object's address was converted to a uintptr_t.

After some googling, I found that what I was seeing was expected behavior for class pointers.  You can only safely convert a class pointer to and from a void* or unitptr_t if the type of the reinterpret_cast to get the pointer back is exactly the same as the type that was initially converted to a uintptr_t.

So this works:

Bar b;
uintptr_t address = &b;
Foo *f = static_cast<Foo*>(reinterpret_cast<Bar*>(address));

That's a ridiculous example, of course, but it illustrates the point. If you use void*'s or uintptr_t's to store class pointers, it's possible you might run into this behavior.


Saturday, May 11, 2013

Fast Android Iteration, Part II

In part I of this series, I wrote about how I've achieved fast iteration on device with Android.  I just realized this morning though, that I forgot a couple things, so here's part II.

Eclipse is a popular tool for developing Android applications, and it even supports native and java on-device debugging. But it didn't work with this fast iteration method, because it does something really annoying: it completely controls the application data folder. Eclipse would rebuild the apk, wipe the data folder, and then reinstall it on each iteration. My "raw" folder would get removed, and so there was no benefit to copying only files that changed. I'd have to wait for the APK to get built and installed every time, which was the typical device iteration problem I always had.

So I'm not using Eclipse. The Android SDK supports ant, so I'm using the command line to build and manage my APK. It's really simple and easy to do. I only have to rebuild the APK when it's time to send a new build with my latest changes to a friend. Otherwise, I'm just running ndk-build and my script to copy changes to "raw". Here's a tutorial for using ant with Android.

The only thing I miss about Eclipse is on-device debugging. I spent a little bit of time trying to get Eclipse's debugging interface working with ndk-gdb, but I didn't get anywhere without using Eclipse for the whole project. After some searching, I found this tutorial on using CGDB to debug Android apps on device. It was really easy to setup.

Of course, CGDB isn't as nice as a graphical interface like Eclipse's. I do most of my debugging on PC (my engine support Linux too), and the Android log and stack traces are usually sufficient to fix problems on device. But when I want to debug something, CGDB gets the job done. The fast iteration time more than makes up for the slower debugging!

Saturday, May 4, 2013

Fast Android Iteration, Part I

The basic programming iteration cycle is:

  1. Make a change.
  2. See if it works.
That's pretty simple in theory, but hidden between those steps is an intermediate step often overlooked by developers: "build your changes, redeploy, and rerun the application".  Sometimes, that hidden step is longer than the other two, and it can seriously impair productivity.

For years, I did Windows development on Windows, and that hidden step was negligible.  But in the last couple of years, I've started doing a lot more mobile development, and that hidden step has brought so much pain into my programming life.

I've used Flash, Unity, and Marmalade for mobile development, and they all handle it the same way: develop and test your game on PC, then deploy it to device with the push of a button, and it's supposed to just magically work the same as on PC.  They're all pretty good about that, and it's true that most development work can be done and iterated on PC.  But none of those tools does it perfectly, because a mobile device is just too different in at least 4 ways:
  • Touch vs mouse input (touch is less accurate, and your hand obscures the screen)
  • Multitouch (just no way to do it on PC)
  • Performance (your game might run great on PC, but not on device)
  • The device's screen is way smaller and higher-resolution (so your game looks different on device)
So while most of your work can be done quickly on PC, any time you have to work on an issue related to one of those, you're stuck in device iteration hell:
  1. Make a change.
  2. Wait forever while your package is rebuilt and redeployed to device.
  3. See if it works.
The last Flash game I made for mobile took 5 minutes to package and deploy.  The last Unity game I made for mobile took over 2 minutes to package and deploy.

Using my new engine, iteration time is back down to just a few seconds.  That's all it takes to make a change on my PC and see it run on my Android tablet.  This is how I do it:
  • All engine and game code are compiled into dynamic library (.so) files.
  • The .so bundled with the .apk simply loads the engine and game libraries using dlopen, and then calls into their entry points.
  • I created a folder in my application directory on device called "raw".  I put all my .so files and assets into that folder.
  • Whenever I make a change, I run a script on my PC that that copies all changed files to the "raw" directory on device (this includes code and data changes), using adb push.
  • When the game runs, it looks first for the "raw" directory, and if it finds one, it loads all .so files and assets from there.  Otherwise it loads from the .apk like a normal Android app.
That's it!  This works great on Android, just as long as you do these things:
  • Pass in the full path to the .so file to dlopen.
  • Call dlclose on all your libraries.  If you don't, then Android caches them, and calling dlopen when you restart the app will cause you to get your old, cached version, instead of the version you just copied.
  • Load all your libraries with dlopen before allocating any dynamic memory.
  • Load your libraries in order of dependency.  If A depends on B, which depends on C, then load C, B, A, in that order.  You can't have circular dependencies at all.
  • Make sure the entry point for your application in your .apk has no references to anything in your engine or game.
None of those things have imposed any real problems for me.  I have my engine and game split into about 8 different .so files, including 3rd party libraries like libcrypto, libssl, and libpng.  By splitting them up, I don't have to build and copy all of the code every time I make a change.

I'll add here too that if you want a commercial engine that does this kind of thing already, check out the Loom Game Engine.  It has rapid device iteration that's even better than mine, because they have a scripting language, so you can see code changes without restarting the app.  Plus, it also works on iOS. I have yet to find out how the method I described above will work for iOS development.

Edit: there were a couple things I forgot here in part I, so check out part II.

Why write my own engine?

Most conversations I have about my business don't get technical.  Usually they never go much beyond the basic fact that I make games and try to sell them.  But other developers I've talked to wonder why I'm spending precious time making my own engine instead of using one of the many great existing engines/libraries (e.g. Unity, Unreal, Ogre3D, cocos2d, loom, SDL, Flash, Marmalade).

There are a few reasons (in no particular order):
  • Fun and Learning
  • Control
  • Customization
Fun and Learning

I love programming, and I enjoy all of game programming, and what I like about it is that it's hard.  But what makes the game side hard isn't so much the technical stuff but the game design issues: How do you communicate X to the player?  What's the best interface for Y?  How should we teach skill Z?  Those are essential problems to solve, but I prefer the technical problems of engine programming.

Control

I've spent a lot of time with Flash and Unity, and I've made games with both, and there are things I like and dislike about both.  I wrote Now Boarding in Flash, and we had some customers who couldn't run the game.  Our only option was to offer a refund and an apology.  Only a handful of customers ever had that problem, so it didn't really hurt us.  The trade-off between the ease and speed of development vs control over technical issues was worth it.

Then I started using Unity.  Unity is also a black box which offers the simple developer like me no control over technical issues.  However, my experience with Unity was that it was more buggy than Flash, and so those technical issues came up much more frequently.  I've spent a lot of time searching and posting on forums, looking for solutions to bugs.  Sometimes I found solutions, sometimes an update was released that fixed the problem, sometimes I found work-arounds, and sometimes I just had to live with the problems.  What else could I do without any control over the technology?

Having all the code, I can debug, fix, and enhance anything.  If a customer has a problem, fixing it is possible.

Lots of developers have founded their businesses on Unity, and many are having great success.  Unity keeps getting better and better, and they can add features to their engine so much faster than I can add to mine. Isn't it worth giving up control to let them handle the technical details so I can focus on my game?

Customization

Since you already know I'm writing my own engine, you already know what I'm going to say.  I would answer that question: "Not for me."  And it's not just because I enjoy engine programming.  This third point is about customization: my engine is exactly what I want, no more, no less.

Existing engines have lots of features I don't need, and they're missing certain key features I want.  This is what I wanted, and I didn't find all of them in any single engine I investigated:
  1. Fast device iteration (Unity, Flash, and Marmalade all require rebuilding the entire package for each change, but Loom got this one right)
  2. On-device debugging
  3. Rich 2D API
  4. Text-based, human-readable scene files
  5. Support for Windows, Mac, Linux, and Android (for consoles like Ouya)
  6. Source code
  7. Cheap
  8. Multithreading
  9. Support for Linux development
Out of those 9 possible points, Unity scores a 2 (5 & 8).  If I were focusing on PC and if I were making a 3D game, then Unity would be a much better choice (1, 2, and 3 wouldn't matter, and I could use the free version, making 7 non-applicable), but still not good enough, because 4 and 6 are all really important to me.  My engine, of course, scores a perfect 9 (or at least it will when I get around to porting to Windows and Mac). =)

Conclusion

So I'm writing my own engine because I love it, I have complete control, and I can have exactly what I want.  It's true that it'll take me longer to release my first game.  It's true that there's a ton of work and problems I could have avoided by using an existing engine.  But once I get through those problems, I'll have a solid foundation on which to build future games, with all control and customizability at my fingertips.

C++ vs C#: Productivity

In my previous post, I compared the performance of C++ and C# for game engines, based on a simple app I wrote that does some basic 3D rendering.  I found C++ to be more than twice as fast as C# for the number crunching and rendering that a game engine has to do.  I thought (and still think months later after making that decision) that switching from C# to C++ was worth it for the gains in runtime performance.

But what about productivity?  C# is more productive in that it takes less time to implement new functionality than it does in C++.  There are a few reasons for this increased productivity:

  1. C# has extensive built-in libraries.  C++'s STL isn't anywhere close.
  2. C#'s syntax is more concise: each class method exists in 1 place (.cs), instead of 2 (.h, .cc).
  3. C# works more consistently across platforms.
  4. C# compiles much more quickly.
  5. C# has a built-in memory manager with garbage collection.
The first 4 of those are things I've run into first hand.  The fifth is the most commonly offered reason, but it doesn't apply to games or other high-performance software.  I'll talk about each of these in order.

1. Built-in Libraries

While C++ has very little built-in, there are a tremendous number of free libraries and example code on the internet for pretty much everything you will ever need.  It's faster to just go to one place for documentation than to google everything; and it's faster to just add a reference to an existing assembly than to find, download, compile, and link against a new 3rd party library.  But while the cost of doing this in C++ is a lot higher than C#, that cost is amortized over time, because incorporating new code into my code is an infrequent task.

There are software industries where gluing together existing technologies in new ways is how things are done, and high level languages really excel at that.  But the time I spend building the core technology in my business is mostly just writing code, not gluing things together.

2. More Concise

The duplication of so much code (99% of method protoypes) is a weakness of C++.  But a good editor eases this pain tremendously.  With a push of a button, I can switch between .cc and .h files.  I can add a new method to a .h file, push a couple buttons, and then a stub is in my .cc file where I just have to write the implementation.  C++ code can be pretty ugly at times, but that's also mitigated by getting used to it.

(Editor features like snippets and macros also help reduce time spent typing in all languages.)

3. Cross Platform Consistency

Developing a cross platform application is more than just compiling and running code on each platform.  Using C# and mono makes developing for all PC platforms (Windows, Mac, Linux) quite easy, but it doesn't help as much with Android and iOS (the application lifetime and interface methods are completely different).

4. Compilation Speed

C# pretty much always compiles faster than C++.  It doesn't really matter how many C# files I change, the app pops up almost instantly.  C++ has to recompile every file I changed and all files that include them, and then relink the whole application.  For small changes, that happens almost instantly; and really, the most painful rebuilds are ones where I change headers that are included everywhere, and those are infrequent.

5. Garbage Collection

When I first switched from C++ to C#, I was excited by the prospect of no longer having to think about memory management.  But it didn't take long for me to learn that it wasn't true.  For games especially, I found that I have to still think about when things will be freed (garbage collection isn't free) and to make sure I null out references so that they will be collected properly.  Having a garbage collector just means than I have to juggle memory differently.  I still have to think every time I type "new" and whenever I store a reference to an object and whenever I subscribe to an event, etc.

Conclusion

C# is certainly more productive than C++, but I'm finding that the difference isn't that big.  I still spend much more time thinking about how to solve problems than about how to implement the solutions.

What makes a language higher level isn't magic: it's abstractions.  The features and libraries that make C# so productive can often be (or have been) duplicated in C++ (like events and delegates, containers, and many algorithms).  Some things can't be duplicated, and that's just part of the cost of using C++ instead.

C++ vs C#: Performance

Most programmers know that performance isn't the most important consideration when picking a language.  Ease of use and community support are also important.  Typically it's wise to pick the highest-level language that can meet the demands of the application, to use the right tool for the job.

My job is game development.  More and more games are being written completely in higher-level languages like C# and Java, which certainly have their advantages.

Last year, I was working on a C# game engine, because I like C# and because it's good enough for a lot of games.  But I was curious about just how much slower it is for a game engine than C++.  So I put together a simple C#-only demo (using OpenTK) that draws a bunch of 3D textured cubes and bounces them around inside the view volume.  Each cube is drawn and animated every frame.  I also implemented dynamic batching whereby individual cubes are combined into 1 draw call.  Then I pumped up the number of cubes until the frame rate got down to ~30 fps (~34 ms per frame).

I ended up with 25600 cubes being drawn and moved each frame.  My PC is starting to age, but I have a Core i7 processor with 8 cores (but the test app is single threaded), 12 GB of RAM, and an nVidia GeForce 8600 GTS video card.

Then I wrote a C++ version of the test app using OpenGL and freeglut.  It did exactly the same thing: the same geometry, the same texture, the same algorithm for bouncing, the same number of cubes.  It looked identical in every way, and it ran more than twice as fast (~13 ms per frame).

If you google "c# c++ performance comparison", you'll get a host of articles comparing performance between the two and other languages, like Java.  I found quite a few claiming that C# is only 10-20% slower than C++, which IMO, makes C# the clear choice.  But there are others that say C# is only half the speed of C++, which is what I found for this case.  To be clear, C# isn't so much slower for every case, and it's perfectly suited to a wide array of applications, including many games.

So after seeing the difference between pure C# and pure C++, I was curious how a hybrid would perform.  I wrote the core app in C++ which embedded the mono runtime and performed all the batching and rendering.  The C++ code loaded a C# assembly which instantiated the cubes and performed the "game logic" of bouncing the cubes around.  I wanted to emulate the way game engines like Unity use mono: C++ engine + game logic in C#.  Again, this hybrid test app produced exactly the same output as the C# and C++ versions, and it ran just a little more slowly than the pure C++ version (~15 ms per frame).

The code for all 3 apps is here.  I built and ran these apps on linux, and so I can't vouch for windows.  I used Eclipse for the C++ version and QT Creator for the hybrid.  I used Mono Develop for the C# code.

A couple other notes:
  • The C# code does not perform any memory allocations during the game loop, so garbage collection isn't an issue at all.
  • The hybrid version uses unsafe code in order for the C# and C++ sides to share the same memory.  Specifically, the C# code updates the positions and velocities of the cube in the same memory that the C++ code then reads when it renders the cubes.  I don't know how Unity does it, but I would imagine that marshaling that data across the C#/C++ boundary would only make it slower.  The code in the zip file shows how I did it.
Conclusion

So did I continue with my C# engine?  Nope.  It was a close call, but I thought that the types of games that I want to make might cause a pure C# game to bog down (like Minecraft does (which is an awesome game...I'm not ragging on it)).  I ran into this situation with Now Boarding.  It performed great until we started getting closer to the end after adding more and more visual content.  Since I'm starting a business now, laying a new foundation for future work, I don't want to get stuck later with a code base I can't make fast enough.  So I decided to lay C# aside and pick C++ back up again.

What about the hybrid version?  Isn't it the best of both worlds?  Yes and no.  Mono is a fast, extremely functional runtime, and it is pretty easy to embed in applications.  But it's a giant runtime with all kinds of features I don't need.  It's also not completely free.  Integrating a scripting language with an engine is a lot of work (I've done it with lua, and I started with mono), and in the end I decided to stick with pure C++.

There are a couple of questions that naturally follow:
  • Isn't the productivity of C# worth the runtime cost?
  • Why would you write your own engine anyway?
I'll write about those separately.