Frame-by-Frame Sprites with Core Animation

Posted on January 12, 2011

So you want to make a 2D game full of real 80s technology: sprites, tiled backgrounds, parallax scrolling, and all that goodness. The last console you played a 2D RPG game in, back when you had hair, is long lost in the rat’s nest behind your TV stand. Your iPhone’s processor can run circles around that old console but you wonder… how do I do it?

Now, if you were that kind of person, you would go to the platform’s documentation site for answers. But instead you check with Google, and after a few visits to random websites and auto–cloning forums you reach the following consensus with the rest of the internets:

You need to use OpenGL for the ultimate in both power and beginner’s frustration.
Or better, use one of those open source frameworks that will make your transition from ActionScript a breeze. Their forums hopefully will answer any questions that the non–existent documentation cannot. Cocos2D seems popular.
The losers that read documentation keep asking about something called Core Animation, but wiser and more anonymous users know better than to trust Apple “technologies”. Core Animation is slow, buggy, and inappropriate for anything harder than a match–3 game.

I say, not so fast. You see, I have a problem with thesis that are never proved. I especially don’t get what in that screenshot you sent me of three bats, a rat, and a stick figure is so sophisticated that requires low–level access to the hardware. So let me present you a different thesis, and then let’s try to prove it:

Thesis A: Core Animation is just fine for most 2D games.

Before we begin: no, I have no idea if thesis A is true or not. I am making it up as I go, so consider it an exploration of the capabilities and limitations of Core Animation and related technologies. If we don’t end up with a whole game, at the very least we will gain some insight into how this fundamental part of any iPhone app actually works.

Elephants

Although there has not been much visible progress in any of the 7 evil projects, the elephant racing game seemed like a good one to tackle so I could explain how to make frame-by-frame Core Animation sprites. The only problem was I didn’t have any art for it, or even an idea of what style I wanted, so I spent some nights this past week trying to define how the game should look.

This is kind of a weird habit of mine. If you have read any articles about quickly prototyping your games, you already know starting with the art is the wrong thing to do. But I kind of like having a better picture of what the game is going to look like in general, think about its theme, colors, if is going to use realistic graphics, cartoony illustrations, or other styles, what’s the music going to be like, etc. Anything that makes it more clear in my mind how the game experience will feel.

For this game I wanted the tone to be light-hearted and frantic. I was imagining the elephants stomping through villages in the jungle, tearing down anything in their way, obsessed with the bananas that hysterical mahouts swing in front of them. Mobs in the background will either cheer from their roofs, or yell in hot pursuit cause their house has just developed new elephant-shaped doors. The player has to be quick-witted, and use any dirty trick to stop his opponents: steal their bananas, throw them into a devious path for their elephants to follow, scare them off with noises, bump into them, or share that tiger that has been following you around…

After a few sketches I came up with this:

As you can see is missing lots of the elements I was just talking about, but at the same time it has enough detail to get into the mood of the game, and create an interesting prototype.

Layers, not Views

So the first thing to understand about Core Animation is that is ever-present. If you are rendering into an iOS device screen by any means, Core Animation is being used under the hood for compositing images. Internally, Core Animation will use OpenGL, or whatever other hardware abstraction is in place in the future, to render your final image with hardware acceleration.

To do all this rendering and compositing CA uses a light weight object known as a CALayer (or just layer). After looking at its properties you would think a layer is kind of like a view. It has the same frame and bounds to define its size and position, layers can contain other layers in a hierarchy same as with views… the interesting part are the differences though.

A UIView is part of the Responder chain so it can handle touch and motion events, decide when it becomes the first responder, etc. Every view is backed up by a layer that handles all its rendering and animation (at least in UIKit, AppKit in the Mac makes the NSView/CALayer dependence optional). The reason the 2 classes have similar properties is cause the view works as a proxy for its internal layer. Note however that the view hierarchy and the layer hierarchy are different; you can have one view with a whole layer tree attached, that is also part of its own view tree. The thing to remember is that for each view you will have at least one layer.

For all our sprites we will use layers embedded in a single view. Why not use UIImageViews instead? So that we don’t incur in the event-processing overhead; we don’t need our sprites to handle input, so this way our Responder chain will be simpler, and less time will be used navigating its hierarchy whenever an event fires.

Layer Content

A nice thing about layers is that you can provide image content pretty easily. If you have a reference to a CGImage, all you have to do is set the layer’s contents property to it. Like this:

NSString *path = [[NSBundle mainBundle] pathForResource:@"yourImage.png" ofType:nil];
CGImageRef img = [UIImage imageWithContentsOfFile:path].CGImage;

CALayer *layer = [CALayer layer];
layer.contents = (id)img;
layer.bounds = CGRectMake( 0, 0, CGImageGetWidth(img), CGImageGetHeight(img) );

If you have multiple elements in the game sharing the same graphic, nothing easier than to load the image once, then use the CGImageRef as the content for a bunch of layers (you would think that’s obvious, but you probably have not looked through the internets for UIImage examples).

You can see the problem with this though. If we have to load separate images from disk for each type of sprite we do, loading the game is going to slow to a crawl, not to talk about the havoc caused by loading too many textures into the graphics card. That’s why texture atlases where invented.

Texture Atlases in Core Animation

By now you should know what a texture atlas is, but if you don’t check Owen’s description. Apart from the tools he mentions you can find nice GUI apps to convert a bunch of PNGs into a compact atlas, like Zwoptex. The problem is how to render only part of it with Core Animation.

This too is pretty easy. Imagine we have the following atlas with a simple 5 frame walking animation of our elephant:

(Notice this image doesn’t have a power of 2 width and height. Core Animation doesn’t require it, but is probably better if you do it anyway with your own atlases).

If we use the previous code to load this image into a layer we will see the above image. In addition we need to specify the contentsRect property:

CALayer *layer = [CALayer layer];
layer.contents = (id)img;

CGSize size = CGSizeMake( 160, 198 ); // size in pixels of one frame
CGSize normalizedSize = CGSizeMake( size.width/CGImageGetWidth(img), size.height/CGImageGetHeight(img) );

layer.bounds = CGRectMake( 0, 0, size.width, size.height );
layer.contentsRect = CGRectMake( 0, 0, normalizedSize.width, normalizedSize.height );

In this case we set the bounds property to the size in pixels of one frame, and the contentsRect property to the normalized rectangle of the first frame (meaning the top/left of the atlas is coordinate 0,0 and the bottom/right is 1,1). If we render that layer, we will only see the first frame.

Impertinent Implicit Animation

So how do we make that elephant walk? In theory we could just change the contentsRect property so it moved from the first frame to the second, then the third, etc. But we would find a little quirk of CALayer that is probably one of the reasons most people abandon Core Animation in frustration: all changes to its properties are automatically animated. What does that mean? It means if later in our code we do this to our elephant layer…

layer.contentsRect = CGRectMake( normalizedSecondFrame );

…we will end up seeing something like the image to the left. This is the result of the default implicit animation linked to the contentRect property, the same thing that gently fades views when you change the opacity property, or slides them when changing their position.

So that’s a bummer. We need to deactivate the default animation, and you can do that in multiple ways (all of them confusing, by the way). So before we get there, what more do we need for this to work?

Well, we need to time how long one frame shows before the next appears, then change to the next one in the sequence, maybe loop back and forth or just back from the beginning, and we need to do this while the sprite changes position in the screen.

We can do all that updating each sprite’s state every so often in our game loop, or with Timers, delegate callbacks, etc. But that sounds like lots of work for something that should be easier. This easier:

CABasicAnimation *anim = [CABasicAnimation animationWithKeyPath:@"sampleIndex"];

anim.fromValue = [NSNumber numberWithInt:1]; // initial frame
anim.toValue = [NSNumber numberWithInt:6]; // last frame + 1

anim.duration = 1.0f; // from the first frame to the 6th one in 1 second
anim.repeatCount = HUGE_VALF; // just keep repeating it
anim.autoreverses = YES; // do 1, 2, 3, 4, 5, 4, 3, 2

[layer addAnimation:anim forKey:nil]; // start

What’s that sampleIndex property you ask? No, is not part of the basic CALayer class. Let me show you how to subclass CALayer with a dozen lines of code so it can handle this.

MCSpriteLayer

Core Animation can interpolate between more than just the basic CALayer properties. It can actually do it with any values of the following types:

integers and doubles
CGRect, CGPoint, CGSize, and CGAffineTransform structures
CATransform3D data structures
CGColor and CGImage references

You see integer there? That means we can have Core Animation take care of the counting of frames, the looping, the duration, add timing functions, refresh the layers that need it automatically, and well, do all the things that Core Animation is supposed to do! We can synchronize that with other animatable properties (like position) with a Core Animation group in a transaction. The only thing we need is that integer property to be recognized. For that, nothing easier than to subclass CALayer:

#import <QuartzCore/QuartzCore.h>

@interface MCSpriteLayer : CALayer {
unsigned int sampleIndex;
}

@property (readwrite, nonatomic) unsigned int sampleIndex;

I decided to use the name sample index to avoid confusion with the frame property. That’s actually all the code you need to have a new animatable property. The previous code should already work, and cycle through sampleIndex values in an autoreverse loop. Problem is that alone is not going to change what we see on the screen!

#import "MCSpriteLayer.h"

@implementation MCSpriteLayer

@synthesize sampleIndex;

+ (BOOL)needsDisplayForKey:(NSString *)key;
{
return [key isEqualToString:@"sampleIndex"];
}

- (void)display;
{
unsigned int currentSampleIndex = ((MCSpriteLayer*)[self presentationLayer]).sampleIndex;
if (!currentSampleIndex)
return;

CGSize sampleSize = self.contentsRect.size;
self.contentsRect = CGRectMake(
((currentSampleIndex - 1) % (int)(1/sampleSize.width)) * sampleSize.width,
((currentSampleIndex - 1) / (int)(1/sampleSize.width)) * sampleSize.height,
sampleSize.width, sampleSize.height
);
}

@end

But that will. The display method will now be called any time the sampleIndex changes, and will change the contentRect to select the next element in the atlas as long as all the frames are the same size and you organize them by rows.

Two problems though. First, as we saw before, changing the contentRect will activate the default implicit animation. We can solve this by deactivating that animation:

+ (id < CAAction >)defaultActionForKey:(NSString *)aKey;
{
if ([aKey isEqualToString:@"contentsRect"])
return (id < CAAction >)[NSNull null];

return [super defaultActionForKey:aKey];
}

Second, what if our frames are not ordered by rows, or are all different sizes. In that case the class can call its delegate to provide a specific coordinate rectangle for each sample index. An example project is provided below containing examples of both fixed size, and variable size samples.

Update: I totally forgot to explain why the animation we created above goes from 1 to 6 instead of the expected 1 to 5 for 5 frames. It seems Core Animation transforms internally integers to float values to produce their animation. While the value goes through the decimals from 4 to 5, sampleIndex maintains a value of 4. For the animation to spend the same amount of time in frame 5 we have to actually animate through all the decimals from 5 to 6. So in summary, we always need to set the toValue to one more than the last sample index for the animation to run properly.

In Other News…

After last week’s blog post a number of people joined in the fun of Open Development (not sure I like that name, but whatever sticks). Don’t hesitate to check out all the participant blogs!

George Sealy from Acorn Heroes: Faerie’s Journey
Jerrod Putman from Tiny Tim Games: Safety First
Zaid Crouch from Third Raven: Project NP

If you want to participate, just start blogging, and send us some links through Twitter or on the comments. Is more fun (and less scary) when is not just one person doing it.

Apart from that Mike Acton from Insomniac decided to create an alternative iDevBlogADay blogging group (if you don’t know who Mike Acton or Insomniac are, you need some quality PlayStation time, OK?)! It’s called #altDevBlogADay and they are scheduled to start sometime this weekend. Go check them out!

CoreCastlevania Source Code!

Sorry about not giving you my elephant sprite to play, but I may still need it :) In its place, I found some Castlevania sprites of Richter running (Symphony of the Night maybe?)!

Have fun experimenting!

Download CoreCastlevania project

32 Responses

I really dig this method, Thank you!

Jason Lee; January 13, 2011, 8:30 AM

Reply
Awesome stuff. I’ve actually been curious about using core
animation, but a little nervous about jumping straight in.
(Conceivably I could do NP using CA and ditch cocos2d — might give
it a shot in the future). Might be worth some exploration after all
:) And yeah, Open Development is a little dry… Can’t say I have
any better ideas though…

Zaid Crouch; January 13, 2011, 5:53 PM

Reply
Also check this out: Create sprite sheets from SWFs
http://www.bit-101.com/blog/?p=2939

Mike Berg; January 13, 2011, 7:39 PM

Reply
Nice tutorial. After your post last week i decided to
develop my next game completely in the open too, and possibly open
source the finished game. But i haven’t actually started it yet
(still working on current projects).

Freerunnering; January 14, 2011, 4:04 AM

Reply
Great post, I’m trying out the demo and it runs great, the
only thing I noticed is that the sprite stops animating if you exit
the app (and open it again). thanks for sharing!

David; January 14, 2011, 6:47 PM

Reply
Hi David,

Yes, the active animations of a layer are automatically released when you go to the background. You need to recreate the animations and add them back to the layers when you come back. Easier way is to add the animation from applicationDidBecomeActive: instead of doing it from the controller’s init method like I am doing in the demo. Hope that helps :)

Miguel A. Friginal; January 14, 2011, 11:13 PM

Reply
I’m a little confused as to why you’re jumping all around
your texture atlas like that. Isn’t it easier to just move from
left to right, top to bottom, for each frame?

Michael Potter; January 16, 2011, 11:20 PM

Reply
@Michael: Indeed, but that would not allow me to explain
how to deal with the “not ordered by rows, or with frames of
different sizes” case. As you can see in the example code the
default case (left to right, top to bottom, same size) is covered
by the default display method; and the other one by the
displayLayer: delegate method.

Miguel A. Friginal; January 16, 2011, 11:43 PM

Reply
Hi There;

Instead of returning null for actions associated with a “contentsRect” change; why not just encapsulated the “layer.contentsRect = x” within a CATransaction.

Wouldn’t that be a better approach?

Nader Eloshaiker; January 17, 2011, 9:23 PM

Reply
@Nader: Hi!

I could do it in a transaction, but that would require me to add more code around any animations involving sampleIndex whose purpose is not immediately obvious.

Since I decided to create a sprite subclass of CALayer, and don’t see the point of animating the contentsRect for such a thing, I preferred to hide those details. If you check the code you will see that I do the same thing for the bounds property so changes to the sample size are not animated either (same deal, what’s the point of animating bounds when dealing with a frame-by-frame animated sprite?).

In case an animation is required for these 2 properties for a special case, you can still do all the normal things:

– use the delegate method actionForLayer:forKey:
– use the actions dictionary of the layer
– or use the style dictionary

All those will execute before defaultActionForKey: has a chance to do its thing.

Miguel A. Friginal; January 17, 2011, 10:48 PM

Reply
Pingback: Core Animating Interfaces at Under The Bridge
Amazing;

I have been using CADisplayLink and manual frame by frame animation at 60 FPS on an iPhone 3G and I could tell that the iPhone was struggling with all the animation I was managing on all the CALayers (all at varying frame rates, not all at 60 FPS). Using this approach seems to be more manageable for the phone. This leads me to believe that CoreAnimation is accessing the OpenGL mechanics directly and animating at that level.

However, (and yes there is one), there are some instances where I still have to use manual animation and it is more to do with colliding objects so that I get granular control over logic update and screen render per frame (there maybe a design pattern around this using Core Animation, I just haven’t discovered it yet).

A quirk I am noticing that I haven’t debugged yet (perhaps you can save me some time), is that the sampleIndex doesn’t seem to be consistently incrementing at every display call and nor does it reach the end value. I am using a duration value of 0.8 sec and trying to shift the contentsRect 10 times in that duration. So, either, something is not working as it should or the CoreAnimation cannot keep up with the 10 image frames in 0.8sec.

Nader Eloshaiker; January 20, 2011, 5:26 PM

Reply
Hi Nader!

Yes! That’s one thing I forgot to explain. It seems CA transforms integers to float for animating them, so what you need to do is set the animation with an end value 1 more than the number of samples you have. I.e., my example above with 5 frames should be:

anim.fromValue = [NSNumber numberWithInt:1]; // initial frame anim.toValue = [NSNumber numberWithInt:6]; // last frame + 1

I am thinking this is because the value of sampleIndex is animated as a float, through the decimals from 4 to 5 rounded as 4, and the moment it hits 5 it stops. What we need is that it animates through the decimals from 5 to 6 (rounded as 5) too. It should not have a problem with your duration of 0.8, that’s only 12.5 FPS.

I will update the post later, thanks!

Miguel A. Friginal; January 20, 2011, 5:49 PM

Reply
That kind does and doesn’t explain what I am seeing.

The floating point conversion explains why I see display being called with the same currentSampleIndex (oh and by the way, in the display, you should check to see if the current contentsRect equals the new one to be assigned and return rather than set the same value – CGRectEqualToRect).

However, what I am seeing is that with a GroupAnimation, the animation of contentRect cannot complete in time with the movement. Hence why it never reaches the end value (I wanted 10 images but it only ever reached 3-4). So I decided to remove the GroupAnimation and just use the animation of the TextureAtlas. Sure enough, the animation does not complete within the duration value, it is taking a considerably longer time to complete.

I am suspecting that my PNG that was generated using GIMP might be the issue. The Atlas is only 400×240 with each cell being 40*40, so it can’t be the size of the texture. Perhaps it might be the transparent background of the cell?

Nader Eloshaiker; January 20, 2011, 7:14 PM

Reply
Not sure what can cause that, but I would start making the atlas width and height a power of 2 (512×256).

You are right that display should return earlier if sampleIndex has not changed. This is a side-effect of int being converted to float, since display should not be called by a sampleIndex animation more than once with the same value (I already filled a bug with Apple about it). Instead of comparing CGRects I would just try comparing the sampleIndex with a static var or similar so it can return even sooner. I will add that to the sample code the moment I have some spare time, thanks!

Even so, I am not seeing this slow down you are experiencing when running sprites way faster than that, so there must be other reason. Please come back if you find out!

Miguel A. Friginal; January 20, 2011, 8:20 PM

Reply
OK I solved it and man was it a lame bug.

I had legacy code which was setting the position after the animation completed and for some reason it still executed the spritesheet animation as well.

At anyrate, I discovered that instead of using your approach to spritesheet animation which invloves overriding animatable properties, you can actually do this using keyframe animation and let CoreAnimation handle the animatable contentsRect property:

CAKeyframeAnimation *animAtlas = [CAKeyframeAnimation animationWithKeyPath:@"contentsRect"]; animAtlas.duration = timeVelocity; animAtlas.repeatCount = 1; animAtlas.values = travelAtlasSequence; // This is an array of normalised CGRect representing spritesheet map animAtlas.timingFunction = [CAMediaTimingFunction functionWithName:kCAMediaTimingFunctionLinear]; animAtlas.calculationMode = kCAAnimationDiscrete; //This tells CA not to try and insert values between each CGRect animAtlas.delegate = self;

This is by far, more simple to implement and you do not need all the overrides you had.

Nader Eloshaiker; January 22, 2011, 3:06 AM

Reply
Nice! Core Animation does provide lots of ways of doing the same thing (not sure if that’s good, although it helps when you have something specific in mind).

The part I like the most of using keyframe animation is not having to subclass, but that’s also the part that is more problematic for me, since knowing what frame is currently visible is important for blending and syncing animations together. The information stored in the layer will need to be stored then in a separate model object, and I think I prefer using the layer as the model.

The moment you have frames of different sizes (you can see in my example code how much space in the atlas this can save in certain cases), you will also need to change not only the contentsRect, but also the bounds.

I don’t quite understand your comment about “all the overrides” though. There are exactly 2 things overridden by my method:
- the display method. Is actually being used as recommended by Apple in their documentation.
- the contentsRect & bounds default animation. As I was saying in an early comment, I cannot really think of any case in which you will want these defaults to apply for sprite animation.
To top it off, creating a default animation for the sampleIndex property becomes also trivial via subclassing, and then the whole block of animating code can be reduced to:

layer.sampleIndex = 5;

As always is nice to know there are multiple options, but they are not quite the same. Use the one better suited to your project :)
Miguel A. Friginal; January 22, 2011, 6:45 PM

Reply
I agree with your idea of creating the sample index as it provides a way for me to do collision detection, but I prefer to keep the contentsRect managed by keyFrame.

I think that is where you design pattern has the most strength, being able to execute logic updates as core animation is executed (so long as you group the animations such as contentsRect and position with sampleIndex). Actually, I think this is brilliant and you don’t even need to deactivate the action for contentsRect this way.

I actually think we have solved the replacement of a master clock (Game Loop) with CoreAnimation. Absolutely brilliant.

Nader Eloshaiker; January 22, 2011, 7:34 PM

Reply
Nice method! :)

can this be used if you wanna animate for example a racetrack like in the classic game OUTRUN? (the view is from behind the car)

Chris; February 1, 2011, 6:59 AM

Reply
Hi Chris,

Outrun used scanline transformation methods to simulate 3D perspective. You can find a pretty detailed explanation of similar techniques here: http://www.gamedev.net/topic/373043-classic-outrun-esque-racing-game-display/page__view__findpost__p__3505894

Doing this with sprites is probably possible (and painful), but I would not recommend it over real 3D graphics nowadays :)

Miguel A. Friginal; February 1, 2011, 12:45 PM

Reply
Wow this was the best explanation I have found describing sprite animation. Thank you so much!

nate; April 5, 2011, 9:10 AM

Reply
Very nice code!
The example demonstrates a frame-by-frame animation at a fixed duration. Is it possible somehow to modify the code to allow the animation to increase and decrease the duration during the runtime?
I know that I can re-start the animation with a new duration, but it is not good enough and I need to be able to change the duration from the current running frame.

Alex; August 23, 2011, 11:12 AM

Reply
@Alex: Maybe check the 2nd part of the series? :) http://mysterycoconut.com/blog/2011/01/cag2/

Miguel A. Friginal; August 23, 2011, 12:21 PM

Reply
I was able to get this to work, but found out that it does not work with @2x atlas images?

At @2x the frames still sample as 1:1

Still trying to find out why…

Alex; August 26, 2011, 3:27 PM

Reply
@Alex: yes, the way you “get this to work” is downloading the code and compiling it. Even with the latest SDK it only needs to go through the automatic ‘project modernization’ and it runs fine.

About the @2x atlas, there is a simple reason: imageWithContentsOfFile: doesn’t automatically get 2x images. Use imageNamed: or roll your own method.

Miguel A. Friginal; August 27, 2011, 10:25 AM

Reply
I am not sure why you are getting offended, all I meant is that I was able to make it work with @2x atlas as you did explain yourself.

I do load the @2x images and have a special mark on them to be sure.
The problem is that the frames get corrupt…

Alex; August 27, 2011, 5:19 PM

Reply
Not offended, just making sure people reading later don’t get the idea the code doesn’t work :)

Even so, this was created a while ago, so going through the code really quick now, wondering what part can give problems with 2x images, the only thing I see is the use CGImageGetWidth/Height in the helper method initWithImage:sampleSize:, since is not taking into account the scale of the source UIImage. Best option is to send the scale of the image to this method too, and multiply it by the size.width and size.height in the line that initializes sampleSizeNormalized.

Hope that helps.

Miguel A. Friginal; August 27, 2011, 10:47 PM

Reply
Pingback: Using CALayers as HUD elements – Part I. | Bitongo
Pingback: Using CALayers as HUD elements – Part II. | Bitongo
I tried converting this to ARC and the only code which gets changed is the bridge between the CGImageRef and the UIImage in the init method:

– (id)initWithImage:(CGImageRef)img;
{
self = [super init];
if (self != nil)
{
self.contents = (__bridge id)img;
sampleIndex = 1;
}

return self;
}

Which works perfectly fine in the simulator and on an iPad 2, but crashes miserably on a first-gen iPad. ARC may be cleaning out under me somehow and I’m trying to get my head around this, but perhaps someone has done this before?

Renaud; December 14, 2011, 6:45 PM

Reply
The fact that the contentsRect is in the coordinate space with the image bounds as ((0,0),(1,1)) means the coordinates of the contentsRect have to be small fractions, which cannot be represented exactly as floating-point numbers unless the number of rows and columns are a power of 2. As a result of the inexactness, when mapped to the image, the rectangle won’t be at exact integer pixel coordinates, and blurring will happen. Does anyone have a solution for this?

bob; June 4, 2015, 5:56 PM

Reply
- Hey Bob, I talked a bit about this in the 4th article in the series, under “Scrolling and Blurry Pixels”.
  
  Miguel A. Friginal; June 5, 2015, 8:56 AM
  
  Reply

Mystery Coconut. Resident Alien