Creating Unit Tests With Xcode 5

Introduction

Test Driven Development (TDD) is a process that I'm passionate about. I also love working with Xcode, developing wonderful Apps for Apple's iOS devices. If you have tried to write Unit Tests and integrating them into your Xcode development in the past you may have experienced a lot of frustration. Truth is, it was a lot of confusing work to get it all setup right.

But with Apple's release of Xcode 5 all of that has changed for the better. Now, Unit Testing is a first-class citizen in the development process. In fact, Apple makes it so you have no extra work to do to start writing Unit Tests with Xcode 5. Yay! In fact you might even say that Xcode 5 makes it difficult to ignore writing unit test cases.

I thought it might be appreciated to create a detailed tutorial showing how to use and write Unit Tests for Xcode projects. We are not just going to learn how to use Xcode 5 for writing Unit Tests, we are going to examine how you design and write good Unit Tests.

Follow along if you want to learn something new and exciting.

Start a New Xcode Project

For our example we will keep things simple. Create a new Xcode 5 project, choosing an iOS Single View Application for our project template.

Choosing Single View Application Template

Click "Next" and fill out your project options in the next dialog. For our example we're only going to select iPhone for the Device. I called the project "UnitTestsCardExample". Click Next and choose a location for your project. I recommend the use of Git as a source control system. Xcode 5 will have that option checked for you by default.

Git is the default

After you have created your project you should see contents like this in your Xcode navigation pane on the left side.

Let's Write Some Code

For the purposes of understanding how to use Xcode 5 to create and manage Unit Tests, we're not going to concern ourselves with the user interface at all. In the Model-View-Controller paradigm, for this effort, I'm going to stay focused completely on the Model. What models should we use?

Through the iTunes University, an excellent iOS programming course is available from Stanford University. As with all the iTunes University material, it is available to the user at no cost. If you haven't been taking advantage of this amazing free resource, and the list of participating schools is amazing, you are missing out on something pretty special. The on-line class Developing iOS 7 Apps for iPhone and iPad is pretty good. What I'd like to do is just borrow a few of the models developed by the end of Lesson 2 and use them here to create some interesting Unit Tests with Xcode 5. You do not need to go through those lessons to follow along here. I'll be showing all the code you need to type.

Let's get started.

I want to create an Xcode "Group" to hold all of our Model code. Select the UnitTestsCardExample folder in the Navigation pane of Xcode and right click to get to New Group menu.

Call the new group "Models".

We will create the Models we will test in this Group. After we get our models defined we need to take a moment to talk about running Unit Tests in Xcode and how and where you write them. Also, please note that this is quite the opposite of Test Driven Development (TDD). We are approaching the unit test development question as if we already have model code to test. For our little exercise here they do not already exist, but we are going to type them all in and then write the unit tests. For a true TDD experience we would write tests for each model before we make the models work. I'm a strong advocate for TDD as a core development process. Again, I'm just trying to show how you would create unit tests that are absent from an existing Xcode 5 project.

The Models

Create a new model class called Card as a subclass of NSObject. For this first one I'll walk you through how that's done. With the newly created group Models selected, right-click and choose the menu option New File....

In the dialog window that opens, choose Objective-C class.

Click "Next" and fill in the Class information as follows:

Click "Next" again and make sure the Group is set to Models before you click "Create".

Replace the code of the Card.h file as follows.

#import <Foundation/Foundation.h>

@interface Card : NSObject

@property (strong, nonatomic) NSString *contents;
@property (nonatomic, getter=isChosen) BOOL chosen;
@property (nonatomic, getter=isMatched) BOOL matched;

- (int)match:(NSArray *)otherCards;

@end

Replace Card.m with:

#import "Card.h"

@interface Card()

@end

@implementation Card

- (int)match:(NSArray *)otherCards
{
    int score = 0;
    for (Card *card in otherCards)
    {
        if ([card.contents isEqualToString:self.contents])
        {
            score = 1;
        }
    }
    return score;
}

@end

There are 4 model classes involved in this initial exercise. Now that you have one created, add the other 3 with these code snippets.

Add the class PlayingCard as a subclass of Card. Here is the code for PlayingCard.h.

#import "Card.h"

@interface PlayingCard : Card

@property (strong, nonatomic) NSString *suit;
@property (nonatomic) NSUInteger rank;

+ (NSArray *)validSuits;
+ (NSUInteger)maxRank;

@end

And PlayingCard.m

#import "PlayingCard.h"

@implementation PlayingCard

@synthesize suit = _suit;

+ (NSArray *)validSuits
{
    return @[@"♥",@"♦",@"♠",@"♣"];
}

+ (NSArray *)rankStrings
{
    return @[@"?",@"A",@"2",@"3",@"4",@"5",@"6",@"7",@"8",@"9",@"10",@"J",@"Q",@"K"];
}

+ (NSUInteger)maxRank
{
    return [[self rankStrings] count] - 1;
}

- (void)setRank:(NSUInteger)rank
{
    if (rank <= [PlayingCard maxRank])
    {
        _rank = rank;
    }
}

- (NSString *)contents
{
    NSArray *rankStrings = [PlayingCard rankStrings];
    return [rankStrings[self.rank] stringByAppendingString:self.suit];
}

- (void)setSuit:(NSString *)suit
{
    if ([[PlayingCard validSuits] containsObject:suit])
    {
        _suit = suit;
    }
}

- (NSString *)suit
{
    return _suit ? _suit : @"?";
}

@end

Two more classes to go to complete our models. Add the class Deck as a subclass of NSObject. Here is Deck.h.

#import <Foundation/Foundation.h>
#import "Card.h"

@interface Deck : NSObject

- (void)addCard:(Card *)card atTop:(BOOL)atTop;
- (void)addCard:(Card *)card;
- (Card *)drawRandomCard;

@end

And Deck.m.

#import "Deck.h"

@interface Deck()
@property (strong, nonatomic) NSMutableArray *cards;
@end

@implementation Deck

- (NSMutableArray *)cards
{
    if (!_cards) _cards = [@[] mutableCopy];
    return _cards;
}

- (void)addCard:(Card *)card atTop:(BOOL)atTop
{
    if (atTop)
    {
        [self.cards insertObject:card atIndex:0];
    }
    else
    {
        [self.cards addObject:card];
    }
}

- (void)addCard:(Card *)card;
{
    [self addCard:card atTop:NO];
}

- (Card *)drawRandomCard
{
    Card *randomCard = nil;
    if ([self.cards count])
    {
        unsigned index = arc4random() % [self.cards count];
        randomCard = self.cards[index];
        [self.cards removeObjectAtIndex:index];
    }
    return randomCard;
}

@end

Our last model class PlayingCardDeck is a subclass of Deck. Here is the code for PlayingCardDeck.h.

#import "Deck.h"

@interface PlayingCardDeck : Deck

@end

And the PlayingCardDeck.m source...

#import "PlayingCardDeck.h"
#import "PlayingCard.h"

@implementation PlayingCardDeck

- (instancetype)init
{
    self = [super init];
    if (self)
    {
        for (NSString *suit in [PlayingCard validSuits])
        {
            for (NSUInteger rank = 1; rank <= [PlayingCard maxRank]; rank++)
            {
                PlayingCard *card = [[PlayingCard alloc] init];
                card.rank = rank;
                card.suit = suit;
                [self addCard:card];
            }
        }
    }
    return self;
}

@end

Some Basics About Xcode 5 and Unit Tests

Probably the first thing to review is what we are expected to do when we create unit tests. A discussion of why we write unit tests, and benefits that come from that, is another discussion. Let's just assume you know you want to write them. The thing is, a Unit Test is designed to test One Thing and One Thing Only. Our examples will clarify what the heck this might really mean.

Beginning with Xcode 5, Apple made Unit Testing a first class citizen. What do I mean by that? When you create a new project everything you need to write and run Unit Tests is included automatically. For example if you select the Xcode project document you will see that a test target was created for you.

The Unit Test Target

Also in your project navigator there is a new Group where your unit tests code goes. In our case it's called UnitTestsCardExampleTests. Inside that folder you will find a pre-built Unit Test class UnitTestsCardExampleTests.m.

When you select the file you will see and be able to edit its contents.

#import <XCTest/XCTest.h>

@interface UnitTestsCardExampleTests : XCTestCase

@end

@implementation UnitTestsCardExampleTests

- (void)setUp
{
    [super setUp];
    // Put setup code here. This method is called before the invocation of each test method in the class.
}

- (void)tearDown
{
    // Put teardown code here. This method is called after the invocation of each test method in the class.
    [super tearDown];
}

- (void)testExample
{
    XCTFail(@"No implementation for \"%s\"", __PRETTY_FUNCTION__);
}

@end

Let's take a minute and talk about this file. The first thing to note is that there is no .h file. All unit test code goes in a .m file. You can see that the framework for XCTest was automtically included. In Xcode 5, all unit test classes are subclasses of XCTestCase.

There are only 3 methods in here. We will discuss the setUp and tearDown methods a little later. The method testExample deserves a good look right now. After a glance at the code you may have concluded that it was automatically created as a failing unit test. Let's see what happens when we try to use it.

Running Unit Tests

If you check out the Product menu you will see the "Test" menu option.

You can select it or perform a Command-U to run it. This will ask Xcode to build your target image on the device or simulator selected in your Scheme. It will then find and run all XCTestCase subclasses and run any methods that begin with the prefix test. The method generated for us called testExample will run.

It's important to remember that all test prefixed methods will be run but you cannot assume control of the order they are run. There's some simple things you can do if for some reason order is important to you. We will cover that later.

Once we run we see a failure. It may take a few seconds since it has to build the test image and load it onto the simulator and then report the results. You should see an error that looks like this:

Sure enough, the testExample method does fail. The idea here is that you are encouraged to replace the test condition for one you want. Or just delete the method entirely. For the moment just comment the line of code out so the method looks like this:

- (void)testExample
{
//    XCTFail(@"No implementation for \"%s\"", __PRETTY_FUNCTION__);
}

If you run the tests again you will see the failure go away. But what should we be doing to create meaningful unit tests?

Create Unit Tests

I usually use a class like the generated UnitTestsCardExampleTests class for more general test code. So for now, it's okay with me that nothing happens there. In fact I usually just delete the existing class since I'm not interested in unused classes hanging around. However, I'm making an exception as part of this teaching exercise because there's a really nice gem of code in there that I want to talk about a little later. We'll get back to it. Just leave the method commented out and for now we will ignore that Test Case class.

Card Model

My usual practice is to create unit test classes for each model I want to test. So let's do that. It makes sense to begin with the Card class, so let's see what we can do.

The first thing to do is to create a new unit test class. With the UnitTestsCardExampleTests group selected, right click and select the "New File..." menu.

This time we will select "Objective-C test case class" for our template.

On the next screen we will define a new unit test class called CardTestCase. Xcode will have already selected the class XCTestCase as the superclass.

Note that on the next screen, when Xcode asks you to save the code, the Group and Target have already been properly selected for this new test case class.

The newly created CardTestCase.m class should look familiar. Delete the testExample method. We are going to write new test methods.

As I think about which unit tests need to be created, I like to configure Xcode to use the Assistant editor and then manually choose the .h file related to the test case class. In this case we want to see the Card.h contents in the Assistant editor.

Since we are certain we will be writing unit tests against the Card class, it's a sure thing we need to import the class header to get its definition. Add the import statement for our model class as shown:

#import <XCTest/XCTest.h>
#import "Card.h"

@interface CardTestCase : XCTestCase

We should write some unit tests to check the behavior of that (int)match:(NSArray *)otherCards method. A couple of ideas come to mind. Let's create a unit test where a card should be able to match against an identical card. Look at the implementation for Card.m...

- (int)match:(NSArray *)otherCards
{
    int score = 0;
    for (Card *card in otherCards)
    {
        if ([card.contents isEqualToString:self.contents])
        {
            score = 1;
        }
    }
    return score;
}

The Positive Test Case

It looks like the method is pretty simple. If any card matches in internal contents, it returns a 1. If no match is found it answers 0. Begin with the following new test method.

- (void)testMatchesDifferentCardWithSameContents
{
}

We have defined a new test with a fairly descriptive method name. If you do this well, the intent of the test is revealed through the method name. It just has to begin with the prefix test. All unit test methods answer (void).

Starting to fill the method in...

- (void)testMatchesDifferentCardWithSameContents
{
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"one";
}

Here we create 2 instances of Card and put the same contents in each. Since the match: method wants an NSArray as an input argument we could create a temporary variable to hold the card we want to compare. Here's an update to the method.

- (void)testMatchesDifferentCardWithSameContents
{
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"one";
    NSArray *handOfCards = @[card2];
}

I think we can now add code to actually test that match: method.

- (void)testMatchesDifferentCardWithSameContents
{
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"one";
    NSArray *handOfCards = @[card2];
    int matchCount = [card1 match:handOfCards];
}

We have two cards and we ask the first card if it matches the second card. It should. Here's how we write the last step in the test to confirm this behavior of our Card object.

- (void)testMatchesDifferentCardWithSameContents
{
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"one";
    NSArray *handOfCards = @[card2];
    int matchCount = [card1 match:handOfCards];
    XCTAssertEqual(matchCount, 1, @"Should have matched");
}

The XCTAssertEqual function compares the first argument with the second. In this case, if the arguments are equal the test passes. If they are not, and in this case they should be, the the NSString that is the third argument is shown in the error message log. We expect the cards to match of course and we expect the match: method should answer the number 1.

When we run tests via Command-U we get what we expected.

Xcode 5 has a new Navigator view called the Test Navigator. Here you can see it in action. Just click the icon highlighted as shown in the Navigator toolbar.

The Test Navigator collects together all the Unit Tests you have defined for your project along with their run results. You can also re-run individual tests by clicking on the little icon on the right edge for each test entry. When you hover the cursor over the icon it changes to a tiny run icon.

Also when you select the method in the Test Navigator view the source code of the test is conveniently showing.

The Negative Test Case

Okay. That was cool. But there are a lot of interesting things we should do now. We should prove the "negative" condition works as expected too. Here's my initial next method. Just write it in the same test case class file (CardTestCase.m).

- (void)testDoesNotMatchDifferentCard
{    
}

Again, the method name reveals the intent of the test and it begins with the test prefix. Setting this test up should be a lot like the previous one with only one small difference.

- (void)testDoesNotMatchDifferentCard
{
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"two";
}

This time we want the second card to be different from the first. We can prove that different cards, when run past the match: method should answer a 0 result. Here's how it looks with the logic completed.

- (void)testDoesNotMatchDifferentCard
{
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"two";
    NSArray *handOfCards = @[card2];
    int matchCount = [card1 match:handOfCards];
    XCTAssertEqual(matchCount, 0, @"No matches, right?");
}

Run the unit tests again and we get green checkboxes. Making progress.

Test Case Permutations

Since the match: method we are testing can accept an array of input cards, we should validate that it works as expected if we give a few cards and only one matches.

- (void)testMatchesForAtLeastOneCard
{
}

We can build a hand of 2 cards, each one unique, but have one of them be a match and see what happens. Rather than draw out the individual steps, since you've now been through it twice, I'm just going to show the test method as complete.

- (void)testMatchesForAtLeastOneCard
{
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"two";
    Card *card3 = [[Card alloc] init];
    card3.contents = @"one";
    NSArray *arrayOfCards = @[card2, card3];
    int matchCount = [card1 match:arrayOfCards];
    XCTAssertEqual(matchCount, 1, @"Should have matched at least 1");
}

Here we are creating 3 cards. The second and third card are different from each other but the third one should cause a match against the original first card. When we run this again we get more green boxes!

What About Those SetUp and TearDown Methods?

The XCTestCase framework will run every one of your test methods. Sometimes you find yourself doing the same "preamble" work in every test case. Or maybe you need to access a document and want to create it in a specific way before every test method runs. These scenarios are supported by the setUp and tearDown methods.

Take a look at the default implementation for each.

- (void)setUp
{
    [super setUp];
    // Put setup code here. This method is called before the invocation of each test method in the class.
}

- (void)tearDown
{
    // Put teardown code here. This method is called after the invocation of each test method in the class.
    [super tearDown];
}

The setUp method begins by calling the super class' setUp method and then performs whatever you need to have happen before the test. The tearDown method does the "local" work first and then calls the super class to perform whatever it needs as part of tearDown. And what about those existing comments? Is it clear how often these methods will actually run? Will the setUp method run once for all the methods in the class, or will it run repeatedly with each individual test method? How often will the tearDown method run?

Our CardTestCase class currently has three test methods. We could probably conduct a little experiment to get a precise answer to those questions.

But first we should go back and examine that "little gem" I mentioned. Go back and look at the UnitTestsCardExampleTests.m test case class. Remember the test we commented out? First of all, it's pretty silly to keep it around since we know it does nothing and, more importantly, it is deceptive to keep the test in there. The passing but useless result is shown as part of every Unit Tests run. We gain a false sense that we have a useful and positive test result in there based upon the green checkboxes we get. But it's not real. Either we delete the test case, the whole class can go away if we want, or make it do something useful. But I digress. Uncomment that line of code in testExample and let's re-run to see exactly what is getting reported in the log.

First the code restored...

- (void)testExample
{
    XCTFail(@"No implementation for \"%s\"", __PRETTY_FUNCTION__);
}

And the result:

Here's the part that I think is the "little hidden gem". The error says...

failed - No implementation for "-[UnitTestsCardExampleTests testExample]"

The __PRETTY_FUNCTION__ is very smartly logging both the name of the class and the name of the current method. How many times have you used something like NSLog(@"In method xyz in class ABC"); as a tracing function?

This means we could write a really clean logging method that would tell us exactly where we are. Maybe...

NSLog(@"%s", __PRETTY_FUNCTION__);

Let's clean up some code and then put our new knowledge to good use in an experiment to see how setUp and tearDown really work. First; cleanup.

Delete the Unused Test Case

Remember that when we run Unit Tests the XCTest framework looks at all subclasses of XCTest and then runs any instance methods that begin with the test prefix. We can delete the UnitTestsCardExampleTests class that was automatically generated for us by Xcode when we created the project. We currently have no meaningful test behavior in this test case class so we can delete it and create it again if needed later. Select the test case class and right-click to select the "Delete" menu action.

At this point Xcode presents a Delete Dialog. Choose "Move to Trash".

If you run unit tests again you will notice the total number of reported test cases drops because of this removal. But we also know the test case did nothing, so this is not a compromise of overall test quality.

Experiments With SetUp and TearDown

Select the CardTestCase.m file and modify the setUp method as follows.

- (void)setUp
{
    [super setUp];
    NSLog(@"%s", __PRETTY_FUNCTION__);
}

We are using the trick we learned earlier to log everytime this method runs. Make a similar change to the tearDown method.

- (void)tearDown
{
    NSLog(@"%s", __PRETTY_FUNCTION__);
    [super tearDown];
}

Before we run I want to make one more change to illustrate how the setUp and tearDown methods interact with the test methods. Modify the testMatchesForAtLeastOneCard method by adding an NSLog() call at the beginning.

- (void)testMatchesForAtLeastOneCard
{
    NSLog(@"%s doing work...", __PRETTY_FUNCTION__);
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"two";

We now have logging in several interesting places. Run the tests!

If the console log is not showing you may need to open that view using the middle icon in the set of three in the code toolbar.

Sometimes the console log may be only half of the lower view pane, like this:

In this case you want to deselect the middle icon of the three in the lower right of the window. Only the right icon should be blue (selected).

Lets take a look at the console log from when that last batch of unit tests were run. The view you are seeing clears each time you run unit tests, so you will be seeing the results of the latest run. I've replicated the entire console run log here for review.

Test Suite 'All tests' started at 2014-01-24 13:23:56 +0000
Test Suite 'UnitTestsCardExampleTests.xctest' started at 2014-01-24 13:23:56 +0000
Test Suite 'CardTestCase' started at 2014-01-24 13:23:56 +0000
Test Case '-[CardTestCase testDoesNotMatchDifferentCard]' started.
2014-01-24 07:23:56.650 UnitTestsCardExample[89838:70b] -[CardTestCase setUp]
2014-01-24 07:23:56.651 UnitTestsCardExample[89838:70b] -[CardTestCase tearDown]
Test Case '-[CardTestCase testDoesNotMatchDifferentCard]' passed (0.001 seconds).
Test Case '-[CardTestCase testMatchesDifferentCardWithSameContents]' started.
2014-01-24 07:23:56.651 UnitTestsCardExample[89838:70b] -[CardTestCase setUp]
2014-01-24 07:23:56.652 UnitTestsCardExample[89838:70b] -[CardTestCase tearDown]
Test Case '-[CardTestCase testMatchesDifferentCardWithSameContents]' passed (0.001 seconds).
Test Case '-[CardTestCase testMatchesForAtLeastOneCard]' started.
2014-01-24 07:23:56.652 UnitTestsCardExample[89838:70b] -[CardTestCase setUp]
2014-01-24 07:23:56.652 UnitTestsCardExample[89838:70b] -[CardTestCase testMatchesForAtLeastOneCard] doing work...
2014-01-24 07:23:56.653 UnitTestsCardExample[89838:70b] -[CardTestCase tearDown]
Test Case '-[CardTestCase testMatchesForAtLeastOneCard]' passed (0.001 seconds).
Test Suite 'CardTestCase' finished at 2014-01-24 13:23:56 +0000.
Executed 3 tests, with 0 failures (0 unexpected) in 0.003 (0.003) seconds
Test Suite 'PlayingCardTestCase' started at 2014-01-24 13:23:56 +0000
Test Suite 'PlayingCardTestCase' finished at 2014-01-24 13:23:56 +0000.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'UnitTestsCardExampleTests.xctest' finished at 2014-01-24 13:23:56 +0000.
Executed 3 tests, with 0 failures (0 unexpected) in 0.003 (0.004) seconds
Test Suite 'All tests' finished at 2014-01-24 13:23:56 +0000.
Executed 3 tests, with 0 failures (0 unexpected) in 0.003 (0.005) seconds

Look for the line where it says: Test Suite 'CardTestCase' started at 2014-01-24 13:23:56 +0000.

Note that your date and time entry will be different from mine. This is where XCTestCase began to run the CardTestCase. Keep following the log until you find the entry that contains Test Suite 'CardTestCase' finished. I've extracted just that section below.

Test Suite 'CardTestCase' started at 2014-01-24 13:23:56 +0000
Test Case '-[CardTestCase testDoesNotMatchDifferentCard]' started.
2014-01-24 07:23:56.650 UnitTestsCardExample[89838:70b] -[CardTestCase setUp]
2014-01-24 07:23:56.651 UnitTestsCardExample[89838:70b] -[CardTestCase tearDown]
Test Case '-[CardTestCase testDoesNotMatchDifferentCard]' passed (0.001 seconds).
Test Case '-[CardTestCase testMatchesDifferentCardWithSameContents]' started.
2014-01-24 07:23:56.651 UnitTestsCardExample[89838:70b] -[CardTestCase setUp]
2014-01-24 07:23:56.652 UnitTestsCardExample[89838:70b] -[CardTestCase tearDown]
Test Case '-[CardTestCase testMatchesDifferentCardWithSameContents]' passed (0.001 seconds).
Test Case '-[CardTestCase testMatchesForAtLeastOneCard]' started.
2014-01-24 07:23:56.652 UnitTestsCardExample[89838:70b] -[CardTestCase setUp]
2014-01-24 07:23:56.652 UnitTestsCardExample[89838:70b] -[CardTestCase testMatchesForAtLeastOneCard] doing work...
2014-01-24 07:23:56.653 UnitTestsCardExample[89838:70b] -[CardTestCase tearDown]
Test Case '-[CardTestCase testMatchesForAtLeastOneCard]' passed (0.001 seconds).
Test Suite 'CardTestCase' finished at 2014-01-24 13:23:56 +0000.

Here is the way I read the story this test case log tells.

  1. The method testDoesNotMatchDifferentCard was first to run
  2. CardTestCase setUp shows that the setUp method ran first for that test
  3. CardTestCase tearDown ran next
  4. I believe that the body of the testDoesNotMatchDifferentCard method was executed between the setUp and tearDown but we do not log anything
  5. testDoesNotMatchDifferentCard was logged to have passed
  6. The method testMatchesDifferentCardWithSameContents was next to run
  7. The setUp and tearDown were logged and again we have no logging during the content of that method
  8. `testMatchesDifferentCardWithSameContents' passed
  9. testMatchesForAtLeastOneCard ran next. This one will reveal more of the entire flow because we had inserted logging into its method
  10. CardTestCase setUp ran
  11. And then we see our log from inside the test method: -[CardTestCase testMatchesForAtLeastOneCard] doing work...
  12. We can then see that the tearDown ran and the method completed as passing
  13. The final entry from that section of the log shows Test Suite CardTestCase finished

What we learned from our experiment is that the setUp and tearDown methods are called for each test method in our Test Case class.

We can remove the diagnostic information now by either deleting or commenting-out the lines we added containing the __PRETTY_FUNCTION__ function.

Testing More Models

You are probably starting to get the idea of how all this can work. But we have a few more model classes and really should write unit tests for those too. In the process of working on the next one we will learn more about ways to think about unit test design.

PlayingCard Model

The PlayingCard class provides for some subtle and nuanced situations for test cases. As we did before, create the new test case class PlayingCardTestCase. As we did before we know we should import the model header file in our new test case class.

#import <XCTest/XCTest.h>
#import "PlayingCard.h"

@interface PlayingCardTestCase : XCTestCase

We can also delete the testExample method. Let's get a look at the PlayingCard.h file as we consider what tests we should create.

#import "Card.h"

@interface PlayingCard : Card

@property (strong, nonatomic) NSString *suit;
@property (nonatomic) NSUInteger rank;

+ (NSArray *)validSuits;
+ (NSUInteger)maxRank;

@end

There are class methods that declare the valid card suits and the maximum card ranking. For the previous model class we examined, Card, we didn't concern ourselves too much with writing test cases to ensure the setters and getters work correctly for the public properties. There were no special methods that override the default behaviors. But look at the methods defined in PlayingCard.m. I used Xcode's ability to collapse method body contents so we only see the names.

The suit property has special setter and getter methods that override the default behaviors. The tip-off is the @synthesize suit = _suit; line of code. With modern Objective-C and ARC, the only reason to create an @synthesize is if both the setter and getter have been overridden. Let's take a closer look.

The suit getter method implementation looks pretty simple.

- (NSString *)suit
{
    return _suit ? _suit : @"?";
}

It looks like it would be a simple matter here to write a test case that proves when the value of suit is nil, a "?" character is answered. Okay. What about the setter method?

- (void)setSuit:(NSString *)suit
{
    if ([[PlayingCard validSuits] containsObject:suit])
    {
        _suit = suit;
    }
}

The setter is making a sanity check. If the argument given is evidently not contained in the validSuits array it is ignored and left at nil. That looks interesting from a testing perspective. We will want to write unit tests to validate the contents of valid suits and check what happens when good suit values are set and bad suit values are set. For this class we now realize we want to test the accessors of that property since it has special behaviors.

It's also worth thinking about what can happen over the life-cycle of a development project. As it starts out we can clearly see the developer of this class was thinking about a typical card deck where there are exactly 4 suits (Hearts, Diamonds, Clubs, and Spades). But what would happen if in the future someone wants to use this class for a specialized deck that has less than the typical 4 suits? Or what if there were more than 4? Or even, what would happen if a developer accidently makes a typo and changes the symbol of one of the valid 4 suits? One of the strengths of having good unit test coverage is that it can protect you from accidental changes to behavior of your objects.

We're going to craft a few careful test cases here to protect the requirements as implemented by this class. It may seem like a lot of work ahead of us, but you will see this is still pretty easy and simple to do.

Wait. You might not have caught what I meant by that "requirements" statement. I tend to think of Unit Tests as having multiple purposes. First, if you have a good suite of test cases, you are protected from future code failures -- provided that the tests are always run. Second, you can capture your requirements for this model class by describing them with unit tests. And third, once you have a decent set of unit tests "shored-up" around your model classes, you can refactor code fearlessly and simplify. The unit tests will always tell you if your code changes broke something. I never feel safe refactoring or rewriting anything but very simple code unless they are unit tests in place.

Let's write a simple test first. We would like to be "protected" if a developer, maybe even our future self, adds or modifies what defines valid suits for PlayingCard.

Add this test method and then we'll review what it does.

- (void)testTheValidSuits
{
    NSArray *theSuits = [PlayingCard validSuits];
    int howMany = [theSuits count];
    XCTAssertEqual(howMany, 4, @"Should be only 4");
    XCTAssertTrue([theSuits containsObject:@"♥"], "@Must have a heart");
    XCTAssertTrue([theSuits containsObject:@"♦"], "@Must have a diamond");
    XCTAssertTrue([theSuits containsObject:@"♠"], "@Must have a spade");
    XCTAssertTrue([theSuits containsObject:@"♣"], "@Must have a club");
}

What I'm doing here is exercising the public class method validSuits. The test checks that there are exactly 4 suits and that they are the types expected. This kind of unit test may seem a little trivial but I think of these kinds of tests as a way fo checking that core knowledge within PlayingCard will continue to work as expected.

Add this test method:

- (void)testSetSuitAnyValidAccepted
{
    PlayingCard *card = [[PlayingCard alloc] init];
    [card setSuit:@"♥"];
    XCTAssertEqualObjects(card.suit, @"♥", "Should be an Heart");
    [card setSuit:@"♦"];
    XCTAssertEqualObjects(card.suit, @"♦", "Should be a Diamond");
    [card setSuit:@"♠"];
    XCTAssertEqualObjects(card.suit, @"♠", "Should be a Spade");
    [card setSuit:@"♣"];
    XCTAssertEqualObjects(card.suit, @"♣", "Should be a Club");
}

Remember that the suit property had special accessors and setter methods? Here we are checking that when we set the suit value with a proper value, and I'm checking all 4, that they work. In fact this test case is also validating the getter method as well.

It makes sense to write a unit test to evaluate negative inputs. Add this test:

- (void)testSetSuitInvalidRejected
{
    PlayingCard *card = [[PlayingCard alloc] init];
    [card setSuit:@"A"];
    XCTAssertEqualObjects(card.suit, @"?", "Should not have been recognized");
    XCTAssertNotEqualObjects(card.suit, @"A", "Should not have matched");
}

Here we are checking if we attempt to set suit to a nonsense value that it will both always answer ? when checked and that suit did not accept the nonsense input.

Deck Model

The Deck model provides interesting behaviors. Let's see what we can do to write unit tests for it. As we did earlier for PlayingCard, create a new Objective-C Test Case Class called DeckTestCase. Add the model's import to the test case following the pattern we've seen before.

#import <XCTest/XCTest.h>
#import "Deck.h"

@interface DeckTestCase : XCTestCase

If we set the Assistant Editor manually to view the model header we can evaluate what we might want to test.

#import <Foundation/Foundation.h>
#import "Card.h"

@interface Deck : NSObject

- (void)addCard:(Card *)card atTop:(BOOL)atTop;
- (void)addCard:(Card *)card;
- (Card *)drawRandomCard;

@end

That (Card *)drawRandomCard method looks ripe for unit tests. We could write tests for drawing a card from an empty deck; drawing from a deck with only one card in it; drawing random cards and verify they are not the same (assumption is the card deck has unique cards); and even drawing all the cards from a deck to prove there should be none left. Here we go. Before we add test methods to our DeckTestCase we can delete the uninteresting testExample method.

Let's see what the unit test for an empty deck would do. Add this test method.

- (void)testDrawCardFromEmptyDeckAnswersNoCard
{
    Deck *deck = [[Deck alloc] init];
    Card *drawnCard = [deck drawRandomCard];
    XCTAssertNil(drawnCard, @"Should not crash; just answer a nil object");
}

Here we are creating a new Deck object but not adding any cards to it. Then we attempt to draw a random Card from the deck. Our unit test checks that we should get a nil result.

This time, we will create a test where the Deck has only one Card in it. When we draw a random card it would hardly be random at all and should be easy to determine. Add this test method.

- (void)testOneCardDeckShouldAnswerThatCard
{
    Deck *deck = [[Deck alloc] init];
    Card *card = [[Card alloc] init];
    card.contents = @"test";
    [deck addCard:card];
    Card *drawnCard = [deck drawRandomCard];
    XCTAssertEqualObjects(card, drawnCard, @"Should have drawn the same card we added.");
}

We create one card and put it in our deck. When we draw a random card we check that it is the same as the card we originally put in the deck.

Writing the test to check that a small deck with, say two cards in it, yield different cards as each is drawn should be simple. Here is the test method.

- (void)testDrawnRandomCardsAreDifferent
{
    Deck *deck = [[Deck alloc] init];
    Card *card1 = [[Card alloc] init];
    card1.contents = @"one";
    Card *card2 = [[Card alloc] init];
    card2.contents = @"two";
    [deck addCard:card1];
    [deck addCard:card2];
    Card *drawnCard1 = [deck drawRandomCard];
    Card *drawnCard2 = [deck drawRandomCard];
    XCTAssertNotNil(drawnCard1, @"Should have found a card");
    XCTAssertNotNil(drawnCard2, @"Other card should have been found too");
    XCTAssertNotEqualObjects(drawnCard1, drawnCard2, @"The cards must be different");
}

How about a test that fills a deck with several cards and then proves, after it draws them all out, that the deck is empty and that it held the correct number of cards? Another test method to add.

- (void)testDeckWithMultipleCardsWillRandomlyDrawThemAll
{
    Deck *deck = [[Deck alloc] init];
    int numberOfCards = 16;  // arbitrary number > 1
    Card *card;
    for (int index = 0; index < numberOfCards; index++)
    {
        card = [[Card alloc] init];
        card.contents = [NSString stringWithFormat:@"%d", index];
        [deck addCard:card];
    }
    Card *randomlyDrawnCard;
    for (int index = 0; index < numberOfCards; index++)
    {
        randomlyDrawnCard = [deck drawRandomCard];
        XCTAssertNotNil(randomlyDrawnCard, @"Should have found a card.");
    }
    randomlyDrawnCard = [deck drawRandomCard];
    XCTAssertNil(randomlyDrawnCard, @"No more cards left.");
}

In this test case we decide upon a Deck with 16 cards in it. We populate the deck with unique cards by giving each Card contents with a number. Then we draw 16 cards out of our Deck, checking that they really are Card objects, and then attempt to draw one more card and prove it is nil.

We could enhance a test like this, since each card is well known, and build a list of expected cards and then as we pull the random ones out of the deck, check each one off to make sure we saw them all.

PlayingCardDeck Model

The last model for us to test is PlayingCardDeck. Let's look at its header file to see the public API.

#import "Deck.h"

@interface PlayingCardDeck : Deck

@end

Nothing special going on here. It must all be specialized behavior while overriding methods in the CardDeck superclass. Let's take a look at the PlayingCardDeck.m contents.

#import "PlayingCardDeck.h"
#import "PlayingCard.h"

@implementation PlayingCardDeck

- (instancetype)init
{
    self = [super init];
    if (self)
    {
        for (NSString *suit in [PlayingCard validSuits])
        {
            for (NSUInteger rank = 1; rank <= [PlayingCard maxRank]; rank++)
            {
                PlayingCard *card = [[PlayingCard alloc] init];
                card.rank = rank;
                card.suit = suit;
                [self addCard:card];
            }
        }
    }
    return self;
}

@end

The biggest difference here is the the PlayingCardDeck gets initialized with instances of PlayingCard instead of plain Card and that the deck is populated with a ranks and suits defined by the PlayingCard class. Still, it seems there are some very useful unit tests we could create.

We first need to create the Unit Test TestCase Class. Create a new Objective-C Test Case Class called PlayingCardDeckTestCase. Remove the testExample method. Also add the model's #import statement.

#import <XCTest/XCTest.h>
#import "PlayingCardDeck.h"

@interface PlayingCardDeckTestCase : XCTestCase

Since we saw from examining the model's code that it has a method to populate a complete card deck, we could write a unit test to check that behavior. We willl write this one in piece-meal fashion; like we did at the beginning of this tutorial. Here's the initial test method.

- (void)testPlayingCardDeckHasTheCorrectInitialCards
{
}

The name is intention revealing. We can begin by building a PlayingCardDeck and then creating a sorted place to put each card, by suit.

- (void)testPlayingCardDeckHasTheCorrectInitialCards
{
    PlayingCardDeck *deck = [[PlayingCardDeck alloc] init];
    NSMutableArray *hearts = [@[] mutableCopy];
    NSMutableArray *diamonds = [@[] mutableCopy];
    NSMutableArray *clubs = [@[] mutableCopy];
    NSMutableArray *spades = [@[] mutableCopy];
}

Now this gets interesting. We want to draw random cards from this deck but to check the model objects we have to import another header. We need PlayingCard.

#import <XCTest/XCTest.h>
#import "PlayingCardDeck.h"
#import "PlayingCard.h"

@interface PlayingCardDeckTestCase : XCTestCase

Okay we can now add the random drawing work to the test case, sorting the drawn cards by suit.

- (void)testPlayingCardDeckHasTheCorrectInitialCards
{
    PlayingCardDeck *deck = [[PlayingCardDeck alloc] init];
    NSMutableArray *hearts = [@[] mutableCopy];
    NSMutableArray *diamonds = [@[] mutableCopy];
    NSMutableArray *clubs = [@[] mutableCopy];
    NSMutableArray *spades = [@[] mutableCopy];
    PlayingCard *randomCard;
    do {
        randomCard = (PlayingCard *)[deck drawRandomCard];
        if ([randomCard.suit isEqualToString:@"♥"]) [hearts addObject:randomCard];
        if ([randomCard.suit isEqualToString:@"♦"]) [diamonds addObject:randomCard];
        if ([randomCard.suit isEqualToString:@"♠"]) [spades addObject:randomCard];
        if ([randomCard.suit isEqualToString:@"♣"]) [clubs addObject:randomCard];
    } while (randomCard);
}

This algorithm will continue to draw random cards until it picks up a nil. You may remember we have already written a test case that proved we can rely on nil to indicate an empty deck. As each card is drawn we check the suit and sort it accordingly.

Time to add some verifications and wrap this unit test up.

- (void)testPlayingCardDeckHasTheCorrectInitialCards
{
    PlayingCardDeck *deck = [[PlayingCardDeck alloc] init];
    NSMutableArray *hearts = [@[] mutableCopy];
    NSMutableArray *diamonds = [@[] mutableCopy];
    NSMutableArray *clubs = [@[] mutableCopy];
    NSMutableArray *spades = [@[] mutableCopy];
    PlayingCard *randomCard;
    do {
        randomCard = (PlayingCard *)[deck drawRandomCard];
        if ([randomCard.suit isEqualToString:@"♥"]) [hearts addObject:randomCard];
        if ([randomCard.suit isEqualToString:@"♦"]) [diamonds addObject:randomCard];
        if ([randomCard.suit isEqualToString:@"♠"]) [spades addObject:randomCard];
        if ([randomCard.suit isEqualToString:@"♣"]) [clubs addObject:randomCard];
    } while (randomCard);
    NSUInteger expectedCount = 13;
    XCTAssertEqual([hearts count], expectedCount, @"Should be 13 cards");
    XCTAssertEqual([diamonds count], expectedCount, @"Should be 13 cards");
    XCTAssertEqual([spades count], expectedCount, @"Should be 13 cards");
    XCTAssertEqual([clubs count], expectedCount, @"Should be 13 cards");
}

There's one more test I would like to add. It's a subtle thing to check. I want to be sure that the cards we pull out of this deck are instances of PlayingCard.

Add this method and then let's review.

- (void)testPlayingCardDeckAnswersPlayingCards
{
    PlayingCardDeck *deck = [[PlayingCardDeck alloc] init];
    id card = [deck drawRandomCard];
    XCTAssertTrue([card isKindOfClass:[PlayingCard class]], @"We should be drawing instances of PlayingCard from this deck.");
}

The unusual step in this test case is that I wanted to leave the resulting object pulled out of the deck randomly to be "neutral". That is, I didn't want the unit test to type-cast the resulting object and, instead, check what kind of object instance it is. Hence the use of id (which is inherently a pointer to any kind of object).

Conclusion

I hope this somewhat lengthy exercise proves useful to you. It can be so helpful to the overall maintenance of code for larger projects to have a suite of unit tests that confirm the integrity and quality of the source throughout its life-cycle.

Another observation that I have is that once you get used to thinking about Unit Tests as you write code, you tend to write code in nice tight modular pieces. You write code that is both testable and less likely to be tangled up with other code, or even worse, tangled up with controllers.

One area I did not discuss are all the different kinds of assertion tests you can perform within the XCTestCase framework. You can even write unit tests to check if the proper exception conditions are thrown. Also, you may have noticed that it often takes more work to write a good suite of unit tests than it does to create the models themselves. That's true, but I think the trade-off in reliability and improved design thinking that comes from focusing on Unit Testing is worth the effort.

A last piece to share about creating Unit Tests with Xcode 5: In addition to making the whole process simpler and easier, Apple made it very straight-forward to add Unit Test support to existing Xcode projects you have already created. Do a little research and you'll find it built-in to Xcode 5.

That's it. Have fun and keep writing great software with Xcode.

Questions or Comments? Downloads

Feel free to contact the author with comments and questions. My contact information is:

Stephan B Wessels

stevewessels@me.com

Personal Web Home: http://www.preeminent.org/steve/

My Blog: http://squeak.preeminent.org/blog/

Board Games Blog: http://preeminent.org/wp/awwos/

My Linked-In Profile: LINKED-IN Profile


05-Mar-2014

Document Version: 1.3

Downloads

This tutorial as a PDF.

Zip file of the Xcode project from this tutorial.