Continuous Integration for iOS with Jenkins

Development Tools

Whether you’re just starting a new app or maintaining a legacy code base, Continuous Integration (CI) is a valuable addition to your workflow. Software teams use CI to run a series of scripts or automated tests after each commit to a central repository, to gauge the performance and quality of the codebase. CI reduces the risk associated with multiple developers pushing and pulling code on a daily basis while also enabling "one-click" distribution of your app to your testers.

Since savvy apps focuses on mobile development, there are a different set of challenges than running CI with a conventional web or server application. One of these issues is simply maturity. A language like Java that’s been around for almost 20 years has a great number of toolsets for building, testing, and deploying code. The tools for iOS, Android, and Windows apps are still growing. Although testing frameworks today for these are platforms are improving every day, they are still a little rough on the edges, especially for UI testing. Another challenge is that for the most part, mobile apps have to be developed in a simulated or emulated environment rather than the environment where it will eventually exist for production. For example, consider a simulator for apps versus running a web app in the same browser it was tested in during development. A third challenge for CI for mobile apps is managing security, especially for iOS and Windows apps; correctly signing your apps with the right certificate and entitlements can be difficult. The good news is once you’ve setup your CI automation, you can let it do that for you too!

Jenkins

One of the more popular tools that has emerged to to achieve CI is Jenkins. Jenkins is simply a Java application that monitors and executes "jobs" that automate a certain task. For example, a job could include pulling source code from your source control management (SCM), compiling the latest code, running automated tests, pushing to another branch, or deploying it to your development, beta, or production environment. You may be thinking, "but I don’t know how to script out any of that stuff." Here’s the best part: you don’t have to! There are already hundreds of plugins other developers have written and contributed to the Jenkins community. Installing each one is as easy as a one-click operation.

Although it's always good practice to fully test your code before pushing it into SCM, Jenkins is a great safety net. When all of your code is aggregated and tested thoroughly on every push, it reduces risk.If the Jenkins’ job that runs your automated UI tests fails, your team can be sent notifications via email. Instead of discovering the problem a few hours or days later—after other features have been built on and around that buggy code—immediately the whole team knows that there is an issue that needs to be addressed. Jenkins and CI could even save you the embarrassment of potentially deploying an unstable version of your code to your beta testers or worse, into app stores.

So now that you’ve decided you want to improve your team’s quality of code, reduce time spent tracking old bugs and reduce risk early on with merging and running tests on your code base as early and often as possible with Jenkins, you’re probably wondering "Where do I start?" Let’s discuss, shall we?

Jenkins Setup Tutorial

While there are some hosted CI solutions, we haven't been impressed with what's available for mobile. So our setup focuses on rolling your own solution using Jenkins. Thus to start, you’ll need a CI machine. You could just use any old Mac you have lying around but with all the jobs and test suites you’ll be running on it, it pays to have a machine that can handle the load. We've found that a supped up Mac mini works nicely. You can host this on your network or use a colocation service like Macminicolo.net.

Getting Started

When setting up Jenkins, you need to make sure you create a unique user with admin rights to run the launch daemon. We named ours ’jenkins’ and did all of the following under that user:

  • Install Xcode and its command-line tools.
  • Generate an email account for your Jenkins build bot, which will be used to notify users of broken builds.
  • Get Jenkins access to your Apple Developer Program. Once that's done, you can log in via Xcode to download the applicable provision profiles that you’ll need. You can also manually import the needed provision profiles with SCP or a file manger like Dropbox.
  • Copy your development and distribution certificates that you use for distributing your iOS apps to the CI machine. Highlight your distribution key and certificate in the OSX Keychain Access app and select "Export 2 items..." to export them as a .p12 file. I've found that you need to import this to your CI machine’s System Keychain. You need to make sure Jenkins will have access to those certificates so after you import them, right click each key > "Get Info" > "Access Control" and either allow all applications use of the key or add Xcode to the list of allowed applications.
  • We also need to give the Jenkins user access to the source code.This step depends on what you use for SCM but will likely require you to generate an SSH key. Instructions can be found in Steps 1 & 2 here. If you use GitHub to host private repositories like we do, then you’ll need to create an account for your Jenkins build bot so your jobs can push and pull code.

Install Jenkins

Next, you need Jenkins! If giving commands in the Terminal isn’t your style, check out the OS X installer but the preferred way is using Homebrew. Either way, we’re setting up Jenkins as a launch daemon which will start up Jenkins when your machine boots.

If you don’t have Homebrew, it's a great package manager for OSX and super simple to use. You can install Homebrew with one command and Jenkins with another:

jenkins @ dev > ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

jenkins @ dev > brew install jenkins

Notice after installing Jenkins, Homebrew gives you some helpful tips:

==> Downloading http://mirrors.jenkins-ci.org/war/1.570/jenkins.war
######################################################################## 100.0%
==> Caveats

Note: When using launchctl the port will be 8080.
To have launchd start jenkins at login:
    ln -sfv /usr/local/opt/jenkins/*.plist ~/Library/LaunchAgents
Then to load jenkins now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist
Or, if you don’t want/need launchctl, you can just run:
    java -jar /usr/local/opt/jenkins/libexec/jenkins.war
==> Summary
   /usr/local/Cellar/jenkins/1.570: 3 files, 65M, built in 9 seconds

If you want to run Jenkins on start up, you need to link the .plist files Homebrew intalled for Jenkins to your Mac’s Launch Agents with

ln -sfv /usr/local/opt/jenkins/*.plist ~/Library/LaunchAgents

and then to start it up right now, run

launchctl load ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist

See the next section if these commands don’t work for you initially...

Java

When we upgraded our Mac mini to Mavericks, Java needed to be reinstalled.Apple was releasing their own version of Java 6 with older OS versions and with Java 8 out now, you may need an upgrade.If you see something like "No Java runtime present, requesting install," when setting up the launch daemons for Jenkins, just follow the dialog box that pops up to Oracle to install the latest Java SE Dev Kit.If your Mac already has Java installed, Jenkins will work but I would suggest you update too.

Launch Daemon

Homebrew sets us up nicely with a launch agent but what we want is something that will boot Jenkins regardless of the Jenkins user being logged in (one of the key differences between a launch daemon and a launch agent). To do so, type the following command in the Terminal:

jenkins @ dev > sudo touch /Library/LaunchDaemons/org.jenkins-ci.plist

All we need to do is copy the plist Homebrew gave us and add two (shown in bold):

<?xml version="1.0" encoding="UTF-8"?>

<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>Jenkins</string>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/bin/java</string>
      <string>-Dmail.smtp.starttls.enable=true</string>
      <string>-jar</string>
      <string>/usr/local/opt/jenkins/libexec/jenkins.war</string>
      <string>--httpListenAddress=127.0.0.1</string>
      <string>--httpPort=8080</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
<b>    <key>UserName</key></b>
<b>    <string>jenkins</string></b>
  </dict>
</plist>

If you used a different username, be sure to use it in your plist. We need to specify the username or else it will run as the system root user which can get you into trouble when you’re actually using Jenkins.

Now you can reboot or just run this command to see Jenkins running at http://localhost:8080.

sudo launchctl load -w /Library/LaunchDaemons/org.jenkins-ci.plist

Jenkins

Woo hoo! You’ve successfully got Jenkins running! Now it's time to put him to work.

Users

You’ll probably want to create users in Jenkins, so that the rest of your team can get access to your jobs whilenot leaving it open to just anyone. First open the "Manage Jenkins" page on the left-hand menu and go to "Configure Global Security".Enable security and unless you want to get fancy, we’re going to let Jenkins manage the usernames and passwords and allow any logged in user to do anything. For now, allow users to sign up but remember to turn this feature off later once your team members are added.

Jenkins - Security

Clicking "Save" on this page will prompt you to log in or create an account.Create an account and log in to see your name at the top right.

Plugins

Before we make our first job, let's browse the available plugins we can use for our iOS app. Go back to "Manage Jenkins" and then"Manage Plugins". Some of the plugins that will be useful include:Xcode Integration, Git Plugin, GitHub Plugin, Environment Script Plugin, Publish Over FTP and Mailer Plugin. Note that you may not need all or any of these, depending on your setup.These are just some of the ones we find helpful but there are literally hundreds of plugins to select. Remember, even if you can’t find a plugin to do exactlywhat you want, you can always script it yourself. After selectingthe plugins you want, make sure the checkbox next to it is checked and click "Download now and install after restart". Jenkins will reboot when it's ready with your new plugins available.

Credentials

The last step before creating your jobs is setting up Jenkins to have access to all the systems it needs to integrate with. Go to "Manage Jenkins" then "Configure System". Again, this area is heavily dependent on your team’s setup and I encourage you to browse Jenkins’ plugins to find what works best for you. Here, we just need to set up the Git plugin user.name and user.email but you may need to configure CVS, Subverison, Ant, Maven or whatever else you need your job to do.

Jobs

You’re finally ready to start your first job. Go ahead and click "New Item" from the main menu. We like to name our jobs prefixed with the appname, followed by the order number of the job, then finally the role of the job. For example, the jobs myApp_1_DevBuild, myApp_2_UnitTests and myApp_3_Distribute would run in order, first compiling "myApp", then running unit tests, and finally building the .ipa binary package and sending it to the staging server for testers to download. Give your jobs a good name to help distinguish it from others while also describing what the job is going to do.

After naming your job, click the "free-style software project" radio button and then "OK". Later, when you’re adding the rest of your apps and jobs, the "Copy existing Item" option comes in handy since you likely won’t have to create every job from scratch in the future. Upon creation of your job, Jenkins takes you to that job’s "Configure" page. Add a good and meaningful description for your job like "Compiles the source code for myApp, creates the .ipa and manifest .plist files and distributes them to the staging server." You may need to go back to review previous jobs’ logs so let’s check the box to "Discard Old Builds" and set the "Max # of builds to keep" to a safe number like 10.

Next, choose your SCM and configure the branch and repository info. Savvy Apps hosts our projects privately on GitHub, so we just need to pass in the credentials for the Jenkins build bot GitHub account.

If you want your job to only run when you tell it to, then you can skip the "Build Triggers" section. Otherwise, here is where you can set up jobs to run on a schedule, after pushes to your SCM, after another job successfully runs or just check every minute for updates to pull. Let’s set up our job to check our SCM every 15 minutes for changes. Select "Poll SCM" and use the schedule "H/15 "

Next we have the the "Build", this is where you can do any scripting to set up your job.Click "Add Build Step" and select "Execute Shell". Here you can script pretty much anything: you can copy needed provision profiles to the CI machine, setup xcode build parameters, tweak your app’s info.plist and more here.For now, we’re just going to create a simple manifest .plist file to use to distribute our app binary. The manifest .plist is the plist Xcode creates for you when you archive your code and select "Save for Enterprise Distribution". It's just a list of a few key-value pairs needed to distribute .ipa files OTA. Copy this into your Command section:

cd $WORKSPACE
mkdir -p manifest
cd manifest
rm -rf *.plist
touch myApp-${<b>BUILD_NUMBER</b>}.plist
manifestPlist="myApp-"${<b>BUILD_NUMBER</b>}".plist"
echo ’<?xml version="1.0" encoding="UTF-8"?>

<plist version="1.0">
<dict>
<key>items</key>
<array>
<dict>
<key>assets</key>
<array>
<dict>
<key>kind</key>
<string>software-package</string>
<key>url</key>
<string>__URL__</string>
</dict>
</array>
<key>metadata</key>
<dict>
<key>bundle-identifier</key>
<string>com.myApp.distribution</string>
<key>bundle-version</key>
<string>’${<b>BUILD_NUMBER</b>}’</string>
<key>kind</key>
<string>software</string>
<key>title</key>
<string>myApp ’${<b>BUILD_NUMBER</b>}’</string>
</dict>
</dict>
</array>
</dict>
</plist>’ > $manifestPlist

Make sure you read it over and change the app name and bundle identifier and url to match your app and staging server setup. All we’re doing here is creating a myApp-XX.plist to use as our manifest .plist file where the XX is filled in with the build number parameter of the job, ${BUILD_NUMBER}, this increments every time the job runs, pass or fail.

Next, we’ll dive into the Xcode plugin. Add another build step for Xcode. This Xcode plugin isn’t necessary, you could write all the xcodebuild commands yourself in your own custom scripts but this plugin certainly does do a lot for you. Every project is going to be different and so you may need to vary your configuration here, as well. You need to either give Jenkins your project target name or the workspace, in addition to the configuration you want to use (debug, release, or your custom ones), the schema name, the build output directory you want and make sure you check the "Pack application and build .ipa" and give the .ipa a name.If you’re like us and using custom configurations and schemas and a CocoaPods workspace, then your setup may look like the following:

Jenkins - Build Settings

Notice we gave a configuration, .ipa filename, output directory name for the .ipa, the scheme, the workspace name, and the build output directory.

The next sections are going to be "Post-build Steps", run after building the binary .ipa of your app. You can add post-build scripts to keep track of the version number and update your info.plist, you can make other changes to any other configuration files you may have and push them back to your master branch. You can copy your successful artifacts to Dropbox, post them to OTA servers with the TestFlight plugin orFTP plugin. Or you can write a simple cURLor SCPscript to send them where you want them. The sky’s the limit with what you want to do with your files now. If you don’t want to send them anywhere just yet, Jenkins will just hold onto them for you as long as you tell it to archive the artifacts you want to keep.

To store our important files you’ll need to add a post-build action for "Archive the artifacts". All you need to do here is tell Jenkins which files you want to save after the job. Right now, we’ve set up Jenkins to store the manifest.plist in the ${WORKSPACE}/manifest directory and the .ipa file will be put in ${WORKSPACE}/build/artifacts. Since, the ${WORKSPACE} is implied for this step, we can just say:

build/artifacts/*.ipa, manifest/*.plist

Finally! You’re ready to "Save" your configuration and build your job! Click "Save" then "Build Now" on the left menu. You’ll see Jenkins churning away at the build without any more work from you.

Jenkins - Build Now

If your build succeeds, you’ll see a blue dot in the "Build History" and if you refresh the page, you’ll also see your artifacts neatly posted and readily available for download.

Jennkins - Build History

If your build did not succeed, you’ll see a red dot and you’ll have to do some digging. The most common issues include provision profiles missing, certificate/key permission errors and library linking errors. You’ll have to click the failed job instance and take a look at the "Console Output" for the failing error.

There you have it, you got your hands dirty and took the plunge into the scary world of CI only to come out feeling empowered and anxious to start building all your projects this way! Granted, it does take some time to setup but once it is, it scales beautifully. Copying existing jobs is a breeze and adding new plugins and features couldn’t be easier. With one click (or less if you rely on timed builds!) you can pull the latest source code, compile it checking for errors, and distribute your app to your beta users and testers.

Matthew Tea is an iOS developer with a passion for quality, tested code. He's a team player with a strong desire to learn new and upcoming technologies.

You made it this far so...