Using TestFlight to Distribute Multiple Versions of an iOS App

Development iOS Process QA

Development doesn't stop once a v1.0 app is live. Your newly launched app will need to have regular app updates to thrive. Whether it’s a new feature or a bug fix, it will be important to continue to develop and test these updates prior to shipping them to app stores. This can be challenging, however, as you need to develop these updates and distribute them to testers without interrupting the experience of users who are interacting with the live app.

While we previously used a number of different tools to distribute iOS apps, since October 2015 the Savvy Apps team started heavily relying on TestFlight for distributing external, and even internal, builds. It's an approach that came with some hurdles at first because of how TestFlight is tied to a review process. With some refinement to our approach though, we can now easily distribute different builds of the same app to internal testers, beta testers, and end users against different databases. We've also connected our distribution process to our continuous integration setup. Here we explain our setup and how you could integrate something similar into your own processes.

Creating Three Separate iOS App Installs for One App

As proponents of on-device testing, data integrity, and shipping bug-free apps, we create and maintain three separate app installs for any one app. This approach allows us to continue to test and iterate on the app without interrupting App Store users. The three versions purposefully do not influence or interact with each other. Even though there’s some more upfront work, our efforts allow us to have an app that's available in the App Store, push out an update to beta testers, and work on an future build all at the same time.

The three different versions of the app we have at any one time are as follows:

  • App Store: The publicly available version of the app that end users encounter in the App Store itself.
  • Beta: A private version of the app that contains new features or bug fixes not yet available publicly.
  • Interim: Also a private version of the app where our team begins to create the features and bug fixes beta testers will eventually get access to in the beta version.

Beta versions are first installed through TestFlight and are what will next be submitted to the App Store to replace the current live app. Interim builds are used mostly by developers, QA, product, and other team members to access and analyze in-progress work. Because these kinds of builds are largely for in-progress work only, we mostly rely on Beta by Crashlytics to distribute them. Keep in mind that ahead of submitting a v1.0 app to an App Store, we start with the interim build, work our way to a beta version, and finally prepare the App Store release. That is, this process is in place from day one; we just outlined it this way for the sake of readability.

Should the app have a backend—and most do today—each app install will point to one of our various database environments: development, staging, or production. The development database is mostly used for interim builds while staging is for beta builds. Production is solely for the live app. This sort of setup is important for data integrity and to eliminate interruption for live App Store users. For example, a beta version may have a new feature that changes the app's data model and could cause the app to crash for live users.

How TestFlight Simplifies the iOS App Distribution Process

Prior to October 2015, we used HockeyKit to distribute non-App Store builds. HockeyKit and comparable tools require the manual collection and input of every user's UDID to allow testers to install the app. Since most testers didn't know where to find their UDIDs, we had to spend a significant amount of time educating them on what a UDID was and how to locate it. We'd then have to manually input the UDIDs into Apple’s Developer Portal.

After we pushed up a new build to the server, we would then have to email our customers to notify them that the app was ready. We would outline what exactly we wanted them to test. Our customers would subsequently send out a notice to their testers themselves. Of course, we also had to experience the fun that is redoing a build if another UDID needed to be added.

TestFlight has since simplified a number of challenges of HockeyKit and comparable solutions, the biggest of which is related to UDIDs. To onboard testers all you need to do is enter an email address. TestFlight then sends a notification for the user to join the designated app. TestFlight also notifies all testers of an app when a new build is ready and what they need to test within the TestFlight app itself. In the “What to Test” section of TestFlight we include information on how to provide feedback as well as a bulleted list of items that are new, updated, fixed, or in-progress for the next build.

Our main concern in moving to TestFlight centered around its review process and how that could slow down testing. Thankfully, we’ve been able to overcome that in two main ways. The first is with TestFlight’s “Internal Testers,” which are users who are part of the iTunes Connect team. Once an app is submitted for internal testing, it goes out to these testers as soon as they are ready and does not need to be reviewed by Apple.

When you’re ready to test your app with a wider group of testers as with a beta build, you’ll need to move to “External Testing” in TestFlight. Unlike with internal testing, external testing builds must be reviewed by Apple the first time they are submitted. Note that this process is not the same review process Apple has for the App Store from what we’ve seen. It usually takes about two to three business days to review the build before you can release it to external testers, but we've seen it done in less than 12 hours.

With the review time in mind for these kinds of builds, the second approach to addressing the review time “problem” really is about understanding Apple’s requirements for external builds. For example, we always account for the two- to three-day review time for distributing beta builds. Apple also does not require re-reviewing an external beta build if significant changes were not made since the last submission. Knowing that, we try to get a version of the app submitted for review that substantially represents the key features of the app. We then can indicate during submission that there are no new significant changes from the previous build and beta builds go out to external testers immediately.

Apple Developer Portal, iTunes Connect Setup, and Xcode Schemes

For our distribution process to work, we create three App IDs, three iTunes Connect apps, four provisioning profiles (more on “four” shortly), and the corresponding certificates. In the Developer Portal, we create three App IDs corresponding to the App Store, beta, and interim app installs described above. This means that two of them will never be published on the actual App Store and won't be otherwise available publicly.

For the provisioning profiles, three are iOS distribution profiles and need to match our three app versions. There’s a fourth profile we call “debug” created as an iOS development profile so we can develop locally. This particular provisioning profile uses the interim build bundle ID. Because we have both distribution and development profiles, we need certificates to match each type of profile. We also add certificates for push notifications or other entitlements as needed.

The other consideration relates to Xcode schemes. We create various Xcode schemes to make it easier to create any one of these builds either manually or via continuous integration (see the next section). We use four Xcode schemes in total to correspond to the four provisioning profiles. Since each scheme has its own provisioning profile and certificate, they need to match the respective bundle ID.

Our approach to bundle IDs is to use com.companyname.appname along with a suffix that corresponds to the app install version. For example, we use “beta” at the end for beta builds such as com.companyname.appnamebeta. The live App Store build does not include a suffix.

To help make this setup clearer, here is what it looks like in its entirety for one of our customers, Leap, which is a new dating app currently in soft launch. In this example we’ve called the interim version “Ad Hoc” and use “adhoc” as the suffix for the bundle ID. Note that we have production push certificates for each app version so that we can send push notifications to each corresponding version of that app.

App IDs, Bundle IDs, & Provisioning Profiles

  • Leap Debug uses com.leap.leapappadhoc
  • Leap Ad Hoc uses com.leap.leapappadhoc
  • Leap Beta uses com.leap.leapappbeta
  • Leap App Store uses com.leap.leapapp

Certificates

  • iOS Development
    • Various Savvy Apps team member names
  • iOS Distribution
    • S & S Leap, Inc.
  • APNs Development iOS
    • com.leap.leapappadhoc
  • APNs Production iOS / Apple Push Services
    • com.leap.leapappadhoc
    • com.leap.leapappbeta
    • com.leap.leapapp

iTunes Connect

Using Continuous Integration with iOS Build Distribution

Our continuous integration (CI) setup has evolved to accommodate our build distribution strategy. It helps us make our build process for each app install fairly automated. With our approach, anyone on our team can easily install the app without even firing up Xcode. For example, our designers can grab the latest version of the app we're working on and do a design or UX audit.

When code is pushed against specific branches, CI triggers the creation of one of our three app installs whether it’s an interim (“Ad Hoc” above), beta, or App Store build. We start by ensuring that each and every commit can successfully create a build. We use Fastlane and Beta by Crashlytics for this process. After each commit, a build is created and available for install. We also have a Slack integration notice if the build fails.

As we review the various interim builds and the tickets are accepted, we update the app version number in Xcode and eventually push the relevant work out to a specific branch such as beta. In this case, CI then creates the beta build for us. We grab the build, which gets hosted in a GitHub release, and post it up to TestFlight for external testing. We continue this process until the build is ready for the App Store and then eventually merge all the work back into the master branch.

Concluding Note

The process of creating multiple app installs of the same app and using TestFlight in particular to distribute them has been very helpful from a product, design, QA, and beta testing perspective. Not having to educate end users about UDIDs, train non-technical team members on pulling code or Xcode nuances, or wasting any time waiting for builds to be created makes our jobs much more enjoyable. Additionally, it allows us to do on-device testing and continue to work on bug fixes and features for an app without affecting App Store users. While it took a little longer to piece it together, this process is now simple to manage and well worth the effort.

Richard is the Managing Director of Savvy Apps. His role is to keep both customers—and just as importantly—the Savvy Apps team thrilled and excited about their work.

You made it this far so...