Mobile Tophatting at Shopify

At Shopify, the leading cloud-based, multi-channel commerce platform for 600,000+ merchants in over 175 countries, it’s crucial to test and verify the functionality of the new features that get introduced in the platform. Since the company doesn’t have a QA team by design, testing features is the developer's responsibility. To do so, we set up a project to contain automated test steps which execute via our continuous integration infrastructure (CI) and additional manual checks are performed by developers.

One of those manual checks is trying out the changes before merging them into the codebase. We call this process “tophatting” after the 🎩 emoji. Back when Github didn’t have support for code review requests, Shopify relied on emojis to easily communicate the state of the code review process. 🎩 indicates that the reviewer not only looked at the code but also ran it locally to make sure everything works as expected, especially when the changes affected the user interface.

The tophat process requires the developer to save their current work, checkout a different git branch, set up their local environment for that branch and build the app. For mobile developers, this process is tedious because changing the git branch often invalidates the cache, increasing the build time in Xcode and Android Studio. Depending on the project, it can take up to 15 minutes to build the app, during which developers can’t do any other work in the same project.

To eliminate their pain points and facilitate best practices, we’ve created a fast and frictionless tophatting process which integrates seamlessly with our CI infrastructure and dev, Shopify's all-purpose development tool that all mobile developers have running in their environments. In this post, I’ll describe how we built our frictionless tophatting process and show you an example of what it looks like.

Setting up Projects for Tophat

The slowest part of the mobile tophatting process is compilation. To speed this up for mobile developers we skipped the compilation step. We already build the apps on CI, so the application binaries are available in the disposable environments we created for running the PR builds. We updated the projects pipeline to export the binaries so that we can list and access them through the CI API. Depending on the platform (iOS or Android) the exported app has a different format:

• iOS: Apps are folders and we zip the folder using a naming convention that includes the name of the app and its version. For example, an exported Shopify app version 3.2.1 would be named Shopify-3.2.1.app.zip. 
• Android: APK files are zip archives, so we export them with its existing name. 

Once the apps are exported we leverage GitHub commit statuses to let developers know that their PRs have tophattable builds:

Command line interface

Dev is an internal tool that provides a set of standard commands across all the projects at the company (you can read more about it on devproductivity.io). One of the commands that backend developers use is tophat and we extended its use to support mobile projects.

The command looks like:

Where platform can be either ios or android and the resource can be any of the following:

• Pull request URL: For tophatting other developer’s work
• Repository URL: For tophatting the main branch of a repository
• Repository branch URL: For tophatting a specific branch
• Build URL: For tophatting a specific build from CI

For example, if a developer would like to tophat the pull request 35 of the project android, they could run the command:

dev android tophat https://github.com/shopify/android/pulls/35

Under the Hood

When the tophat command is run, the following steps are executed:

1. The user is authenticated on the Buildkite and GitHub API if they aren’t already authenticated. The access token is persisted in the macOS keychain to be reused in future API calls.
2. If the given resource is a GitHub URL, we use commit statuses to get the URL of the build.
3. Since the list of artifacts might contain resources that can’t go through tophatting, we filter them out and only show the valid ones. If there’s more than one in the repository, the developer can select which app they’d like to tophat.
4. After selecting the app:
1. For iOS projects, we list the system simulators and boot into the one the user selects. Most times, developers tend to use the same simulator so the command remembers and suggests it as default
2. For Android projects, we list the emulators available in the environment and a few more default ones in case the developer doesn’t have any emulators configured locally yet.
5. Once the simulator is booted, we install the app and launch it. 

The example below shows the process of tophatting Shopify Mobile for iOS:

Future Improvements

We’re thrilled with the response received from our mobile developers; they love the feature. Since we launched, Shopifolks enthusiastically submitted bug reports and proposals with many ideas about how we can keep improving the tophatting process. Some of the improvements we’re currently incorporating are:

• Caching: Every time we start the tophat process, we pull artifacts from Buildkite, even if we already tophatted the build. Adding a local cache will prevent downloading the artifact again and copy it from the cache instead.
• Real devices: Developers usually try the apps on real devices and we’d like to facilitate this. For iOS, the builds need to be signed with a valid certificate that allows installing the app on the testing devices.
• Tophat from commit and branch: Rather than passing the whole GitHub URL we simplify the input by letting developers specify the repository and the branch/commit they’d like to tophat.

Testing someone else’s work is now easier than ever. Our developers don’t need to know how to set up the environment or compile the apps they are tophatting. They can run a single command and the tool does the rest. The Mobile Tooling team is committed to gathering feedback and working with our mobile developers to add improvements and bug fixes that facilitate their workflows.