Continuous integration (CI) is a development approach that favors testing and reporting of changes to a given build as they happen, rather than at set artificial intervals, or at the end of a given process. This means that if a given change to the code base is implemented and is faulty, the flaw will be picked up immediately, and reported in near real-time, allowing the team to react accordingly. The goal of Continuous Integration for Mobile Apps is to establish a rapid and consistent feedback system that will allow developers to fix or adapt to constantly changing application or software.
“CI is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.”
– Martin Fowler 2006
While originally used for standard software development, continuous integration for mobile apps has long been adopted as well.
As Martin’s quote above illustrates, CI is strongly associated with automated build practices, including automated testing. The goal is to build the software or application whenever a change is applied, in order to detect and confirm any integration errors as they happen. By building, we mean a process of compiling, inspecting, testing and deploying integrated segments of source code to prove that it all works together as it should. This, of course, requires specialized tools and platforms, some of which we will discuss in today’s article, in particular those that focus on mobile application development.
Considerations for CI for Mobile Apps
Platform
One of the primary considerations is, of course, what platform you are developing for. For example, Shippable supports Android, but not iOS (according to comparison chart at bottom of their page). If developing for both iOS and Android, Travis CI and CircleCI might be better choices, based on this criteria alone. Of course, the list of considerations is a lot more complicated than that, and the right product for one company is not going to be the right one for another.
If supporting more than one platform, this consideration may be adapted to which tool best supports both platforms. Provided a CI toolset supports both, gear your decision towards the side that provides your primary project workload. Supporting more than one CI tool for different development teams can prove both costly and problematic, especially if you would like to implement cross-platform testing.
If you plan on adding a development platform in the near future (for example, if Windows gains a more considerable app market share), you will want to be sure to factor this into your decision. If the platform does not yet support the expansion platform, it should be a prominent part of their product roadmap.
Hosted vs On-premise (or Self-Managed)
Another consideration to keep in mind is whether the CI server used is on-premise or a cloud service. Each provides its own strengths and potential weaknesses to the discussion. This decision rests strongly on the degree of control you want over your CI solution.
On-site/Self-managed Solutions for Continuous Integration for Mobile Apps
This is not a matter of security – hosted CI solutions can be very secure if implemented correctly – it has to do with the configurability and extensibility inherent in an on-premise (self-managed) offering. One example is branching. With externally hosted solutions, you rarely get automatic branch management. Typically there has to be a build plan for each branch, created manually. There is often no support for branch management strategies, such as downstream or upstream merging. Manual intervention is typically required for most changes to an external CI solution. CI solutions such as Bamboo, TeamCity and Jenkins have support for automatic branching of build plans, either out of the box or via plugins. With on-premise solutions, there are hundreds of hooks, triggers, integrations (e.g. JIRA + Bamboo) which can be configured to automate many tasks that would be manual on a hosted solution.
Another potential issue is migration. By selecting a hosted CI, you are often tying yourself to both their good and bad points, and you have limited options should the hosted solution fail or get bought out. If you think this does not happen, consider the fate of ship.io, one of the oft-recommended solutions for mobile CI. With on-premise servers, most offer some form of migration services, as each is more than happy to poach a customer from a competitor. An example of this is Bamboo, Atlassian’s CI server solution, which offers an option to migrate your Jenkins build.
Custom in-house tools can also be a reason to stick with an on-premise solution – the more control you have over your CI server, the easier it is to create and manage in-house tools and integrate them into your build plan.
Note that solutions such as Jenkins, Bamboo, TeamCity, do not have to be “on-premise” to give these benefits. They can be hosted in the cloud and managed by your organization in order to provide the best of both worlds. An example of such a hybrid offering is Cloudbees Jenkins in the cloud solutions. Further proof of this is that a strong iOS mobile focused hosted solution, Hosted CI, boasts that it is based on a “mature Jenkins Server.”
An on-premise solution is typically more scalable than a hosted solution, with the number of builds that can be run determined by the resources you provide the server, and the human resources you have available for configuration and customization.
Hosted Solutions for Continuous Integration for Mobile Apps
One of the key selling points of hosted CI solutions is the ease with which you can get started. Building a mature and tailored Jenkins CI server that fits your organizations needs to the degree of customization described above can take a great deal of time and effort. Much like any in-house system, this growth is typically organic, and the true benefits of the platform may not be seen immediately. By contrast, with hosted solutions, you typically see immediate gains in productivity and quality control, as everything is already pre-configured or requires minimal configuration. However, further gains are likely to be determined by the addition of new features or fixes from the CI platform development team, not internal innovation.
Hosted continuous integration solutions typically are very user friendly and have invested a great deal more effort into UI considerations. By contrast, Jenkins, the leading open source on-premise server, is widely regarded to have a poor or dated UI. Indeed, as Gus Reiber put it in a blog post about upcoming UI changes:
“I will even go so far as to say, I believe it is a platform liability at the moment, and even among we the Jenkins faithful, few of us look forward to using it.” – Gus Reiber
The good news is that this shows that an on-site server is aware of the issue, but in terms of focus on user experience, hosted platforms have by and large maintained this as a priority and are not playing catch-up.
Another selling point is that in their efforts to make CI simple, hosted solutions have begun to tailor their solutions towards mobile development. This means that out-of-the-box, these solutions are often a better fit with your existing development processes, which is important particularly for startups with little time to invest in tailoring their own solution. Some, like Hosted-CI, have focused entirely on one platform. By doing so, they can offer iOS developers features that other platforms don’t, such as automated deployment to Testflight, or the capability to manage dependencies through CocoaPods. Some platforms that instead invest equally in Android and iOS include Codeship, and CircleCI. Travis CI also supports Android, but it appears to still be in the beta stage. However, Travis CI’s strong and dedicated open source community might help it outpace its competitors in features for mobile development. It is worth noting, however, that most of the hosted solutions have only just begun to commit to mobile development and tend to have a richer feature set for web apps (as stated by Paul Biggar, founder of CircleCI, on Quora.)
Another selling point is that these tools offer seamless pushes to production platforms likeHeroku, Amazon EC2 and Google App Engine. While this is equally possible on self-managed servers such as Jenkins, Bamboo, TeamCity, etc, this functionality is typically not out of the box, but covered by configuration and plugins. This can in itself be part of your selection criteria…when considering which hosted platform to use, the production platforms supported might be a key consideration. To compare hosted platforms side by side, try stackshare.io or slant.co.
Automation (and Automated Testing) Key to Continuous Integration Success
Automation is key to Continuous Integration success. Every aspect of continuous integration can be automated, from building the dev and test environments, configuring each environment, maintaining the correct configuration for each environment, remediating configuration drift and scaling up as needed. It is no coincidence that the first of these tasks is building the dev and test environment…automated testing is at the heart of any CI pipeline. Without proper automated tests that are quick to run, have good coverage, and produce accurate results, the feedback needed to keep your build on track will not be received, and the changes needed to keep your processes running smoothly will not be initiated.
Automating your testing, and integrating it with your CI pipeline is one of the most important things you can do to ensure success. For continuous integration to be successful, each individual test must be tested from every angle, in order to ensure that it doesn’t break something else and that it works as expected. What good is a fix or feature that causes another problem? So what should you look for in your automated testing solution, in order to fit it into your CI pipeline?
Cross-platform capabilities
You need to be able to test not only the basic code, but the user experience as well. Consistency is key, even when building on different platforms. If you are building on separate platforms (ie, Android and iOS), a solution that allows you to create test cases that work across both platforms will save time and money by preventing duplication of effort. For example, at testmunk, we leverage Calabash and the Page Object Framework to create test cases that reuse test logic where it is uniform, but that allow for differentiation if the steps cannot be identical.
Visual Confirmation for Design and UI issues
Another key feature that can prove invaluable to a CI effort is visual confirmation that test steps are performing as expected. It doesn’t matter if the step is technically successful if the button is too large or small for the screen, or if it is inaccessible when the keyboard is up. These are just some of the issues we’ve seen resolved quickly because of our automated testing.
Intuitive and Human Readable Test Cases
If trying to get buy-in for your testing solution, it is important that not only the developer understands what you are testing and what the results are, but that others in the organization – including design, marketing, company leaders, and other stakeholders – can do so as well. Test cases that are easily legible also help in ensuring that these stakeholders can confirm that the test case as written matches what was envisioned for the product (ie that the user story matches the test logic). If the test logic is inscrutable code, this transparency is not likely to happen. This comes back to communication. The output of your automated testing should be a deliverable that helps to move the application forward and allows for quick and easy interpretation. A good CI pipeline depends on this type of continuous and accessible feedback.
Ease of Integration
Users of on-premise solutions such as Jenkins rarely have to worry that a solution is incompatible, as they are used to uttering the phrase “there’s a plug-in for that.” However, hosted solutions often prefer you to use their own too, and are not necessarily as forgiving in the integration department. However, most can work with the REST API, even if it might take some configuration on their (or your) end. For our part, testmunk integrates well with both Jenkins and Travis, and for other solutions, the REST API should allow for a successful implementation. For more information about integrating Jenkins or Travis with testmunk, check our documentation.
Summary
When considering your Continuous Integration solution, there are numerous factors to consider, all of which come back to the goals and primary workload of your organization. Your CI platform needs to meet the needs of your organization, whether this means a greater degree of security, customization and control (on-premise or self-managed solutions), or quick and easy set up, immediate gains, and less reliance on internal expertise (hosted solutions). No matter the solution, however, automation and feedback are the keys to success. Your solution must produce actionable feedback that is accurate and timely, allowing your continuous integration implementation to achieve its intended goal: continuous improvement.
