Comparing Mobile Automation Tools: Appium, Selendroid and Calabash

Whether you’re just starting your migration from desktop or web, or already developing on a mobile platform, finding the right tools for your mobile application testing environment is paramount to delivering effective, high-quality mobile apps.

Because of the upsurgence of mobile device use, it’s essential to build and deploy useful and intuitive apps. This is where effective mobile app testing services come into the picture. It’s not a question of whether to go with manual testing OR test automation services, but rather understanding how to use a combination of mobile app testing methods for the best results.

Any professional software testing company will most likely employ a mixed approach depending on different scenarios. While automation is the latest industry buzzword, which certainly comes with a ton of advantages, it isn’t always the most suitable approach for all testing. For example, mobile automation is ideal for regression testing whereas manual testing is more appropriate for exploratory testing.

Once you’ve determined when to deploy automation versus manual mobile app testing, you can buckle down in finding the right tools. Let’s review three of the most popular mobile QA testing utilities to help you with your decision-making to find the best fit for your environment and available skillsets.

While there are several tools and frameworks available for mobile automation testing, we’ll look at the three most popular open source programs being used today: Appium, Selendroid, and Calabash.

Appium Overview

Mobile automation using Appium can be used for native, hybrid and mobile apps compatible with Android, iOS, and Windows. It supports Python, Node.js, PHP, Objective-C, Ruby, and C#, and uses the popular WebDriver API to receive a connection and command request from the client, which is then executed on the mobile device.

Architecture

The Appium setup comprises a client-server architecture. The server is coded in Node.js and facilitates communication between the mobile device and the client. The client represents the libraries such as Python, PHP, Ruby, Java, C# and JavaScript that provide the commands specific to mobile devices.

When the server starts, the REST API is exposed starting a session between the server and client using a session ID. Once established, the client automatically sends commands to the server, which then executes the commands on the mobile device indirectly using automation frameworks. The Appium server uses different frameworks for different platforms:

• iOS – UIAutomation from Apple
• Android 2.3+ – UIAutomator from Google
• Android 4.2+ – Instrumentation from Google
• Windows – WinAppDriver from Microsoft

Installation

Some of the prerequisites for Appium include:

• Java Development Kit or JDK
• Android SDK Studio to manage the Appium Android Studio
• Eclipse and TestNG for Eclipse
• Appium Client Library
• Selenium Server JAR
• APK app info on Google Play
• Appium Desktop

For your Appium installation on Windows or to install Appium on Mac, check out the Appium technical guide for each specific platform.

Selendroid Overview

Selendroid is a great tool for mobile automation where you’re focusing on developing Android apps and its backward compatibility supports Android API 10 (Android 2.3.3) through API 19 (Android 4.4). If you’ve heard of Selenium, then Selendroid should ring a few bells as it’s basically Selenium for testing mobile apps on Android that are published on Google Play. It can be used for both native and hybrid applications and allows for simultaneous automated testing of one app on several devices (just like Selenium’s cross-browser testing ability).

Architecture

The Selendroid-Android framework is made up of four major components: (1) Selendroid-Client is the java client library, which is based on the Selenium Java client, (2) Selendroid-Server, which runs parallel to the app on the Android mobile device, (3) AndroidDriver-App is the built-in Android driver WebView to test the mobile app and (4) the Selendroid-Standalone is what manages multiple Android devices.

Installation

You can install Selendroid on Mac, Linux and Windows and it will require the following prerequisites based on your environment:

• Java SDK (minimum 1.6) must be installed and JAVA_HOME configured.
• The latest Android SDK.
• A minimum of one Android virtual or physical device connected to the computer with the Selendroid-Standalone installed.

These Selendroid setup instructions provide a comprehensive guide to installing and configuring your environment.

Calabash Overview

Calabash is a mobile testing framework in which to write and execute tests for both Android and iOS mobile devices. One of the best advantages with Calabash is that the tests work seamlessly on Android and iOS if the mobile app is identical. The tests are conducted using Cucumber, which makes it an easy to use tool that enables user interface interactions with a mobile app on real physical devices. It’s a Behavior-Driven Development (BDD) test automation framework meaning that testers define application behaviors.

Architecture

Calabash has a comprehensive API that allows for easy integration with development, testing framework tools or test management software. Calabash works with Cucumber to write and execute automated tests using Gherkin syntax that offers a plain readable format without the need for programming knowledge.

Installation

Before setting up Calabash for testing, you’ll need to have the following installed:

• Ruby (preferably the latest version).
• Bundler to manage dependencies.
• For Android testing, install the latest Android SDK.
• For iOS testing, it’s recommended to have Mac OS 10.10 with Xcode 6.x.
• Lastly, you will need a physical mobile device or emulator connected.

Tools Comparison Chart

For a quick summary on the differences between the mobile app testing tools, we put together the following table: