Testing Techniques for Mobile Applications

 Testing Techniques for Mobile Applications

There are 3 types of mobile applications


1 Browser Based application

  • Applications built for mobile browsers
  • Can be accessed by entering the specific URL in mobile browser. 
  • No Installation/Un-installations involved
  • No Upgrade involved
  • Browser Based Application users always expect connectivity
    • Network speed and coverage become an important test case 
  • Speed is a critical aspect: 
    • User entered values can’t be stored locally in device database, until user allows storing information in Cache, he has to enter all his information manually every time.

2 Pre-installed application

  • Applications which are shipped as in built software with the mobile device
    • If devices are not launched yet, a prototype of that device model is given by the device manufacturer. 
  • No Downloads involved.
  • No Installation/Un-installations involved
  • Automatic Upgrades can be done.
    • This is specific to application.
  • Testing of these applications in prototypes is very critical.
  • They deal with the very core data and functions of the device.
    • If anything goes wrong, the core database might get corrupt.
  • They cannot be uninstalled /deleted ever.
  • Crashes can cause severe damage to ROM 
  • Only way to remove these applications would be to flush ROM and then upgrade it.

3 Installable applications

  • Can be upgraded:
    • This is specific to the application design 
    • This also depends on the source from where application was downloaded 
  • Applications whose executable files/packages can be downloaded / received by wireless media or wired media. 
    • Platform Specific
  • Can be installed/uninstalled in the device.
  • They can be downloaded from:
  • App Store 
    • OTA (Over The Air) 
    • A URL which leads to download of the application.
      • Transfer via USB cable
    • Can be transferred from wireless media like Blue Tooth, Infra Red etc.

Mobile testing generics

Testing in various network types:

  • 2G:
    • GPRS
    • CDMA
    • EDGE
  • 3G
  • Wi-Fi
  • Different types of Plans based on service provider

Monitoring Battery consumption patterns

  • Observe battery consumption rate as the application is being
    • Run in Background 
    • Run in Foreground 
  • Run the application for long time.

Check memory usage patterns when in device apart from the pre installed applications

  • No other applications are installed
    • Lot of free memory available in device.
  • Some third party applications are installed
    • Lesser free memory available in device
  • Lot of third party applications are installed
    • Very less free memory available in device

Memory Consumption Pattern (RAM usage)

  • Check memory consumption pattern when multiple applications are running while the application is being:
    • Launched
    • Run
      • In Background
      • In Foreground
    • Exit

Interruptions:

  • Activities which can occur parallel within the device while application is being:
    • Installed
    • Launched
    • Run
    • Exit
    • Upgraded
    • Uninstalled/ Deleted

Examples of Interruptions:

  • Incoming call
  • Receiving incoming call
  • Receiving message
  • Device shutdown
  • Remove battery
  • Camera activated
  • Lose network connectivity and then regain it

Debug Build:

  • A very important asset in helping reproduce “difficult to reproduce” bugs
  • A build released in debug mode which has logs enabled.
  • Enabling logs based on sequence of keys
  • Logs can be read continuously, and can help retrace steps to any issue, provided application is running.

Build Signing:

  • Required for a build which has to be pushed to an app store.
  • Confirms that norms as per the authority of OS or device manufacturer are followed.

Testing round after build signing needs to be done

  • Ensures the final build being submitted is of appropriate quality standard.

Platforms covered

  • iPhone
  • Android
  • BlackBerry
  • Symbian
  •  Windows Mobile
  • J2ME Applications

Input models in different platforms

  • Touch Screen
    • Testing with single touch inputs
    • Testing with multiple touch inputs
    • Testing for Virtual Keypad
  • Track Ball/ TrackWheel
  • Keypad
  • Test cases around multiple inputs at the same time

Device modes in different platforms

iPhone

  • Additional Testing aspects:
  • UI Guidelines from Apple need to be adhered to.
  • Backward OS compatibility
  • Allows running only one application at a time.
  • Can debug via USB cable connection
  • Device Reboot not much needed until for clearing cached data

iPhone testing tool

  • Screenshots:
    • Can be taken by clicking the Home key and the Lock key together.
  • No other explicit tool needed
  • Memory Sweep
    • A tool with which a memory snapshot of iPhone device can be obtained at any given time.

Android

Additional Testing Aspects:

  • Allows running multiple applications in background
  • Application gets normally minimized on exiting.
    • So a reboot needed to launch application again.
    • Reboot needed often

Android - Debugging

  • Select "Enable USB Debugging" in device settings.
  • Then connect device to system via USB cable.

Android -Testing Tools

  • Dalvik Debug Monitor Server (DDMS)
    • Port-forwarding services
    • Screen capture on the device
    • Thread and heap information on the device
    • Logcat
    • Process, and radio state information
    • Incoming call and SMS spoofing
    • Location data spoofing, and more.
  • Monkey Tool
    • Runs on simulator or device
    • Generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events.
  • TaskKiller
    • A utility application used to kill process.
    • Useful to skip device reboot.

Blackberry

Additional Testing Aspects:

  • Allows running multiple applications in background
  • Device Reboot
    • Not much needed until for clearing cached data
    • Needed mainly for while uninstalling/installing any application.

Build Signing/Verification

  • Signed/certified by RIM

BlackBerry – Debugging

  • USB cable connected debugging
  • Can be done via JDE console after attaching debugger

Testing Tools

  • Native Logs:
  • A combination on Alt+LGLG keys opens the device logs in device.
  • Has 3 types of filters:
    • Debug
    • Information
    • Error
  • This helps analyze type of exceptions.

JL_Cmder:

  • A little batch script to manipulate JavaLoader.
  • JavaLoader, a tool shipped with JDE, can be used to easily:
    • Display Device Info
    • Display an event log
    • Take a screenshot
    • Wipe the handheld
    • Reset it to factory conditions

BBScreenshooter

  • Desktop Based tool
  • Needs device to be connected to desktop system for capturing screenshots

CaptureIt

  • Device Based tool
  • Device need not be connected to desktop for capturing screenshots.

Symbian

Additional Testing Aspects:

  • Allows running multiple applications in background
  • Device Reboot not much needed until for clearing cached data

Build Signing/Verification

  • Signed/certified by Symbian
  • By device manufacturing authority like Nokia

Symbian - Debug

  • Can be connected to the IDE via:
    • USB cable connected debugging
      • More preferred
    • Via BlueTooth
      • Less Preferred due to device driver issues

Testing Tools

  • Screenshot tools:
    • All tools are device based
    • After installing the tool
      • Set a shortcut key for screen capture.
      • Start the tool and then let it run in the background while application is run.
      • Clicking on the shortcut key captures screenshot and stores it in device.
    • Examples:
      • ScreenSnap S60 (5th Edition)
      • ScreenTaker for Nokia 7650/3650/N-Gage/SX1
  • Yucca Tools:
    • File browsing in Symbian device
    • Crash Monitor
    • Shows all supported fonts in device with preview.
    • Memory status

Windows mobile

Additional Testing Aspects

  • Allows running multiple applications in background
  • Can exit the application from task manager
    • Needed in case application gets hanged in background and its process needs to be killed

Windows Mobile

  • Debugging
  • Debugging
    • USB Cable connection needed between device and desktop
    • Can use visual studio

Testing Tools

  • Hopper
    • For stress testing.
    • Throws random events such as click events, entering text if text field in application run in background etc.
    • Logs are stored in device; screenshot is also stored if device hangs or if application crashes.

J2ME applications

J2ME is a Development Platform

Input Modes:

  • Follows input modes of the platform, some restrictions might exist.

Device Modes:

  • Based on platform

Additional Testing Aspects:

  • For mass markets handsets
  • Porting Friendly
    • Should work well on the supported device/ platform on which its installed.
    • Should be able to work well after the device firmware upgrades.

Build Signing/Verification

  • Verified by Verifying Authority.
  • Verified by device manufacturing authority like Nokia

Testing Tools

  • Tools provided by device platforms can be used.
  • Wireless Toolkit
  • Debug mode build.

Testing on simulator

Simulators:

  • A software application that can accurately imitate mobile phone.

The need to test on simulators:

  • Non availability of device
  • Testing same build in multiple device screens
  • Non availability of certain network for device testing in testing area.

Advantages:

  • Can help in isolating issues which are not volatile network connection dependent.
  • Can provide a wide variety of testing over different types of device simulators for the same build.

Disadvantages:

  • Simulators of older generation handsets don't resemble the device as closely
  • Some issues which are hit by the speed at which input was given cannot be reproduced easily
  • Hardware/Firmware environment variations detectable in device testing only
  • Device testing is always preferred as it represents more likely end user scenarios
Posted on 18/02/2011 1:57pm

Knowledge Center

Knowledge Center

Subscribe

*- required fields