Tools for Load Testing: benchmarking study
The article provides a brief overview and comparison of the following tools for load testing: Gatling, WAPT, CloudTest, SmartMeter, and NeoLoad. The review is called to assist in selecting a proper tool for verification of system capacities.
Tools for load testing were selected according to the following criteria:
- ability to carry high loads
- support of WebSockets
- multifunctionality (API, web and mobile traffic)
- currently supported
Gatling is an open-source framework for performance and load testing of web applications. Its first release dated in January 2012. Now Gatling FrontLine, an enterprise version, is available. The tool is based on three technologies: Scala, Netty, and Akka. It is a free framework.
The majority of QAs prefer to use JMeter for load testing. But it is not suitable for checking sockets. And Gatling is able to fulfill that task. Every Galing script consists of two parts: profile and configuration. Scenarios are developed in DSL (Domain-specific language).
WAPT is a tool for performance testing of web software including ERP (Enterprise Resource Planning) and CRM (Customer Relationship Management), mobile applications, and web API services. Two versions are available: WAPT and WAPT Pro.
A Pro version can be used in a continuous integration environment, thus it supports DevOps integration. It is a cloud-based solution for testing. The tool supports all popular web development frameworks. It provides test reports that can be customized.
CloudTest is a tool for load and performance testing of mobile and web software products. It was developed by SOASTA that is now a part of Akamai. The tool supports cloud-based testing. Test scripts designed in JMeter can be imported to CloudTest. SmartMeter is a tool for performance and load testing. It is based on JMeter.
SmartMeter is a commercial tool, but it has a free version with rather wide functionality. It generates test reports automatically. The tool has a test scenario recorder that also records logs. SmartMeter supports the integration with Selenium.
NeoLoad is a platform for load testing of cloud solutions, mobile and IoT (Internet of Things) applications, and web software products. It supports testing of APIs and web services. Also, its API supports customized integration.
Comparison of technical parameters
Gatling is rather limited, and it has no interface. CloudTest and SmartMeter have specific license programs. WAPT is very easy and intuitively clear to learn and use. It is more user-friendly. WAPT and NeoLoad have similar functionality.
NeoLoad has a better arrangement of test execution in a cloud. In case of WAPT, you have to arrange everything by yourself using commercial EC2. NeoLoad has a larger community and advanced marketing promotion in comparison to WAPT.