Performance Testing nuts and bolts
Now a days, we have so many open source or licence tool to help us doing performance testing, however many of us still struggle with executing effective performance testing due to lack of knowledge or experience in this area.
This article will provide an overview of performance testing nuts and bolts – covering essential concepts & techniques needed for successfully conducting efficient Performance Testing
What exactly do you want to measure?
Defining the objectives of performance testing is critical for ensuring that you get valuable results.
It’s important to start with a business needs analysis and identify what output you expect from the load test as well as what questions you want to answer. This helps prevent paying for unnecessary tools or features.
If your application uses APIs, run performance tests against APIs, and if it has clients worldwide, consider running load tests from different locations in order to find latency issues. Keep in mind which countries/regions anticipate high usage.
It’s also essential to make these relevant to your business stakeholders by agreeing on success criteria that define exit criteria based on their expectations.
Get to know your application and its infrastructure first
To conduct effective performance testing, it’s important to get to know your application and its infrastructure first. Understanding the infra will help you to design your performance test and analyse it’s result effectively.
For example :-
- On which cloud platform your application services are deployed GCP, AWS, Azure or on premises servers.
- Is it on monolith application or it is combination of several micro services.
- Is it deployed on Kubernetes, or EC2 or GCP App Engine.
- Do your application have load balancer. Is there any autoscaling.
Having all these information will greatly help you to plan your performance testing and have better results.
Study your application’s analytics
Almost each application which is client facing, have analytics integrated in them. It can be one or combination of multiple third party provider like Google Analytics , MoEngage Etc.
Spend you time to analyse the user behaviour and traffic trends on the application. That help you design tests that are actually representative of your users. In this regard, analytics are a tester’s best friend!
Make sure to enable monitoring of the system under test
Make sure to enable monitoring of your servers before starting load testing on your application.
All cloud platform have their on monitoring system like cloudwatch on AWS, Cloud monitoring in GCP etc.
Having monitoring enabled help you to see how system behaves with different load. like CPU and Memory utilization, load balancer or multiple instance and traffic distribution among them.
Don’t forget the SQL sever
Application databases are often a critical component of an app’s functionality; therefore database performance is equally important . It’s important to enable monitoring on DB servers for not only CPU usage but also I/O and memory consumption.
Start small and go until you break your application
Start with running a test with just one user (client) per second, and check the performance metrics your are interested in.
Then go with 5, 10, 50, 100, 500 , 1000 users (clients) respectively.
Try to store the results in an easy to inspect manner, so that when you want to compare different test runs this would be a very easy task.
Take your time to Analyse load tests results:
Data collected from load tests should be analyzed carefully in order identify trends , patterns or issues.
Take your time and try to understand the results of your various tests, based on number of concurrent (parallel) users, duration, delay between clients start hitting the application, location etc.
- Inspect performance metrics like response time, latency, requests per second, errors %.
- Analyze CPU and Memory utilization.
- If there are load balancer or multiple instance of applications are running how the load is being distributed.
- If there is auto-scaling then at which % of load increase, a new instance is spawning.
All these information will help you make a better performance testing report for stack holder that will not only comprise of numbers but the insight of system that will be very useful to identify bottlenecks/errors/weaknesses in system.
Integrate tools to save your historical runs
Integrating tools to save your historical runs is a crucial aspect of performance testing. It allows you to keep track of past test results and compare them with current ones, identify patterns in system behaviour over time, and make data-driven decisions based on this information.
There are several ways to integrate tools for saving historical runs:
- Use cloud-based solutions: Many cloud-based load testing services offer built-in functionality for storing test results. These can be accessed from anywhere at any time via the internet.
- Leverage third-party plugins or extensions: There are many third-party plugins or extensions available for popular load testing tools like JMeter, K6 or Locust.
- Store your test result in any time series DB solution (influx, Prometheus, TimescaleDB)
- Access these historical data via tools like DataDog, Elastic Search , Grafana.
Involve the right people for successful testing
Performance Testing requires collaboration between various people from different department i.e. developers , operations team, Network Engineer, DBA, Business Owner and QA engineers.
Each will approach the problem from a different angle and you get an overview of the current performance of your application, website, API infrastructure, content and network. Having your team involve will get the most out of your performance testing run.
Test your production application as well
While it is valuable to test your application when it is in a staging environment, for a number of reasons this can leave some holes in your testing.
- Staging environments are not often exact duplicates of production.
- Staging environments are often accessible from only inside the firewall.
- There is something to be said for testing the same system that you are gathering information about.
Consider free and paid tools
While their are many open source tools available for load testing i.e. K6, locust and gatling and they also provide great flexibility to design your test scenarios. Their are also great feature provided by paid performance testing tool like storing historical run, distributed load testing that can save you a lot of hassle to setup everything by yourself
Make it part of deployment cycle:
We should monitor improvements or degradation of the system when a new build is released (code changes), when hardware is changed, when application architecture is changed, when server-side or database changes are made, when traffic spikes are expected, etc.
Make it part of the process:
Don’t make Performance testing and isolated on demand process. we should consider it an integral aspect of QA process.
It’s important to plan an agile testing methodology where performance testing is integrated into every step of the development cycle. Additional testing should be performed:
- When code is modified or updated.
- A new hardware is introduced in the the system.
- Some changes are made to the application server or DB server.
- If traffic spikes are anticipated due to a New launch or Marketing campaign.
After putting in the required effort to prepare for performance testing, you can take a deep breath and relax. Load testing your application will help you improve both your product and business continually.Performance testing need not be time-consuming, hard to understand or difficult to run. By conducting regular performance tests as part of every software cycle while ensuring they are done correctly, future disasters can be avoided with ease.
