Non-functional Testing - A Brief Introduction

What is Non-functional testing?

It's very hard to define non-functional testing, without saying something like "testing the non-functional aspects of a system".  

To better understand what non functional testing is, it may be a good idea to distinguish between functional and non-functional testing.  

Functional testing verifies that a system adheres to the functional requirements that make up that system, for example, a requirement such as “the website should calculate the VAT on the shopping cart”, would define a unit of functionality that needs to be delivered by the system.   

Non-functional testing is concerned with testing ‘how’ a system performs in delivering on it’s requirements.  Eg.  Our authorisation backend should be capable of servicing X users per second.

Types of non-functional testing

A quick google will give hordes of different types of non-functional testing activities.  

‍Performance Testing: Evaluates the overall performance of the system.

Load Testing:Evaluates whether the system’s performance is as expected under normal and expected conditions.

Stress Testing:Evaluates whether the system’s performance is as expected when it is low on resources.

Volume Testing:Evaluates the behavior of the software when a large volume of data is involved.

Usability Testing:Evaluates the system for human use or checks if it is fit for use.

Compatibility Testing:Evaluates that the application is compatible with other hardware /software with minimum and maximum configuration.

Recovery Testing:Evaluates that the application terminates gracefully in case of any failure and the data is recovered appropriately from any hardware and software failures.

Instability Testing: Evaluates and confirms if the software installs and uninstalls correctly.

Documentation Testing:Evaluates the documents and other user manuals.

Failover Testing: Failover testing is done in order to verify that in case of a system failure the system is capable enough to handle extra resources like servers.

Security Testing:Security testing is done to ensure that the application has no loopholes which could lead to any data loss or threats. It is one of the important aspects of non-functional testing and if not performed properly, it can lead to security threats.

Scalability Testing:Scalability testing is done to verify if the application is capable enough to handle increased traffic, number of transactions, data volume, etc. The system should work as expected when the volume of data or change in the size of data is done.

Compliance Testing:Compliance testing is done to verify if the standards defined are being followed or not. Audits are done to verify the same.

Endurance Testing:Endurance Testing is done to verify the system’s behavior when a load is increased to an extent for a long time.

Localization Testing:Localization testing is done to verify the application in different languages i.e. different locales. The application should be verified for a particular culture or locale. The main focus is to test the content, GUI of the application.

Internationalization Testing:Internationalization testing is also known as i18n testing.

Reliability Testing:Reliability testing is done to verify if the application is reliable and is tested for a specific period of time in the defined environment. An application should give the same output as expected every time, only then it can be considered as reliable.

Portability Testing:Portability testing is done to verify if in case a software/application is installed on a different system or on a different platform it should be able to run as expected i.e. no functionality should be affected because of a change in the environment.

Baseline Testing:Baseline testing is also known as benchmark testing as it creates a base for any new application to be tested.

Efficiency Testing:Efficiency testing is done to verify if the application works efficiently and the number of resources required, tools required, complexity, customer requirement, environment required, time, what kind of project it is, etc.

Disaster Recovery Testing:This testing is done to verify the success rate of recovery of an application or system if any critical failure happens and whether the system is able to restore the data and application or the system could cope up easily to return the way it was working earlier i.e. from the operational front.

Maintainability Testing:Once the application/Product goes live, then there are chances for an issue to come up in the live environment or the customer may want an enhancement for the application which is already live.

This may seem daunting at first, but the different types of non-functional testing are very specialised and it is not often the case that any one person would be involved across the spectrum of non-functional testing activities.  

All things are not created equal

In this lean and agile software world, not every type of non-functional testing activity would deliver the same value for every organisation.  For example, suppose you have a system that provides users with non-sensitive information but it is of critical importance that the information be delivered to as many users as possible.  In such a system, security testing may have less value than performance testing.

So it is therefore important to assess your organization's needs before diving down the non-functional testing rabbit hole and make sure you are doing the right testing.

Where to from here?

Non functional testing is a very specialised practice and at Code Magus our team of engineers are dedicated to providing expert performance testing services, which is a large but focused area of non-functional testing. In this blog series we’ll be focussing on performance testing.

‍