Black Box – Robust Testing in software testing

By Prof. Fazal Rehman Shamil
Last modified on August 3rd, 2020

Black Box – Robust Testing in software testing

In robustness testing, the software is tested by giving invalid values as inputs. Robustness testing is usually done to test exception handling.

In robust boundary value testing,  we make combinations in such a way that some of the invalid values are also tested as input.

The type of testing done by causing the software or system to fail in order to test the robustness is called robustness testing.

Robust Boundary value testing on 3 variables:

How to test robustness of a software

Robust Testing Blac Box Testing analysis software testing

Suppose we have  3 variables X, Y and Z to test

The range of X: 0 to 100

The range of Y: 20 to 60

The range of Z: 80 to 100

        X y z
Min-      -1 19 79
Min 0 20 80
Min+ 1 21 81
Nominal 50 40 90
Max- 99 59 99
Max 100 60 100
Max+ 101 61 101

Figure: Testing points detected in Simple Robust Boundary Value Testing

Test Cases:

Total Test cases =

(Number of variables * Number of testing points without nominal )+ (1 for  Nominal)

These testing points are min-, min, min+, max- and max and max+

19=(3*6)+1

We can generate 19  test cases from both variables X, Y, and Z.

  • There are a total of 3 variables X, Y and Z
  • There are 6 possible values like min-, min, min+, max-, max and max+
  • 1 is for nominal

Logic:

When we make test cases, we will fix the nominal value of the two variables and change the values of the third variable.

For example

  • We will fix the nominal values of X and Y and make a combination of these values with each value of the Z variable.

Fix nominal values of X and Y are 50,40, and we will compare these two values with 79, 80,81,90,99,100 and 101.

  • We will fix the nominal values of X and Z and will make a combination of these values with each value of the Y variable.

Fix nominal values of X and Z are 50, 90, and we will make a combination of these two values with 19, 20,21,40,59,60 and 61.

  • We will fix the nominal values of Y and Z and will make a combination of these values with each value of the X variable.

Fix nominal values of Y and Z are 40, 90, and we will make a combination of these two values with -1, 0,1,50,99,100 and 101.

Test Case# X Y Z Comment
1 50 40 79 Fix Nominal of X and Y
2 50 40 80 Fix Nominal of X and Y
3 50 40 81 Fix Nominal of X and Y
4 50 40 90 Fix Nominal of X and Y
5 50 40 99 Fix Nominal of X and Y
6 50 40 100 Fix Nominal of X and Y
7 50 40 101 Fix Nominal of X and Y
8 50 19 90 Fix Nominal of X and Z
9 50 20 90 Fix Nominal of X and Z
10 50 21 90 Fix Nominal of X and Z
11 50 59 90 Fix Nominal of X and Z
12 50 60 90 Fix Nominal of X and Z
13 50 61 90 Fix Nominal of X and Z
14 -1 40 90 Fix Nominal of Y and Z
15 0 40 90 Fix Nominal of Y and Z
16 1 40 90 Fix Nominal of Y and Z
17 99 40 90 Fix Nominal of Y and Z
18 100 40 90 Fix Nominal of Y and Z
19 101 40 90 Fix Nominal of Y and Z

Figure: Test cases generated in Robust simple Boundary Value Testing.

Video Lecture

Another Example of robust boundary value software testing

nominal
min-
min
min+
max-
max
max+
X
7
4
5
6
8
9
10
Y
66
43
44
45
87
88
89
Z
51
2
3
4
98
99
100

 

test
cases
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
X

7
7
7
7
7
7
7
7
7
7
7
7
7
4
5
6
8
9
10

Y

66
66
66
66
66
66
66
43
44
45
87
88
89
66
66
66
66
66
66

Z

2
3
4
51
98
99
100
51
51
51
51
51
51
51
51
51
51
51
51

Download slides of Robustness Testing PPT

Download PDF of Robustness Testing pdf

Advantages of robustness testing

There are many benefits of robustness testing. Some of the benefits are mentioned below;
1. Better project analysis
Robustness testing means to increase the study of what has already been analyzed about your product. The robustness testing extends the area of testing of the previously tested software components. Robustness testing also test invalid values to satisfy the testing level.
3. Better design
The robustness testing result in more options and better software designs and it is completed before the finalization of the design of the product.
4. Achieve consistency
Robustness testing helps to increase the consistency, reliability, accuracy and efficiency of the software.

Frequently Asked Questions (FAQ)

What does robustness mean in hypothesis testing?
Robustness is the strength of a tests, and procedures according to the specific conditions of the statistical analysis a study hopes to achieve the goals.

Prof.Fazal Rehman Shamil (Available for Professional Discussions)
1. Message on Facebook page for discussions,
2. Video lectures on Youtube
3. Email is only for Advertisement/business enquiries.