An in-depth study of adaptive random testing for testing program with complex input types

Adaptive random testing (ART) is a testing method aimed to improve the performance of random testing. The initial studies of ART merely focussed on programs with numeric input type. However, real life software involves more complex types of inputs. This thesis examines the application of ART on programs with complex input types. Firstly, the application of a distance-based ART technique, namely FSCS-ART, on real programs with complex input types is conducted. This is an extension of two previous studies which developed new approaches to apply FSCS-ART on such programs. However, those studies have not applied the approaches to test any real programs. Here, they are applied to test four real programs with such input types. An exclusion-based ART, namely Restricted Random Testing (RRT), has been demostrated effective to test programs with numeric input type. We are motivated to also apply RRT on programs with complex input types. Some new approaches are introduced to adjust the use of RRT on such programs. Previous studies of ART have identified the regularity of failure causing inputs (FCIs), which are inputs that cause a program under test fails, in the numeric context. They have also observed how ART techniques behaved towards the identified patterns. This thesis investigates the extension of the study to more complex input types. Using the results of the relevant experiments in this thesis, we investigate the FCI patterns in a program with complex input types and analyse how the ART techniques behave towards the identified patterns. When applying ART, the existence of a test oracle is required to identify the detection of failures. Test oracle is more difficult to obtain when the program under test has more complex input types. Therefore, in this thesis we propose the use of Metamorphic Testing (MT) as the test oracle for applying ART on a program with complex input types. We examine how effective is MT compared to a test oracle which has been commonly used in previous ART studies. At last, an application of ART on other area of testing is conducted. We apply ART as the source test case selection strategy of MT. We investigate how effective ART compared to RT in selecting the source test cases of the implemented metamorphic relations.