This sixth post in the series of software testing overviews introduces the first of some 80+ different test types. That number is completely arbitrary. While investigating test types I found over a hundred definitions but I have chosen to leave out a number of those ‘test types’. My choice to do so is based on the interpretation that these test types rather described a test level or a test technique and I could not see how to make a useful test type out of them.
So what is a test type?
To me a test type is a particular approach of testing that has an approach, goal and/or use of oracle(s) that provides information that is typical to that test type.
While going through my overview you might find that some of the test types I mention do not fit the above description of a test type. The reason that they are mentioned despite of this is that I felt that they were so often mentioned as a test type that they should have a mention in this post just for that reason.
This post differs somewhat from the earlier posts as the definitions used are mostly my own and for some of them I have added comments as additional information. Also since a post with over 80 descriptions would be too long I will split up the overview into alphabetical sections. To begin with the A.
A/B testing
A/B testing originates from marketing research used to investigate the more efficient of two possible solutions by presenting them to two different groups and measuring the ‘profits’. In software testing it is mostly used to compare two versions of a program or website, often of which only one contains changes on a single or a few controllable criteria.
Acceptance testing
During acceptance testing requirements, variables, parts or behaviour of a program are compared against measurable aspects of predetermined acceptance criteria of those requirements variables, parts or behaviour of the program. This requires at least four things. First identification of the requirements, variables and parts or behaviour (coverage). Second expressing these in measurable aspects. Third the aspects need to represent the defining elements of the acceptance criteria. Finally the acceptance criteria themselves should represent the needs and wants of the stakeholder.
Active testing
Testing the program by triggering actions and events in the program and studying the results. To be honest I do not consider this a test type as in my opinion this describes nearly all types of testing.
Ad-hoc testing
Ad-hoc testing is software testing performed without prior planning or documentation on direction of the test, on how to test or on the oracles used. Some definitions see this as informal, unstructured and not reproducible. Informal and unstructured (if seen as unprepared) is certainly true as this would be the point of doing it ad-hoc. The not reproducible part depends on whether you care to record the test progress and test results. Something that is in my opinion is not inherently attached to doing something ad-hoc.
Age testing
It is a testing technique that evaluates a system’s ability to perform in the future and usually carried out by test teams. As the system gets older, how significantly the performance might drop is what is being measured in Age Testing. To be honest I found only one reference to this test type but I find the idea interesting.
Agile testing
Agile testing is mentioned often as a test type or test approach but I have added no definition or description here. In my opinion agile testing is not a software test type. Agile testing rather is a particular context in which testing is performed that may have its particular challenges on test execution, on how tests are approached and on choices of test tooling but not a specific test type.
Alpha testing
Alpha testing is an in-house (full) integration test of the near complete product that is executed by others than the development team but still is executed in a development environment. Alpha testing simulates the products intended use and helps catch design flaws and operational bugs. One could argue that this is more of a test level than a test type. I care to view it as test type because it is more about the type of use and its potential to discover new information than that is part of the software development itself. I specifically disagree with the idea that this is an intermediary step towards, or is part of, handing over software to a Test/QA group. In my opinion testing is integrated right from the start of development up until it stops because the product ends its lifecycle or due to some other stopping heuristic.
API testing
API testing involves testing individual or combined inputs, outputs and business rules of the API under investigation. Essentially an API is a device-independent, or component-independent access provider that receives, interprets/transforms and sends messages so that different parts of a computer or programs can use each other’s operations and/or information. Testing an API is similar to testing in general albeit that an API has a smaller scope has, or should have, specific contracts and definitions that describe the API’s specific variables, value ranges and (business) rules. Testing an API should however not be limited to the API alone. Sources, destinations (end-points), web services (e.g. REST, SOAP), message types (e.g. JSON, XML), message formats (e.g. SWIFT, FIX, EDI, CSV), transport- (e.g. HTTP(S), JMS, MQ) and communication protocols (e.g. TCP/IP, SMTP, MQTT, TIBCO Rendezvous) all influence the overall possibilities and functionality of the API in relation to the system(s) that use(s) the API. Typically API testing is semi- or fully automated and requires sufficient tool, message type, and transport- and communication protocol knowledge to be executed well.
