Skip to the content
AI Test AutomationPreview AI Security Learning Center

AI Test AutomationLearnTest types

Test types Preview

Every kind of test, and the tools teams actually use

Unit, integration, API, end-to-end, UI, manual, exploratory, load, stress and soak: what each kind of test is for, when it pays off, what it misses, and the tools teams use for it, with the numbers that show how widely each one is used.

The shape of a test suite

Many small tests, fewer big ones

The test pyramid says to write many small, fast tests at the bottom and few broad, slow ones at the top. Martin Fowler credits it to Mike Cohn's 2009 book Succeeding with Agile (first drawn "in conversation with Lisa Crispin in 2003-4") and states its point as "you should have many more low-level UnitTests than high level BroadStackTests running through a GUI".

Testing trophy (Kent C. Dodds, introduced in a February 2018 tweet): four layers, Static ("Catch typos and type errors as you write the code"), Unit, Integration and End to End, where "The size of these forms of testing on the trophy is relative to the amount of focus you should give them", with integration largest, echoing Guillermo Rauch's "Write tests. Not too many. Mostly integration." Its guiding principle: "The more your tests resemble the way your software is used, the more confidence they can give you." Dodds says he built it for JavaScript apps and has "only considered it for monoliths not microservices or even serverless functions".

Sources: Martin Fowler, TestPyramid (bliki, 1 May 2012) · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 26 February 2018) · ISTQB Glossary: test pyramid (glossary API record) · ISTQB Certified Tester Foundation Level Syllabus v4.0.1, 5.1.6 Test Pyramid and 5.1.7 Testing Quadrants

Levels

From one function to the whole product

The automated levels, from the smallest and fastest to the biggest and slowest. Most suites need all of them, in different amounts.

Unit tests

A unit test checks one small piece of code, such as a function, method or class, on its own, usually with its outside dependencies replaced by test doubles or mocks. ISTQB defines unit testing as "A test level that focuses on evaluating the smallest part of code testable in isolation."

When to use it
Make them the bulk of the suite and run them on every change; AWS says "Teams should be able to run unit tests locally as well as through continuous integration pipelines." Google's Software Engineering book gives "a very rough guideline" of "around 80%" narrow-scoped unit tests "that validate the majority of our business logic", and the Google Testing Blog suggests "70% unit tests" as "a good first guess". They are not enough alone: "even if the units work well in isolation, you do not know if they work well together" (Google Testing Blog), so pair them with integration tests.

JestVitestpytestJUnitNUnitxUnit.netMocha

What it catches, what to watch out for, sources
What it catches
Flaws in the logic of a single unit: wrong output for given inputs (AWS: tests "run the code using predefined inputs and set expectations for a specific output") and regressions introduced by changes (AWS: "reducing the risk of introducing flaws when making changes"). Failures are easy to locate: "if a unit test fails, you only need to search that small unit under test to find the bug" (Google Testing Blog).
Watch out for
There is no single agreed meaning of "unit": Ham Vocke says asking three people gets "four different, slightly nuanced answers", and teams differ on solitary tests (all collaborators stubbed) versus sociable tests (real collaborators allowed). Tests that mirror internal structure break on every refactor; Vocke's advice is "Don't reflect your internal code structure within your unit tests." Slow suites stop being run: the Google Testing Blog says "one tenth of a second is considered slow for unit tests", Microsoft's guidance sets an average under 60 milliseconds per L0 test and says "No test at this level should exceed 2 seconds", and Martin Fowler says the commit suite "should run in no more than ten minutes".

Sources: ISTQB Glossary: unit testing (glossary API record) · ISTQB Glossary: component testing (glossary API record) · ISTQB Certified Tester Foundation Level Syllabus v4.0.1 (2024-09-15), 2.2.1 Test Levels · Martin Fowler, UnitTest (bliki, 5 May 2014) · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 26 February 2018) · Google Testing Blog, Mike Wacker, Just Say No to More End-to-End Tests (22 April 2015) · Same post, text read via the Google Testing Blog Atom feed · Software Engineering at Google, ch. 11 Testing Overview (Adam Bender) · AWS DevOps Guidance, [QA.FT.1] Ensure individual component functionality with unit tests · AWS DevOps Guidance, Anti-patterns for functional testing · Microsoft Learn, Shift testing left with unit tests · Kent C. Dodds, Static vs Unit vs Integration vs E2E Testing for Frontend Apps

Integration tests

An integration test checks that separately built parts work together, such as your code with a real database, file system, message queue or another service. ISTQB defines integration testing as "A test level that focuses on interactions between components or systems."

When to use it
Write them at every boundary where your code talks to something outside itself. Ham Vocke's checklist: code that serializes or deserializes data, calls to your services' REST API, reading from and writing to databases, calling other applications' APIs, reading from and writing to queues, and writing to the filesystem.

TestcontainersMock Service Worker (MSW)

What it catches, what to watch out for, sources
What it catches
Bugs at boundaries that unit tests cannot see: Vocke says these tests ensure "writing data to and reading data from these external collaborators works fine", including serialization and deserialization, and AWS says they help ensure "recent changes have not disrupted any interfaces or introduced undesired behaviors." Kent C. Dodds's one-line definition is "Verify that several units work together in harmony", which covers calls into dependencies that unit tests with mocks cannot confirm.
Watch out for
The term is vague: Vocke calls "integration tests" an even vaguer term than "unit tests", and Martin Fowler separates narrow integration tests (test doubles for other services) from broad ones that "require live versions of all services, requiring substantial test environment and network access". They are slower: AWS says integration tests "often run much slower than unit testing" because they touch real databases, message queues and external APIs, and recommends automating setup and teardown with infrastructure as code and running tests in parallel.

Sources: ISTQB Glossary: integration testing (glossary API record) · ISTQB Certified Tester Foundation Level Syllabus v4.0.1, 2.2.1 Test Levels · Martin Fowler, IntegrationTest (bliki, 16 January 2018) · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 2018) · Software Engineering at Google, ch. 11 Testing Overview · Google Testing Blog, Just Say No to More End-to-End Tests (2015) · AWS DevOps Guidance, [QA.FT.2] Validate system interactions and data flows with integration tests · Spotify Engineering, Andre Schaffer and Rickard Dybeck, Testing of Microservices (11 January 2018) · Kent C. Dodds, Static vs Unit vs Integration vs E2E Testing for Frontend Apps

API tests

An API test exercises a service directly through its programming interface, sending requests and comparing the responses with the expected results, without going through a user interface. ISTQB defines API testing as "A test approach performed by submitting requests to a test object using its application programming interface."

When to use it
For services with an API, treat the API as a real interface: Ham Vocke notes "a REST API or a command line interface is as much of a user interface as a fancy web user interface", and if your REST API is your user interface you can cover it with "proper integration tests around your API". Postman describes several kinds of tests run against an API: unit ("a single endpoint returns the correct response to a given request"), functional, integration, end-to-end, contract, load and security tests, and recommends a dedicated testing environment, automation, reusable subtests and running tests "throughout the API lifecycle".

PostmanBrunoInsomniaREST AssuredKarateHoppscotch

What it catches, what to watch out for, sources
What it catches
Wrong responses for given requests, unmet requirements, and broken communication between services (Postman: integration tests confirm "that APIs can reliably and efficiently communicate and transfer data between one another"). Load tests against an API check it can "operate reliably during times of peak traffic", and API security tests look for weaknesses that "may result in unauthorized access, data breaches, injection attacks, or other security risks" (Postman).
Watch out for
Postman's best practices point at the usual gaps: no dedicated test environment, manual runs instead of automated ones in CI/CD, tests run only late instead of throughout the lifecycle, and unorganized, non-reusable tests. Microsoft warns that tests built on special test-only integration points depend on internal knowledge and "are limited to environments that have the secrets and configuration necessary to run the tests", and that tests must be isolated so they "can run reliably in any sequence".

Sources: ISTQB Glossary: API testing (glossary API record) · Postman, What is API testing? · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 2018), UI Tests section · Microsoft Learn, Shift testing left with unit tests · ISTQB Certified Tester Foundation Level Syllabus v4.0.1, 5.1.7 Testing Quadrants · Pact docs, What is Pact good for? · Google Testing Blog, Just Say No to More End-to-End Tests (2015)

Contract tests

A contract test checks that two services agree on the messages they exchange by testing each side on its own against a shared, written contract, instead of deploying both together. Pact defines contract testing as "a technique for testing an integration point by checking each application in isolation to ensure the messages it sends or receives conform to a shared understanding that is documented in a 'contract'", and ISTQB calls it "A type of integration testing that verifies that the interfaces are used as specified in their contracts."

When to use it
Use them between services built by different teams, such as microservices. With consumer-driven contracts, consumers "write tests that check the interface for all data they need" and the provider team runs them "continuously (in their build pipeline)" (Ham Vocke).

Pact

What it catches, what to watch out for, sources
What it catches
Breaking interface changes before they reach production; AWS says contracts capture "everything from request structures to expected responses", and Vocke says "If they break the interface their CDC tests will fail, preventing breaking changes to go live." Fowler's version catches test doubles that have drifted: contract tests "check that all the calls against your test doubles return the same results as a call to the external service would." Pact notes "only parts of the communication that are actually used by the consumer(s) get tested."
Watch out for
Pact lists where it is a poor fit: APIs where "the consumers cannot be individually identified (eg. public APIs)", "Performance and load testing.", "Functional testing of the provider", "pass through" APIs, use "as a general purpose mocking or stubbing tool for browser driven tests", and teams that "do not have good communication channels". Martin Fowler says a failing contract test "shouldn't necessarily break the build in the same way that a normal test failure would" but should "trigger a task to get things consistent again", and may "trigger a conversation with the keepers of the external service".

Sources: ISTQB Glossary: contract testing (glossary API record) · Pact docs, Introduction · Pact docs, What is Pact good for? · Martin Fowler, ContractTest (bliki, 12 January 2011) · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 2018), Contract Tests section · AWS DevOps Guidance, [QA.NT.7] Verify service integrations through contract testing

End-to-end tests

An end-to-end test runs a complete user journey or business process through the whole, fully integrated system, often by driving the real user interface against deployed services. ISTQB defines it as "A test type in which business processes are tested from start to finish under production-like circumstances."

When to use it
Keep a small set for the most valuable user journeys. Google suggests "10% end-to-end tests" as a first guess (Testing Blog) and "5% end-to-end tests that validate the entire system" (SWE book); Ham Vocke says to "reduce the number of end-to-end tests to a bare minimum" and to "Think about the high-value interactions users will have with your application." AWS recommends "identifying the most impactful user flows" and running scripted end-to-end tests from the continuous delivery pipeline against a test environment.

PlaywrightCypressSeleniumWebdriverIO

What it catches, what to watch out for, sources
What it catches
Failures that only appear in the complete system. Microsoft's playbook notes "if there is any failure or fault in any sub-system, it can adversely affect the whole software system leading to its collapse", and Vocke says end-to-end tests "give you the biggest confidence when you need to decide if your software is working or not".
Watch out for
They are slow, flaky and hard to debug. Vocke: they "are notoriously flaky and often fail for unexpected and unforeseeable reasons", citing "Browser quirks, timing issues, animations and unexpected popup dialogs".

Sources: ISTQB Glossary: end-to-end testing (glossary API record) · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 2018), End-to-End Tests section · Google Testing Blog, Mike Wacker, Just Say No to More End-to-End Tests (22 April 2015) · Google Testing Blog, John Micco, Flaky Tests at Google and How We Mitigate Them (27 May 2016) · Same flaky-tests post, text read via the Google Testing Blog Atom feed · Software Engineering at Google, ch. 11 Testing Overview · AWS DevOps Guidance, [QA.FT.3] Confirm end-user experience and functional correctness with acceptance tests · Microsoft Code-With Engineering Playbook, E2E Testing · Martin Fowler, BroadStackTest (bliki, 22 April 2013) · Martin Fowler, TestPyramid (bliki, 1 May 2012) · Kent C. Dodds, Static vs Unit vs Integration vs E2E Testing for Frontend Apps

UI tests

A UI test checks that the user interface works: "User input should trigger the right actions, data should be presented to the user, the UI state should change as expected" (Ham Vocke). ISTQB calls this GUI testing, "A test approach performed by interacting with a test object using a graphical user interface."

When to use it
UI testing does not have to be end-to-end: Vocke writes "Testing your user interface doesn't have to be done in an end-to-end fashion", and modern frontend frameworks let you test UI behavior at unit level with the backend stubbed out. Use browser-level UI tests where fidelity matters most; Microsoft's Engineering Playbook says UI testing is "mainly applicable when dealing with a public facing UI that is used in a diverse environment or in a mission critical UI that requires higher fidelity", and less of a priority for an admin UI "used by just a handful of people".

PlaywrightCypressSeleniumTesting LibraryStorybookPuppeteer

What it catches, what to watch out for, sources
What it catches
"Runtime" bugs that unit and functional tests miss; Microsoft's example is a submit button that "is rendered but not clickable due to a positioning bug in the UI". The playbook also says UI tests "Detect changes in the UI both across devices and delivery platforms and between code changes", and Playwright's screenshot comparisons flag visual differences against a stored baseline.
Watch out for
UI tests are slow to write and run and often unreliable. Microsoft's DevOps guidance says "Discourage the use of UI tests, because they tend to be unreliable"; its playbook warns that raw Selenium brings "timeouts, missing elements" and friction, and advises a framework built on top of Selenium or one that solves its problems.

Sources: ISTQB Glossary: GUI testing (glossary API record) · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 2018), UI Tests section · Microsoft Code-With Engineering Playbook, UI Testing · Microsoft Learn, Shift testing left with unit tests · Playwright docs, Best Practices · Playwright docs, Visual comparisons · AWS DevOps Guidance, [QA.FT.3] acceptance tests (names Selenium and AWS Device Farm) · Martin Fowler, TestPyramid (bliki, 1 May 2012) · Kent C. Dodds, The Testing Trophy and Testing Classifications (names Cypress for end to end)

People in the loop

Manual, semi-automated and exploratory testing

Automation checks what you already know to ask. People find what nobody thought to ask, and judge what a script cannot.

When a person is still the right tester

  • Judging usability and look-and-feel. Ham Vocke: "Once you want to test for usability and a 'looks good' factor you leave the realms of automated testing"; rely on exploratory testing, usability testing ("this can even be as simple as hallway testing") and showcases with users. AWS describes usability tests that observe participants doing typical tasks and "encourage the participants to verbalize their thought process".
  • Outputs only a person can grade. Google uses human judgment for search-result quality ("Search Quality Raters who execute real queries and record their impressions") and for audio and video quality in telephony and video calls, and says rater evaluation "is critical for nondeterministic systems like machine learning systems for which there is no clear correct answer, only a notion of better or worse."
  • Exploring new or changed features and hunting serious flaws. ISTQB says exploratory testing is "useful when there are few or inadequate specifications or there is significant time pressure on the testing"; Google says "searching for complex security vulnerabilities is something that humans do better than automated systems".
  • Accessibility with assistive technologies. AWS: "Consider regularly conducting manual exploratory tests using assistive technologies to capture issues that automated tools might miss", in addition to automated checkers such as Axe or WAVE.
  • Acceptance by real users. AWS describes user acceptance testing as tests "performed by external end-users of the system to validate that the system aligns with business needs and requirements", and ISTQB says "Ideally, acceptance testing should be performed by the intended users." Azure Test Plans supports UAT plans and inviting testers.
  • Pre-launch bug bashes. Google describes a session where "everyone involved manually tests the product", sometimes with published focus areas.

Semi-automated: "Semi-automated testing" is not an ISTQB glossary term; it describes splitting a test between tools and people, and the sources show two working patterns. (1) Automation prepares and runs, a person judges. ISTQB counts "the setting up of test preconditions" and "the comparison of actual results to expected results" as tool work, and defines test data preparation as "The activity to select data from existing databases or create, generate, manipulate and edit data for testing." Google's A/B diff tests send traffic to the old and new versions and compare the responses, but "Someone must understand the results enough to know whether any differences are expected." Playwright's visual comparisons check screenshots against stored baselines that you commit and "review any changes to", and its Trace Viewer lets a person "explore recorded Playwright traces after the script has run." (2) A person drives, tools record and check.

Sources: Ham Vocke, The Practical Test Pyramid (martinfowler.com, 2018) · Software Engineering at Google, ch. 11 Testing Overview · Software Engineering at Google, ch. 14 Larger Testing · ISTQB Certified Tester Foundation Level Syllabus v4.0.1 (4.4.2, 2.2.1, 6.2)

Manual testing

In manual testing a person carries out the test, running the steps, observing the product and judging the result, instead of a script doing it. ISTQB's definition is simply "A test activity performed manually", and its Foundation syllabus places exploratory testing, usability testing and user acceptance testing in a quadrant whose tests are "user-oriented and often manual."

When to use it
Use it when the judgment needs a person: usability and look-and-feel, outputs with no single right answer, accessibility with assistive technology, acceptance by real users, and exploring new features (sourced examples are in the manual section). Vocke: "Despite your best intentions with regards to test automation, manual testing of some sorts is still a good idea." Avoid it for repeatable regression checks; Google says "manual regression testing does not scale sublinearly".

TestRailXrayQaseKiwi TCMS

What it catches, what to watch out for, sources
What it catches
Issues automation is poor at seeing: design and usability problems, "slow response times, missing or misleading error messages and everything else that would annoy you as a user of your software" (Vocke), accessibility issues "that automated tools might miss" (AWS), quality that needs human judgment such as search results or audio and video (Google), and whether the system "aligns with business needs and requirements" (AWS on user acceptance testing).
Watch out for
It is slow and costly: "Manual testing does not scale sublinearly; that is, it requires human time to perform the manual tests" (Google), and Kent C. Dodds says manual testing would give "really great confidence" but the tests "would be really expensive and slow".

Sources: ISTQB Glossary: manual testing (glossary API record) · ISTQB Certified Tester Foundation Level Syllabus v4.0.1, 5.1.7 Testing Quadrants and 6.2 Benefits and Risks of Test Automation · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 2018), Exploratory Testing section · Software Engineering at Google, ch. 11 Testing Overview · Software Engineering at Google, ch. 14 Larger Testing (Joseph Graves) · AWS DevOps Guidance, [QA.NT.3] Prioritize user experience with UX testing · AWS DevOps Guidance, [QA.FT.3] acceptance tests (user acceptance testing) · Microsoft Learn, What is Azure Test Plans? Manual and automated testing · Kent C. Dodds, Static vs Unit vs Integration vs E2E Testing for Frontend Apps

Semi-automated testing

Semi-automated testing splits a test between tools and people: tools handle repeatable work such as setting up data and state, driving steps, recording and comparing output, while a person makes the judgment call, or a person drives and tools record and check. James Bach and Michael Bolton call this "human/machine checking": "an attempted checking process wherein both humans and tools cooperate to verify propositions about the product."

When to use it
It fits where a machine can do the legwork but cannot decide pass or fail alone. Sourced examples: Google's A/B diff tests send traffic to the old and new versions and compare responses, but "Someone must understand the results enough to know whether any differences are expected"; Playwright stores screenshot baselines you commit and "review any changes to it"; Playwright's test generator records a person's clicks ("generate tests for you as you perform actions in the browser") and adds assertions they pick; Playwright's Trace Viewer lets a person "explore recorded Playwright traces after the script has run"; Azure Test Plans' Test Runner collects screenshots, action logs and screen recordings while a person runs manual tests.
What it catches, what to watch out for, sources
What it catches
Regressions that need a person to call expected or unexpected, such as behavior differences between versions and screenshot differences, plus hard-to-reproduce bugs found in manual sessions, because the tool captured the steps: Azure Test Plans says bugs filed during a run "automatically include all captured diagnostic data to help your developers reproduce the issues."
Watch out for
The human step can become a bottleneck or a rubber stamp: Google lists "Approval" of diffs as a challenge, and Bach and Bolton say tools "necessarily play a supporting role to skilled humans; and the unskilled use of tools may have terrible consequences." Baselines are environment-sensitive, so Playwright advises: "For consistent screenshots, run tests in the same environment where the baseline screenshots were generated." Tool support changes: Microsoft's Test & Feedback extension for Firefox retires on November 30, 2026, while the Chrome and Edge versions continue.

Sources: James Bach and Michael Bolton, Testing and Checking Refined (satisfice.com) · ISTQB Glossary: test execution automation (glossary API record) · ISTQB Glossary: test data preparation (glossary API record) · Software Engineering at Google, ch. 14 Larger Testing, A/B diff (regression) testing · Playwright docs, Test generator (codegen) · Playwright docs, Visual comparisons · Playwright docs, Trace viewer · Microsoft Learn, What is Azure Test Plans? Manual and automated testing · Microsoft Learn, Install the Test & Feedback extension (exploratory testing)

Exploratory testing

In exploratory testing a person designs, runs and evaluates tests at the same time while learning about the product, instead of following a fixed script. ISTQB defines it as "A test approach in which tests are dynamically designed and executed based on tester's knowledge, exploration of a test item, and previous test results."

When to use it
Run it regularly alongside automation. ISTQB says it is "useful when there are few or inadequate specifications or there is significant time pressure on the testing" and "useful to complement other more formal test techniques"; Ham Vocke says to "take some time on a regular schedule, roll up your sleeves and try to break your application." For accountability, use session-based test management (Jonathan Bach with James Bach, 2000): a session is "an uninterrupted block of reviewable, chartered test effort", guided by a charter (ISTQB: "A mission statement that guides an exploratory test session"), recorded on a session sheet (charter, tester names, date and time started, task breakdown, data files, test notes, issues, bugs) and followed by a debriefing using the PROOF agenda (Past, Results, Obstacles, Outlook, Feelings).

Bug MagnetBug Magnet AI assistant

What it catches, what to watch out for, sources
What it catches
Problems nobody predicted: "When conducting an exploratory test, the specific problems to be found are unknown at the start" (Google). Testers look for new paths through the system, behavior that "deviates from either expected or intuitive behavior", and security vulnerabilities (Google), and for "bugs, design issues, slow response times, missing or misleading error messages" (Vocke).
Watch out for
Unstructured exploration is hard to track and report; SBTM adds accountability, but Jonathan Bach found it "weighs especially heavily on the test manager", and dropping debriefings "led to poor session sheets and meaningless metrics." Results depend on the tester: ISTQB says it is more effective if the tester "is experienced, has domain knowledge" and has skills such as curiosity and creativeness. It does not scale like automation, so findings should become automated regression tests (Google, Vocke).

Sources: ISTQB Glossary: exploratory testing (glossary API record) · ISTQB Glossary: session-based testing (glossary API record) · ISTQB Glossary: session-based test management, SBTM (glossary API record) · ISTQB Glossary: test charter (glossary API record) · ISTQB Certified Tester Foundation Level Syllabus v4.0.1, 4.4.2 Exploratory Testing · Jonathan Bach, Session-Based Test Management (STQE magazine, Nov/Dec 2000), satisfice.com · Same article, PDF download · James Bach and Michael Bolton, Exploratory Testing 3.0 (satisfice.com) · Ham Vocke, The Practical Test Pyramid (martinfowler.com, 2018), Exploratory Testing section · Software Engineering at Google, ch. 11 Testing Overview · Software Engineering at Google, ch. 14 Larger Testing

Performance

Load, stress, soak and spike

The same system under different kinds of pressure: expected traffic, more than expected, for a long time, and all at once.

Load testing

A load test runs the system at the traffic you expect, such as a regular day in production, to check that performance and resource use stay stable. ISTQB defines it as "A type of performance testing to evaluate the behavior of a component or system under varying loads, usually between anticipated conditions of low, typical, and peak usage."

When to use it
Run it routinely: AWS recommends integrating automated load testing into the deployment pipeline so "every change undergoes validation of system behavior under expected scenarios". Grafana k6 calls this the average-load test (also "day-in-life test or volume test"), run at "Average production" load for "Mid (5-60 minutes)", with a ramp-up that "usually lasts between 5% and 15% of the total test duration" and a steady state "at least five times longer than the ramp-up". k6: "Start with smoke tests, then progress to higher loads and longer durations"; a smoke test is a minimal-load run to "Verify that your test script doesn't have errors", repeated "Each time you create or update a script".

Grafana k6Apache JMeterGatlingLocustArtillery

What it catches, what to watch out for, sources
What it catches
Bottlenecks and issues at expected load (Microsoft playbook: "Identifying bottlenecks and issues with the system at different load levels"), "early degradation signs during the ramp-up or full load periods" (k6), whether "the system's performance and resource consumption stay stable during the period of full load" (k6), and regressions between releases (Google: ensuring "there is no degradation in performance between versions").
Watch out for
Using functional UI tools as load generators: Selenium's docs say "Performance testing using Selenium and WebDriver is generally not advised. Not because it is incapable, but because it is not optimised for the job and you are unlikely to get good results." An environment unlike production gives misleading numbers (AWS). Not comparing against a baseline hides regressions; AWS says to "Regularly analyze test results against historical benchmarks".

Sources: ISTQB Glossary: load testing (glossary API record) · ISTQB Certified Tester Performance Testing Syllabus, Version 2018, 1.2 Types of Performance Testing · Grafana k6 docs, Load test types (cheat sheet) · Grafana k6 docs, Average-load testing · Grafana k6 docs, Smoke testing · AWS DevOps Guidance, [QA.NT.2] Validate system reliability with performance testing · Microsoft Code-With Engineering Playbook, Performance Testing · Selenium docs, Discouraged behaviors: Performance testing · Software Engineering at Google, ch. 14 Larger Testing, Performance, load, and stress testing

Stress testing

A stress test pushes load above normal, up to and past expected limits, to see how far performance degrades and whether the system survives and recovers. ISTQB defines it as "A type of performance testing to evaluate a system or component at or beyond the limits of its anticipated or specified workloads, or with reduced availability of resources such as access to memory or servers."

When to use it
Run it when the system may face above-average load and after significant application or infrastructure changes; AWS also suggests periodic runs, "such as once a month", to prepare for traffic spikes or DDoS attacks. k6: "Only run stress tests after running average-load tests", reuse the average-load script with a higher target, and run for "Mid (5-60 minutes)". To find the ceiling, k6's breakpoint test (also called "capacity, point load, and limit testing") is the related type: "Breakpoint tests gradually increase load to identify the capacity limits of the system"; "Run breakpoints only when the system is known to perform under all other test types."

Grafana k6Apache JMeterGatlingLocust

What it catches, what to watch out for, sources
What it catches
How much performance degrades and whether the system survives (k6: it "determines how much the performance degrades with the extra load and whether the system survives it"), breaking points (AWS: "Stress tests identify the system's breaking points"), recovery behavior (Microsoft playbook: "does it recover (i.e., scale-out) or does it just break and fail?"), and behavior with reduced resources such as memory or servers (ISTQB).
Watch out for
Running it before average-load tests pass wastes effort (k6). Pass criteria must change: k6 says "Expect worse performance compared to average load." Elastic cloud environments can hide limits: for breakpoint tests k6 warns the environment may grow until you find "only the limit of your cloud account bill" and says "turning off elasticity on all the affected components is strongly recommended."

Sources: ISTQB Glossary: stress testing (glossary API record) · ISTQB Certified Tester Performance Testing Syllabus, Version 2018, 1.2 Types of Performance Testing · Grafana k6 docs, Stress testing · Grafana k6 docs, Breakpoint testing · Grafana k6 docs, Load test types (cheat sheet) · AWS DevOps Guidance, [QA.NT.2] Validate system reliability with performance testing · Microsoft Code-With Engineering Playbook, Performance Testing

Soak (endurance) testing

A soak test holds a normal, average load for a long time, hours or days, to find problems that only appear with prolonged use. Grafana k6 says a soak test "might be called an endurance, constant high load, or stamina test", and ISTQB's term is endurance testing: "Testing to determine the stability of a system under a significant load over a significant period of time within the system's operational context."

When to use it
Run it after smoke and average-load tests pass (k6: don't run soak tests "before running smoke and average-load tests"), after significant changes that might introduce leaks, and periodically; AWS suggests "quarterly or biannually". k6 says to make the duration "considerably longer than any other test", with "Some typical values are 3, 4, 8, 12, 24, and 48 to 72 hours", reusing the average-load script and changing only the peak duration. Microsoft's playbook describes it as "a load test carried over a long duration ranging from several hours to days".

Grafana k6GatlingLocust

What it catches, what to watch out for, sources
What it catches
"common performance defects that show only after extended use" (k6): "response time degradation, memory or other resource leaks, data saturation, and storage depletion". The ISTQB performance syllabus names resource capacity problems such as "memory leaks, database connections, thread pools", and AWS mentions "slow memory leaks or performance degradation".
Watch out for
Failures show up as slow trends, so watch the backend: k6 says "Of all test types, backend monitoring is especially important for soak tests", and AWS says to "Monitor key performance indicators over time and compare against established benchmarks". Names vary (soak, endurance, stamina), and "soak testing" is not an ISTQB glossary term, so agree on terms within the team.

Sources: ISTQB Glossary: endurance testing (glossary API record) · ISTQB Certified Tester Performance Testing Syllabus, Version 2018, 1.2 Types of Performance Testing · Grafana k6 docs, Soak testing · Grafana k6 docs, Load test types (cheat sheet) · AWS DevOps Guidance, [QA.NT.2] Validate system reliability with performance testing · Microsoft Code-With Engineering Playbook, Performance Testing

Spike testing

A spike test hits the system with a sudden, very large burst of traffic, with little or no ramp-up, then checks that it survives and returns to normal. ISTQB defines it as "Testing to determine the ability of a system to recover from sudden bursts of peak loads and return to a steady state."

When to use it
Run it before events that cause sudden rushes; k6's examples are "ticket sales (Taylor Swift), product launches (PS5), broadcast ads (Super Bowl), process deadlines (tax declaration), and seasonal sales (Black Friday)". k6's cheat sheet: "Very high" load for "Short (a few minutes)", used "When the system prepares for seasonal events or receives frequent traffic peaks". The ramp-up is "very short or non-existent", the plateau is absent or brief because "real users generally do not stick around doing extra steps in these situations", and k6 advises "Focus on key processes in this test type."

Grafana k6Artillery

What it catches, what to watch out for, sources
What it catches
Whether the system "survives and performs under sudden and massive rushes of utilization" (k6), responds correctly to sudden bursts and returns "afterwards to a steady state" (ISTQB performance syllabus). k6 notes a spike "might include different processes than the previous test types, as spikes often aren't part of an average day in production."
Watch out for
One run is rarely enough: k6 says "the team must run a spikes test and tune the system several times" ("Run, tune, repeat") and "Backend monitoring is a must for successful outcomes of this test." Do not confuse it with a stress test: per k6, stress raises load above average for a sustained "Mid (5-60 minutes)" run, while a spike has a very short or no ramp-up and little or no plateau.

Sources: ISTQB Glossary: spike testing (glossary API record) · ISTQB Certified Tester Performance Testing Syllabus, Version 2018, 1.2 Types of Performance Testing · Grafana k6 docs, Spike testing · Grafana k6 docs, Load test types (cheat sheet) · Microsoft Code-With Engineering Playbook, Performance Testing

The tools

What teams use, by kind of test

Ordered by how much each is used, from the registries themselves: npm downloads in the last week, PyPI downloads in the last month, GitHub stars, refreshed every morning (numbers as of 2026-11-18). Usage is not quality: it tells you what many teams rely on, and where answers to your questions already exist. Commercial tools have no public download numbers, so they follow, described from their own pages.

17%

of API teams do contract testing, against 67% doing functional and integration testing and 57% performance testing.

Postman, 2025

UI and end-to-end

Playwright, Cypress and Selenium first. In the State of JavaScript 2025, Playwright's satisfaction reached 94% and its use rose 14 points in a year, against 57% satisfaction for Cypress and 24% for Selenium; in the 2024 survey, 3,674 respondents used Playwright at work, 3,603 Cypress and 1,130 Selenium. Selenium reaches further than JavaScript: its WebDriver bindings serve teams writing tests in Java, Python, C# and Ruby.

Playwright

TypeScript/JavaScript (Node.js), Python, .NET, Java

Fast and reliable end-to-end testing for modern web apps. Playwright is a framework for web testing and automation that tests Chromium, Firefox and WebKit with a single API.

  • Auto-wait, web-first assertions, a Trace Viewer, and a test generator that writes tests as you click ('npx playwright codegen'). The generator picks locators by role, text and test id first.
  • Test Agents (planner, generator, healer) plan, write and repair tests. Set them up for Claude Code with 'npx playwright init-agents --loop=claude'. There are also loops for VS Code, Codex and OpenCode.
  • Python users install 'pytest-playwright', the recommended pytest plugin, which uses the 'playwright' package.
Open source · Apache-2.0★ 100.5knpm 136.4M/wkPyPI 134.2M/mo

Sources: Playwright home · GitHub microsoft/playwright · Playwright test generator docs

Cypress

JavaScript, TypeScript

A JavaScript end-to-end and component testing framework that runs tests directly in the browser. The repo calls it fast, easy and reliable testing for anything that runs in a browser.

  • You write and debug tests in real time, inside the browser. Cypress ships official TypeScript type declarations.
  • Cypress Cloud, its hosted service, adds test parallelization, load balancing and spec prioritization.
  • cy.prompt is an AI-powered command that turns natural-language test steps into Cypress commands. It has been in beta since version 15.13.0, needs a Cypress Cloud account, runs only in Chromium-based browsers and supports E2E tests only.
Open source · MIT★ 51knpm 7.3M/wk

Sources: Cypress home · GitHub cypress-io/cypress · Cypress TypeScript support

Selenium

Java, Python, C#, Ruby, JavaScript

Selenium automates browsers. It is mainly for testing web applications, and WebDriver is a set of language bindings that drive a browser the way it is meant to be driven.

  • The project provides infrastructure for the W3C WebDriver specification, a platform- and language-neutral interface that works with all major browsers.
  • Selenium IDE is a record-and-playback add-on for Chrome, Firefox and Edge. Use it for quick bug reproduction scripts and for 'automation-aided exploratory testing'.
  • Selenium Grid spreads tests across several machines and many browser and OS combinations.
Open source · Apache-2.0★ 34.6knpm 2.1M/wkPyPI 30.5M/mo

Sources: Selenium home · GitHub SeleniumHQ/selenium

Testing Library

JavaScript (DOM, React, Angular, Vue, React Native)

A family of packages that helps you test UI components in a user-centric way. Its guiding principle: 'The more your tests resemble the way your software is used, the more confidence they can give you.'

  • The core is @testing-library/dom. There are packages for React, Angular, Vue and React Native, plus a Cypress plugin.
  • It works with Jest, with Mocha plus JSDOM, or in real browsers.
Open source · MIT★ 19.7knpm 98.7M/wk

Sources: Testing Library docs · GitHub testing-library/react-testing-library

Storybook

JavaScript (React, Vue, Angular, Svelte, Web Components and more)

The industry-standard workshop for building, documenting and testing UI components in isolation.

  • Component tests render a component in a real browser and simulate a user interacting with it. Interaction tests use a play function.
  • Visual tests compare snapshots of every story against baselines. An accessibility (a11y) addon runs automated checks.
  • The Vitest addon turns stories into real Vitest tests. Install with 'npm create storybook@latest'.
Open source · MIT★ 91.6knpm 37.1M/wk

Sources: Storybook testing docs · Storybook install docs · GitHub storybookjs/storybook

Puppeteer

JavaScript (Node.js)

A JavaScript library with a high-level API to control Chrome or Firefox over the DevTools Protocol or WebDriver BiDi.

  • Runs headless (no visible UI) by default.
  • The 'puppeteer' package downloads a browser when installed. 'puppeteer-core' is the library alone, with no browser download.
Open source · Apache-2.0★ 95.9knpm 15.7M/wk

Sources: Puppeteer docs · GitHub puppeteer/puppeteer

WebdriverIO

JavaScript (Node.js)

A next-gen browser and mobile automation test framework for Node.js. One framework and one API test browsers, native and hybrid mobile apps, and desktop apps.

  • Built on WebDriver and WebDriver BiDi, the W3C standards that browser vendors ship. Mobile testing goes through Appium.
  • Component testing for React, Vue, Svelte, Solid, Preact, Lit and Stencil, plus visual regression testing.
  • The CLI package is @wdio/cli. Adapters cover Mocha, Jasmine and Cucumber.
Open source · MIT★ 9.9knpm 3.5M/wk

Sources: WebdriverIO home · GitHub webdriverio/webdriverio

API

Postman says over 40 million developers use it. In its 2025 State of the API, 67% of respondents do functional and integration testing of APIs, 57% performance testing and 17% contract testing. Open-source clients keep collections as files in your repository (Bruno) or run on the web (Hoppscotch).

Postman

Desktop and web app, plus CLI; tests any HTTP API

Postman is an API platform for building and using APIs. Its testing features include request collections with assertions that catch regressions, monitors for live endpoints, and load simulation.

  • 'Trusted by over 40 million developers and 500,000 companies.' The about page adds that 98% of the Fortune 500 use it.
  • There is a Free plan at $0 per month. Paid plans are Solo, Team and Enterprise.
  • Newman, the open source command-line collection runner for Postman, is on npm as 'newman' (repo postmanlabs/newman, Apache-2.0).
Freemium

Sources: Postman home · About Postman · Postman pricing

Bruno

Desktop app and CLI; collections are plain-text .bru files

The Git-native API client. It is open source and offline-first, and it keeps API collections as plain-text files in your repo, next to the code they belong to.

  • No cloud sync and no account needed. Teams branch, diff, review and merge collections in Git.
  • The open source edition is $0. Pro ($6) and Ultimate ($11) per user per month, billed annually, add deeper Git, automation and enterprise features.
  • The repo calls it a 'lightweight alternative to Postman/Insomnia'.
Open source · MIT★ 48.4knpm 388.3k/wk

Sources: Bruno home · Bruno pricing · GitHub usebruno/bruno

Hoppscotch

Web, desktop and CLI

An open source API development ecosystem: offline, on-prem and cloud, on web, desktop and CLI. The repo calls it an open-source alternative to Postman and Insomnia.

  • Can be self-hosted (on-prem) or used in the cloud.
  • The CLI is on npm as @hoppscotch/cli.
Open source · MIT★ 80.8knpm 8.7k/wk

Sources: Hoppscotch home · GitHub hoppscotch/hoppscotch

Insomnia

Cross-platform desktop app plus Inso CLI; REST, GraphQL, gRPC, WebSockets, SSE

The open-source, cross-platform API client for GraphQL, REST, WebSockets, SSE and gRPC, made by Kong.

  • Store work locally, in a Git repository, or in the cloud with optional end-to-end encryption.
  • The free Essentials plan ($0) includes Git Sync for up to 3 users. Pro and Enterprise are paid.
  • The Inso CLI runs Insomnia features in the terminal and in CI/CD. It installs via Homebrew, direct download or Docker. The old npm package insomnia-inso is marked deprecated.
Open source · Apache-2.0★ 40.2k

Sources: Insomnia home · Insomnia pricing · GitHub Kong/insomnia

Karate

Karate feature files on the JVM; run via JUnit or a CLI

The open-source tool that combines API testing, mocks, performance testing, and UI automation into a single, unified framework.

  • Tests are written in the Karate language as feature files.
  • 'Execute Karate via JUnit, the CLI, or CI/CD, with full parallel control.'
Open source · MIT★ 9k

Sources: Karate docs · GitHub karatelabs/karate

REST Assured

Java

A Java DSL for easy testing of REST services. It brings the simplicity of dynamic languages like Ruby and Groovy into the Java domain.

  • Tests read as given / when / then, for example .when().get() and .then().statusCode().
  • Maven artifact: io.rest-assured:rest-assured.
Open source · Apache-2.0★ 7.2k

Sources: REST Assured home · GitHub rest-assured/rest-assured

Unit

Jest is the most used JavaScript testing tool, 74.4% have used it, while Vitest grew fastest and leads satisfaction at 97% (State of JavaScript 2025). In Python, 53% use pytest (PSF and JetBrains, 2024); on the JVM, JUnit is the standard.

Jest

JavaScript, TypeScript (Node, React, Angular, Vue, Babel projects)

Jest is a delightful JavaScript testing framework with a focus on simplicity.

  • Aims to work with zero config on most JavaScript projects.
  • Snapshot tests, built-in coverage with the --coverage flag, and easy mocking.
  • Runs each test file in its own process, in parallel, for speed.
Open source · MIT★ 45.5knpm 61.5M/wk

Sources: Jest home · GitHub jestjs/jest

Vitest

JavaScript, TypeScript, JSX

A Vite-native testing framework. It's fast!

  • Jest compatible: expect, snapshots and coverage make migration straightforward.
  • ESM, TypeScript and JSX work out of the box. Smart watch mode reruns only the tests related to a change.
  • Browser Mode runs tests natively in a browser, with Playwright or WebdriverIO as providers.
Open source · MIT★ 17.4knpm 190M/wk

Sources: Vitest home · Vitest Browser Mode · GitHub vitest-dev/vitest

pytest

Python (3.10+ or PyPy 3)

The pytest framework makes it easy to write small, readable tests, and can scale to support complex functional testing for applications and libraries.

  • Uses plain assert statements and shows detailed info when they fail.
  • Modular fixtures, auto-discovery of tests, and it can run unittest suites out of the box.
  • 'Rich plugin architecture, with over 1300+ external plugins'.
Open source · MIT★ 14.7kPyPI 977.1M/mo

Sources: pytest docs · GitHub pytest-dev/pytest

JUnit

Java, Kotlin and other JVM languages

The programmer-friendly testing framework for Java and the JVM. JUnit 6 is the current generation.

  • JUnit 6.0.0 was released on September 30, 2025. The latest release is 6.1.3 (August 7, 2026).
  • JUnit 6 needs Java 17 (and Kotlin 2.1) or above.
  • Maven artifact: org.junit.jupiter:junit-jupiter.
Open source · EPL-2.0★ 7.1k

Sources: JUnit home · JUnit release notes · GitHub junit-team/junit-framework

Mocha

JavaScript (Node.js and browser)

Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing straightforward and fun.

  • Runs tests one after another (serially), so reports are flexible and accurate and uncaught exceptions map to the right test.
  • The repo calls it a 'Classic, reliable, trusted test framework'.
Open source · MIT★ 22.9knpm 14.5M/wk

Sources: Mocha home · GitHub mochajs/mocha

xUnit.net

C#, F#, Visual Basic (.NET)

xUnit.net is a free, open source, community-focused unit testing tool for C#, F# and Visual Basic.

  • Part of the .NET Foundation.
  • xUnit.net v3 targets .NET 8.0 or later and .NET Framework 4.7.2 or later. The NuGet package is xunit.v3.
Open source · Apache-2.0★ 4.6k

Sources: xUnit.net home · GitHub xunit/xunit

NUnit

C#, F#, VB.NET (all .NET languages)

NUnit is a unit-testing framework for all .NET languages. It runs on macOS, Linux and Windows.

  • Originally ported from JUnit, then fully rewritten for version 3.
  • A .NET Foundation member project. The NuGet package is NUnit.
Open source · MIT★ 2.6k

Sources: NUnit home · GitHub nunit/nunit

Integration

Mock Service Worker (MSW)

JavaScript (browser and Node.js)

An API mocking library for JavaScript. You write client-agnostic mocks once and reuse them across frameworks, tools and environments.

  • Intercepts requests in the browser with the Service Worker API and in Node.js at the socket level.
  • Mocks REST, GraphQL, Server-Sent Events and WebSocket APIs.
Open source · MIT★ 18.4knpm 34.9M/wk

Sources: MSW home · GitHub mswjs/msw

Testcontainers

Java, Go, .NET, Node.js, Python, Rust, Ruby, PHP, Haskell, Clojure, Elixir, Scala

Unit tests with real dependencies. An open source library that provides throwaway, lightweight instances of databases, message brokers, web browsers, or just about anything that can run in a Docker container.

  • Needs Docker: each dependency runs in a throwaway container during the test.
  • The Java and Node.js libraries are MIT licensed. The Python library (testcontainers/testcontainers-python) is Apache-2.0.
Open source · MIT★ 8.8knpm 12.3M/wkPyPI 39.7M/mo

Sources: Testcontainers home · GitHub testcontainers/testcontainers-java · GitHub testcontainers/testcontainers-node

Contract

Only 17% of API teams do contract testing (Postman, 2025), the least used test type in that survey.

Pact

More than 10 languages, including JavaScript, JVM, .NET, Go, Python, Ruby, Rust, PHP, Swift

Pact is a code-first tool for testing HTTP and message integrations using contract tests.

  • Consumer-driven: the contract is generated while the consumer's automated tests run, so only interactions the consumer actually uses get tested.
  • Pact defines contract testing as checking each application in isolation to make sure the messages it sends or receives match a shared 'contract'.
  • PactFlow is the commercial product built around Pact.
Open source · MIT★ 1.8knpm 1.1M/wkPyPI 1.3M/mo

Sources: Pact docs · Pact implementations list · GitHub pact-foundation/pact-js

Performance: load, stress, soak

57% of API teams run performance tests (Postman, 2025). These are the open-source tools most of them build on.

Locust

Python

An open source load testing tool. Define user behaviour with Python code, and swarm your system with millions of simultaneous users.

  • Plain Python code: 'No need for clunky UIs or bloated XML.'
  • Runs load tests distributed over multiple machines.
Open source · MIT★ 28.3kPyPI 11.4M/mo

Sources: Locust home · GitHub locustio/locust

Artillery

JavaScript, TypeScript (Node.js)

Load testing, scalable Playwright end-to-end testing, and production monitoring in one toolkit.

  • Reuses existing Playwright tests for browser-based load testing.
  • Distributed load testing is built in and runs on AWS or Azure.
Open source · MPL-2.0★ 9.1knpm 365.7k/wk

Sources: Artillery home · Artillery load testing docs · GitHub artilleryio/artillery

Gatling

Java, Scala, Kotlin, JavaScript, TypeScript

Modern load testing as code. There is an open source Community Edition and a paid Enterprise Edition.

  • Tests are code written in Java, Scala, Kotlin, JavaScript or TypeScript.
  • The JavaScript/TypeScript SDK is on npm (@gatling.io/cli, repo gatling/gatling-js).
Open source · Apache-2.0★ 7knpm 66.3k/wk

Sources: Gatling home · GitHub gatling/gatling · GitHub gatling/gatling-js

Grafana k6

Tests in JavaScript; engine written in Go

An open-source, developer-friendly and extensible performance testing tool that helps you catch performance issues early and proactively improve reliability.

  • The k6 docs define the test types: stress tests 'assess how a system performs at its limits when load exceeds the expected average'. Soak tests 'assess the reliability and performance of your system over extended periods'. The other types are smoke, average-load, spike and breakpoint.
  • Use cases also include browser performance testing, synthetic monitoring, and chaos and resilience testing.
  • Grafana Cloud k6 is the hosted SaaS version.
Open source · AGPL-3.0★ 32.3k

Sources: k6 docs · k6 test types · k6 site

Apache JMeter

Java application (JVM)

Open source software, a 100% pure Java application designed to load test functional behavior and measure performance.

  • 'JMeter is not a browser, it works at protocol level.' It does not run the JavaScript in HTML pages.
  • Protocols include HTTP/HTTPS, SOAP/REST web services, FTP, JDBC databases, LDAP, JMS, mail, TCP and Java objects.
Open source · Apache-2.0★ 9.6k

Sources: Apache JMeter home · GitHub apache/jmeter

Mobile

Appium

Clients in JavaScript, Python, Java, Ruby, .NET

An open-source project and ecosystem of related software designed to make UI automation possible on many app platforms. It is built on the W3C WebDriver protocol.

  • Platform drivers cover iOS, Android, Tizen, browsers, macOS, Windows, and TVs such as Roku, tvOS, Android TV and Samsung.
  • Install the server with 'npm i -g appium'. The Python client is 'pip install Appium-Python-Client'.
Open source · Apache-2.0★ 22.2knpm 1.4M/wkPyPI 6.2M/mo

Sources: Appium docs · GitHub appium/appium · GitHub appium/python-client

Maestro

YAML flows; Android, iOS, web (React Native, Flutter, native, hybrid)

Painless end-to-end UI automation for mobile and web. Its site describes it as 'mobile UI testing at agentic velocity' and stresses that it is open source.

  • Tests are human-readable YAML flows built from commands like launchApp, tapOn and assertVisible.
  • Runs on emulators, simulators and browsers. Physical iOS devices are not yet supported.
Open source · Apache-2.0★ 16.4k

Sources: Maestro home · GitHub mobile-dev-inc/Maestro

Manual test management

Kiwi TCMS

Self-hosted web app (Docker)

The leading open source test management system. Use its dashboard to see pending work, run tests, mark results and report bugs.

  • Container first: runs as a Docker container. The site claims 'Over 2 million downloads'.
  • Plugins for popular testing frameworks collect automation results.
Open source · GPL-2.0★ 1.3k

Sources: Kiwi TCMS home · GitHub kiwitcms/Kiwi

Qase

Web app with API and MCP server

Qase brings your manual runs, automated tests and CI results into one system of record.

  • The Free plan ('$0/user', 'free forever') allows up to 2 projects, 500 MB of storage and 30 days of test history.
  • The free plan also includes an MCP server.
Freemium

Sources: Qase home · Qase pricing

TestRail

Web app with API and CLI; integrates with Jira and Azure DevOps

Test management software that brings test cases, defect details, and manual and automated test results together, so you get one view of test execution, coverage and release risk.

  • 'Join over 10,000 QA teams using TestRail'
  • Fits scripted, exploratory, manual or automated testing. Has two-way Jira integration, an Azure DevOps app, an API and a CLI.
  • Commercial, with a free trial.
Commercial

Sources: TestRail home

Xray

Jira app

AI-powered test management, built inside Jira. It brings the creation, execution and reporting of manual and automated tests into one platform.

  • The Xray Exploratory App records screenshots, video, audio and notes and runs timed test charters for exploratory sessions.
  • '137 of the Fortune 500' use Xray, according to its about page.
  • AI features can write manual or BDD test cases and produce automation scripts from manual tests (Advanced and Enterprise tiers).
Commercial

Sources: Xray home · Xray about · Xray Exploratory App

Exploratory

Bug Magnet

Chrome and Firefox extension

Exploratory testing made easy. Bug Magnet adds common problematic values and edge cases to the right-click menu of editable fields, so they are always at hand during exploratory sessions.

  • Free and open source, made by Gojko Adzic. Get it from the Chrome Web Store or Firefox Add-ons.
  • 'reminds you about useful exploratory testing heuristics'
Open source · MIT★ 283

Sources: Bug Magnet home · GitHub gojko/bugmagnet

Bug Magnet AI assistant

Slash command for AI coding agents (Claude Code, Cursor)

An AI assistant command for comprehensive test coverage and bug discovery. It brings Bug Magnet's exploratory testing heuristics to AI coding agents.

  • Install it by copying commands/bugmagnet.md into .claude/commands (Claude Code) or .cursor/commands (Cursor). Run it with /bugmagnet and a file path, for example /bugmagnet src/auth/file.js.
  • It helps write characterization tests, find coverage gaps, explore undocumented behavior and apply testing heuristics.
  • A small project (about 100 stars) by the author of Bug Magnet.
Open source · MIT★ 108

Sources: GitHub gojko/bugmagnet-ai-assistant · Bug Magnet home (links the AI version)