You are working on a PHP application or a package. You have decided that you want to use:

• infection/infection for running mutation tests
• phpbench/phpbench for running performance tests
• phpunit/phpunit for running unit, integration, functional, and end-to-end tests

Perhaps you are considering other design and testing frameworks as well, for example:

• codeception/codeception
• behat/behat
• pestphp/pest
• phpspec/phpspec

What are your drivers and options for organizing test code?

Systems under test

For this article, let's assume you have relatively simple production code organized in a directory structure as follows (listing generated with the tree command):

.
└── src
    ├── Example.php
    ├── ValueCanNotBeBlank.php
    └── ValueCanNotBeEmpty.php

Test code

Let's also clarify what we understand as test code.

First, you have test files.

When you use a class-based test case framework such as phpunit/phpunit, these test files declare tests in classes. Or, when you use a file-based test case framework such as pestphp/pest, these test files declare tests in functions.

Second, you have supporting test files.

You may extract abstract test cases, helper traits, data providers, and test doubles to share functionality across tests. You may also have fixtures for use in tests. You can organize all of these in classes, functions, or files.

Third, you have configuration files.

When you use a test framework that works with configuration files, you will have at least one configuration per test framework.

When you use composer, you have a composer.json file in which you not only configure your project's dependencies but also configure and document namespaces for autoloading production and test code.

When you use git and are working on a package, you may have a .gitattributes configuration to prevent the distribution of code that users of your package do not need.

Drivers for organizing test code

Let's also talk about potential drivers for organizing test code.

First, the mapping of test classes or files to production code drives the organization of your test code.

You could have a single test class or test file that contains tests for the entire application or package.

As you add more tests, you may find it challenging to decide where to add those tests - or you may have problems finding a specific test. Perhaps you conclude that it is better to have one test class or file per system under test, with a consistent naming (using prefixes, infixes, or suffixes) that makes it easier to find a test class or file.

Second, the kinds of test you intend to write drive the organization of your test code.

When you realize that you have different kinds of tests, for example, unit, integration, functional, or end-to-end tests, you could move different kinds of tests into separate files - each test class or file containing only a specific kind of test.

You could also group these tests by kind in separate directories and namespaces.

Third, the requirement to configure and bootstrap different kinds of tests drives the organization of your test code.

When you have different kinds of tests, you may also realize that these tests require completely different test bootstrapping. You may have to set up a database, run database migrations, and seed the database for integration, functional, and end-to-end tests, but certainly not for unit tests.

A single configuration file will only get you so far. You could use dedicated configuration files for different kinds of tests instead.

Forth, the number of testing frameworks drives the organization of your test code.

You could place all test classes and test files in a single directory and leave it up to your testing frameworks to decide whether they should consider a specific test class or file as a test case.

You could also group tests by testing framework. A grouping like that could considerably simplify the configuration, maintenance, and speed up running of tests.

Options for organizing test code

As far as I can tell, you have the following options for organizing test code:

• no tests
• in the same file
• in the same directory
• in a subdirectory
• in subdirectories
• in subdirectories with separate configuration files
• in a separate directory
• in separate directories
• in separate directories with separate configuration files

No tests

By far, this is the simplest solution for organizing test code.

You have fewer lines of code, dependencies, and configuration files. You probably also have more bugs, downtimes, and sleepless nights.

If you are comfortable with that, you can stop reading now.

Declaring test code in the same file

You can declare test code in the same file as production code, but unfortunately, I have not been able to configure phpbench/phpbench or phpunit/phpunit to find benchmarks or tests in the production files.

Unless you have found a way to make this work, declaring test code in the same file is not an option. If you have found a way, I would like to hear about it.

Declaring test code in the same directory

You can declare test code in the same directory as production code.

.
├── src
│   ├── Example.php
│   ├── ExampleBench.php
│   ├── ExampleTest.php
│   ├── Helper.php
│   ├── ValueCanNotBeBlank.php
│   ├── ValueCanNotBeBlankTest.php
│   ├── ValueCanNotBeEmpty.php
│   └── ValueCanNotBeEmptyTest.php
├── phpbench.json
└── phpunit.xml

In my opinion, this approach has the following disadvantages:

1. It is hard to tell whether the code in the src/ directory is production or test code. How can you know whether a piece of code is only intended for use in test environments, for example, a test helper or a value object? How can you prevent developers from using test code in production?
2. If you want to run unit, integration, functional, and end-to-end tests, how can you distinguish between these different kinds of tests?
3. If you want to run unit, integration, functional, and end-to-end tests, how can you ensure appropriate test bootstrapping for each test? It is not necessary to bootstrap a database and run migrations for running unit tests, but it might be for integration, functional, and end-to-end tests.
4. Where are you going to place abstract test cases, helper traits, data providers, test doubles, and test fixtures if you need any?
5. You must excessively configure exclusions in composer.json to exclude test code from being autoloadable in production systems.
6. You need to configure exclusions for test code in infection.json to prevent infection/infection from mutating test code and reporting false negatives from mutated test code.
7. You need to configure exclusions for test code in phpunit.xml to prevent phpunit/phpunit from reporting test code as used in tests and not being covered by tests.
8. If you are working on a package, you need to excessively configure exclusions for test code in .gitattributes to prevent the unnecessary distribution of test code to users of your package.
9. If you are working on a package, how are you dealing with users who expect a test/ or tests/ directory in the root of your project and skip the inspection and consideration of your package entirely because they suspect you have not written any tests?

If you are interested in what others say about declaring test code in the same directory, check out the replies to this tweet by Brent Roose:

An out of the box idea: why would/wouldn't you store your tests alongside your source code, instead of keeping them completely separated? I can come up with a few pros and cons, but would like to hear your opinion.

— Brent (@brendt_gd) September 15, 2021

Declaring test code in a subdirectory

You can declare test code in a subdirectory of your production code.

.
├── src
│   ├── Test
│   │   ├── ExampleBench.php
│   │   ├── ExampleTest.php
│   │   ├── Helper.php
│   │   ├── ValueCanNotBeBlankTest.php
│   │   ├── ValueCanNotBeEmpty.php
│   │   └── ValueCanNotBeEmptyTest.php
│   ├── Example.php
│   ├── ValueCanNotBeBlank.php
│   └── ValueCanNotBeEmpty.php
├── phpbench.json
└── phpunit.xml

You will often see this organization of test code in repositories split from mono-repositories. For an example, inspect symfony/var-dumper which is a split from symfony/symfony.

In my opinion, this approach has the following advantages over declaring test code in the same directory

1. It is easier to tell whether code is production or test code.
2. The configuration of exclusions in composer.json to exclude test code from being autoloadable in production systems has become simpler.
3. If you are working on a package, the configuration of exclusions for test code in .gitattributes to prevent the unnecessary distribution of test code to users of your package has become simpler.

In my opinion, this approach still has the following disadvantages:

1. You still are not distinguishing between unit, integration, functional, and end-to-end tests.
2. You are still not ensuring appropriate test bootstrapping for unit, integration, functional, and end-to-end tests.
3. You still need to configure exclusions for test code in infection.json to prevent infection/infection from mutating test code and reporting false negatives from mutated test code.
4. You still need to configure exclusions for test code in phpunit.xml to prevent phpunit/phpunit from reporting test code as used in tests and from reporting test code as not being covered by tests.
5. You are still placing test fixtures and utilities along with test code.
6. If you are working on a package, potential users may still skip the inspection and consideration of your package entirely because they can not see a test/ or tests/ directory in the root of your project and suspect you have not written any tests.

Declaring test code in subdirectories

You can declare test code in subdirectories of your production code.

.
├── src
│   ├── Test
│   │   ├── Performance
│   │   │   └── ExampleBench.php
│   │   ├── Unit
│   │   │   ├── ExampleTest.php
│   │   │   ├── ValueCanNotBeBlankTest.php
│   │   │   ├── ValueCanNotBeEmpty.php
│   │   │   └── ValueCanNotBeEmptyTest.php
│   │   └── Util
│   │       └── Helper.php
│   ├── Example.php
│   ├── ValueCanNotBeBlank.php
│   └── ValueCanNotBeEmpty.php
├── phpbench.json
└── phpunit.xml

In my opinion, this approach has the following advantages over declaring test code in a subdirectory:

1. You distinguish between unit, integration, functional, and end-to-end tests by grouping them in subdirectories and corresponding namespaces. Developers will understand where to place performance and unit tests more quickly.
2. If you need test fixtures or utilities, they can all go into subdirectories and corresponding namespaces.

In my opinion, this approach still has the following disadvantages:

1. You are still not ensuring appropriate test bootstrapping for unit, integration, functional, and end-to-end tests.
2. You still need to configure exclusions for test code in infection.json to prevent infection/infection from mutating test code and reporting false negatives from mutated test code.
3. You still need to configure exclusions for test code in phpunit.xml to prevent phpunit/phpunit from reporting test code as used in tests and from reporting test code as not being covered by tests.
4. If you are working on a package, potential users may still skip the inspection and consideration of your package entirely because they can not see a test/ or tests/ directory in the root of your project and suspect you have not written any tests.

Declaring test code in subdirectories with separate configuration files

You can declare test code in subdirectories of your production code with separate configuration files.

.
└── src
    ├── Test
    │   ├── Performance
    │   │   ├── ExampleBench.php
    │   │   └── phpbench.json
    │   ├── Unit
    │   │   ├── ExampleTest.php
    │   │   ├── phpunit.xml
    │   │   ├── ValueCanNotBeBlankTest.php
    │   │   ├── ValueCanNotBeEmpty.php
    │   │   └── ValueCanNotBeEmptyTest.php
    │   └── Util
    │       └── Helper.php
    ├── Example.php
    ├── ValueCanNotBeBlank.php
    └── ValueCanNotBeEmpty.php

In my opinion, this approach has the following advantages over declaring test code in subdirectories:

1. You ensure appropriate test bootstrapping for unit, integration, functional, and end-to-end tests by extracting dedicated configuration files.
2. If you are working on a package, the configuration of exclusions for test code in .gitattributes to prevent the unnecessary distribution of test code to users of your package has become even simpler.

In my opinion, this approach still has the following disadvantages:

1. You still need to configure exclusions for test code in infection.json to prevent infection/infection from mutating test code and reporting false negatives from mutated test code.
2. You still need to configure exclusions for test code in phpunit.xml to prevent phpunit/phpunit from reporting test code as used in tests and from reporting test code as not being covered by tests, and that configuration has become even worse.
3. If you are working on a package, potential users may still skip the inspection and consideration of your package entirely because they can not see a test/ or tests/ directory in the root of your project and suspect you have not written any tests.

Declaring test code in a separate directory

You can declare test code in a directory entirely separate from production code.

.
├── src
│   ├── Example.php
│   ├── ValueCanNotBeBlank.php
│   └── ValueCanNotBeEmpty.php
├── test
│   ├── ExampleBench.php
│   ├── ExampleTest.php
│   ├── Helper.php
│   ├── ValueCanNotBeBlankTest.php
│   ├── ValueCanNotBeEmpty.php
│   └── ValueCanNotBeEmptyTest.php
├── phpbench.json
└── phpunit.xml

You can find documentation for this approach in the official PHPUnit documentation.

In my opinion, this approach has the following advantages over declaring test code in the same directory:

1. It is trivial to tell whether code is production or test code.
2. You do not need to configure exclusions in composer.json to exclude test code from being autoloadable in production systems. Instead, you configure an autoloader for test code in the autoload-dev section.
3. You do not need to configure exclusions for test code in infection.json for infection/infection.
4. You do not need to configure exclusions for test code in phpunit.xml for phpunit/phpunit.
5. If you are working on a package, you only need to configure exclusions for test configuration files and the test/ directory in .gitattributes to prevent the unnecessary distribution of test code to users of your package.
6. If you are working on a package, potential users will not skip the inspection and consideration of your package entirely as they can see a test/ or tests/ directory in the root of your project.

In my opinion, this approach still has the following disadvantages:

1. You still are not distinguishing between unit, integration, functional, and end-to-end tests.
2. You are still not ensuring appropriate test bootstrapping for unit, integration, functional, and end-to-end tests.
3. You are still placing test fixtures and utilities along with test code.

Declaring test code in separate directories

You can declare test code in subdirectories of a directory entirely separate from production code.

.
├── src
│   ├── Example.php
│   ├── ValueCanNotBeBlank.php
│   └── ValueCanNotBeEmpty.php
├── test
│   ├── Performance
│   │   └── ExampleBench.php
│   ├── Unit
│   │   ├── ExampleTest.php
│   │   ├── ValueCanNotBeBlankTest.php
│   │   ├── ValueCanNotBeEmpty.php
│   │   └── ValueCanNotBeEmptyTest.php
│   └── Util
│       └── Helper.php
├── phpbench.json
└── phpunit.xml

In my opinion, this approach has the following advantages over declaring test code in a subdirectory:

1. You distinguish between unit, integration, functional, and end-to-end tests by grouping them in subdirectories and corresponding namespaces. Developers will understand where to place performance and unit tests more quickly.
2. If you need test fixtures or utilities, they can all go into subdirectories and corresponding namespaces.
3. If you are working on a package, potential users will not skip the inspection and consideration of your package entirely, as they can see a test/ or tests/ directory in the root of your project.

In my opinion, this approach still has the following disadvantages:

1. You are still not ensuring appropriate test bootstrapping for unit, integration, functional, and end-to-end tests.

Declaring test code in separate directories with separate configuration files

You can declare test code in subdirectories of a directory entirely separate from production code with separate configuration files.

.
├── src
│   ├── Example.php
│   ├── ValueCanNotBeBlank.php
│   └── ValueCanNotBeEmpty.php
└── test
    ├── Performance
    │   ├── ExampleBench.php
    │   └── phpbench.json
    ├── Unit
    │   ├── ExampleTest.php
    │   ├── phpunit.xml
    │   ├── ValueCanNotBeBlankTest.php
    │   ├── ValueCanNotBeEmpty.php
    │   └── ValueCanNotBeEmptyTest.php
    └── Util
        └── Helper.php

In my opinion, this approach has the following advantages over declaring test code in separate directories:

1. You ensure appropriate test bootstrapping for unit, integration, functional, and end-to-end tests.

In my opinion, this approach has the following disadvantages:

1. You may need to merge code coverage reports if you use different configuration files for unit, integration, function, and end-to-end tests and want to collect code coverage from multiple test runs.

Recommendation

I prefer to declare test code in separate directories with separate configuration files, but I am open to changing my mind and adapting to the circumstances.

Do what works best for you!