PHPunit testing instance of same object under different states.

[NotionCommotion]

I wish to test an API whether users with various permissions can perform actions on resources with various access policies.

I first envisioned using a provider such as the following, however, quickly found it had at least one major flaw and I am certain more.

/**
 * @dataProvider userResourceProvider
 */
public function testUserResource(User $user, object $resource, int $expectedHttpStatusCode):void
{
    $client = $this->createClient($user);
    $client->makeSomeHttpRequest($resource);
    $this->assertStatusCode($expectedHttpStatusCode);
}

public function userResourceProvider(): \Generator
{
    $user = new User();
    $this->updateUserToInitialState($user);
    $this->updateDatabase($user);
    foreach($this->resourceClasses as $resourceClass) {
        $resource = new $resourceClass();
        $this->updateResourceToInitialState($resource);
        $this->updateDatabase($resource);
        foreach($this->getResourcePolicy() as $policy) {
            $resource->setPolicy($policy);
            $this->updateDatabase($resource);
            foreach($this->getUserPermission() as $permission) {
                $user->setPermission($permission);
                $this->updateDatabase($user);
                yield [$user, $resource, $this->getExpectedHttpStatusCode($user, $resource)];                   
            }
        }
    }
}

Instead of passing the $user and the $resource in their current states to the test method, it will their last state after the second and third foreach loop which I confirmed using the following.

public function someProvider(): \Generator
{
    $obj = new SomeObject();
    foreach([4,3,6,1] as $i) {
        $obj->setIndex($i);
        yield [$i, $obj];                   
    }
}

 

Not only will the state of the objects passed to the test be incorrect, I am almost sure the state of the database storing those objects be incorrect as (I think) all database updates be perform before executing the test methods.

Questions:

1. Could/should any portions of modifying the state of the objects and the state of the database be updated in the provider (for instance, updateUserToInitialState and updateResourceToInitialState), or should all be repeated each time within the test method?
2. What are the implications of database transactions when performing tests?
3. Any other common gochas when performing tests on objects of different state?

[kicken]

I don't know much about PHPUnit or unit testing in general (only recently started trying to do it) but the documentation for PHPUnit's data providers has this note:

Quote

All data providers are executed before both the call to the setUpBeforeClass() static method and the first call to the setUp() method. Because of that you can’t access any variables you create there from within a data provider. This is required in order for PHPUnit to be able to compute the total number of tests.

 

To me, it sounds like that means your data provider will be fully iterated before the unit test is run, which would match the behavior you seem to be having.

It's my understanding of unit tests that you generally should avoid having some state / external dependency.  You should instead create mock objects as necessary to return the data you want. 

[requinix]

Yielding the same object multiple times in a generator creates the sorts of problems you get like with SplFileInfo; I'd give a link to what I mean but I can't find one, so I'll summarize by saying that iteration on it (when a directory) yields the same object over and over, which means you can't do things like use iterator_to_array or even hold onto the iterated value for long. It's somewhat similar to the problem of foreaching with a reference.

I recommend cloning the $obj each time - it's a test so who cares about performance and memory usage?

public function someProvider(): \Generator
{
    $obj = new SomeObject();
    foreach([4,3,6,1] as $i) {
        $obj = clone $obj;
        $obj->setIndex($i);
        yield [$i, $obj];                   
    }
}

(Or another solution where $obj changes. Depends on what SomeObject is and how much you can alter it for use as a provider.)

[NotionCommotion]

20 hours ago, kicken said:

To me, it sounds like that means your data provider will be fully iterated before the unit test is run, which would match the behavior you seem to be having.

It's my understanding of unit tests that you generally should avoid having some state / external dependency.  You should instead create mock objects as necessary to return the data you want. 

Similarly, if one executes the same provider multiple times, they will receive the same results.

I have also read so and recognize that external dependencies add additional dimensions, however, see some benefits of doing so and haven't gotten my head around mock objects.

15 hours ago, requinix said:

Yielding the same object multiple times in a generator creates the sorts of problems...

I recommend cloning the $obj each time - it's a test so who cares about performance and memory usage?

I see your point about issues with yielding the same object, and was thinking of cloning as you suggest.

Assuming all agree that I should not attempt to update the database to a given state within the provider but right before actually performing each test.