17 Feb 2015

Code coverage for the Laravel Behat Extension

Categories: Laravel, Mink, Behat, coverage

The Laravel Behat Extension is a great and minimal way to test your Laravel 5 application using Behat 3. It doesn't require an external server and does not connect through a driver like Goutte. Besides it being faster and minimal this approach brings another benefit. Because the Laravel Behat Extension loads the application container in the same process as the Behat tests we can measure our code coverage.

Laravel Behat Extension

Jeffery Way from Laracasts has developed a simple but powerful extension for Behat 3 to execute Behat tests on a Laravel 5 app without the use of an external server.

He has made some tutorial video’s on the extension, I recommend checking them out. They are hosted on Laracasts, which is a paid service but definitely worth it.

Code Coverage

Once you have setup your Laravel 5 application with the Laravel Behat Extension you can include our simple CodeCoverage trait. Place it in your features\bootstrap directory and add use CodeCoverage; to your FeatureContext. Make sure the directory whitelist and blacklist are set correctly.

<?php
trait CodeCoverage {

    /**
     * @var PHP_CodeCoverage
     */
    protected static $coverage;

    /**
     * @BeforeSuite
     */
    public static function setupCoverage()
    {
        if (self::isCodeCoverageEnabled()) {
            $filter = new PHP_CodeCoverage_Filter();
            $filter->addDirectoryToBlacklist(__DIR__ . '/../../vendor');
            $filter->addDirectoryToWhitelist(__DIR__ . '/../../app');

            self::$coverage = new PHP_CodeCoverage(null, $filter);

            self::$coverage->start('Behat Test');
        }
    }

    /**
     * @AfterSuite
     */
    public static function writeCoverageFiles()
    {
        if (self::isCodeCoverageEnabled()) {
            self::$coverage->stop();

            // Use Clover to analyze your coverage with a tool
            if (getenv('CODE_COVERAGE_CLOVER')) {
                $writer = new PHP_CodeCoverage_Report_Clover;
                $writer->process(
                    self::$coverage,
                    __DIR__ . '/../../clover-behat.xml'
                 );
            }

            // Use HTML to take a look at your test coverage locally
            if (getenv('CODE_COVERAGE_HTML')) {
                $writer = new PHP_CodeCoverage_Report_HTML;
                $writer->process(
                    self::$coverage,
                    __DIR__ . '/../../coverage-behat'
                );
            }

            // use the text report to append the coverage results
            // to stdout at the end of the test run
            // so you can view them from the command line
            $writer = new PHP_CodeCoverage_Report_Text(75, 90, false, false);
            fwrite(STDOUT, $writer->process(self::$coverage, true));
        }
    }

    private static function isCodeCoverageEnabled()
    {
        return
            getenv('CODE_COVERAGE') ||
            getenv('CODE_COVERAGE_HTML') ||
            getenv('CODE_COVERAGE_CLOVER');
    }
}

View the file as a gist.

The trait uses PHPUnit’s Code Coverage extension, which in turn requires xdebug. Add the following lines to your composer.json to include the code coverage extension.

{
    "require-dev": {
        "phpunit/php-code-coverage": "~2.0"
    }
}

Scanning for code coverage makes your tests run much slower. So because you don’t want to run code coverage every time you run Behat we check for certain environment variables. In order to run Behat with code coverage use: $ CODE_COVERAGE=on ./vendor/bin/behat. Or if you want a generated HTML page use: CODE_COVERAGE_HTML=on ./vendor/bin/behat.

Integrate with CI

At Label305 we have integrated this code coverage approach into a private project which runs CI builds on Travis and do not use any kind of special code coverage service. We read the results right from the logs.

However you could also use a service like Coveralls. If you want to integrate with coveralls you will need a clover document. Run $ CODE_COVERAGE_CLOVER=on ./vendor/bin/behat to run the behat tests and generate the clover-behat.xml document which you can send to coveralls or a similar service.

Written by: Thijs Scheepers