Skip to content
Snippets Groups Projects
  • Nick Terrell's avatar
    [regression] Add initial regression test framework · 4aaa36f7
    Nick Terrell authored
    The regression tests run nightly or on the `regression`
    branch for convenience. The results get uploaded as the
    artifacts of the job. If they change, check the diff
    printed in the job. If all is well, download the new
    results and commit them to the repo.
    
    This code will only run on a UNIX like platform. It
    could be made to run on Windows, but I don't think that
    it is necessary. It also uses C99.
    
    * data: This module defines the data to run tests on.
      It downloads data from a URL into a cache directory,
      checks it against a checksum, and unpacks it. It also
      provides helpers for accessing the data.
    * config: This module defines the configs to run tests
      with. A config is a set of API parameters and a set of
      CLI flags.
    * result: This module is a helper for method that defines
      the result type.
    * method: This module defines the compression methods
      to test. It is what runs the regression test using the
      data and the config. It reports the total compressed
      size, or an error/skip.
    * test: This is the test binary that runs the tests for
      every (data, config, method) tuple, and prints the
      results to the output file and stderr.
    * results.csv: The results that the current commit is
      expected to produce.
    4aaa36f7