- Dec 01, 2018
-
-
Nick Terrell authored
Dictionaries are prebuilt and saved as part of the data object. The config decides whether or not to use the dictionary if it is available. Configs that require dictionaries are only run with data that have dictionaries. The method will skip configs that are irrelevant, so for example ZSTD_compress() will skip configs with dictionaries. I've also trimmed the silesia source to 1MB per file (12 MB total), and added 500 samples from the github data set with a dictionary. I've intentionally added an extra line to the `results.csv` to make the nightly build fail, so that we can see how CircleCI reports it. Full list of changes: * Add pre-built dictionaries to the data. * Add `use_dictionary` and `no_pledged_src_size` flags to the config. * Add a config using a dictionary for every level. * Add a config that specifies no pledged source size. * Support dictionaries and streaming in the `zstdcli` method. * Add a context-reuse method using `ZSTD_compressCCtx()`. * Clean up the formatting of the `results.csv` file to align columns. * Add `--data`, `--config`, and `--method` flags to constrain each to a particular value. This is useful for debugging a failure or debugging a particular config/method/data.
-
- Nov 30, 2018
-
-
Yann Collet authored
Small changes for buck build
-
Mitchell Grenier authored
-
Yann Collet authored
[regression] Add initial regression test framework
-
- Nov 29, 2018
-
-
Nick Terrell authored
The regression tests run nightly or on the `regression` branch for convenience. The results get uploaded as the artifacts of the job. If they change, check the diff printed in the job. If all is well, download the new results and commit them to the repo. This code will only run on a UNIX like platform. It could be made to run on Windows, but I don't think that it is necessary. It also uses C99. * data: This module defines the data to run tests on. It downloads data from a URL into a cache directory, checks it against a checksum, and unpacks it. It also provides helpers for accessing the data. * config: This module defines the configs to run tests with. A config is a set of API parameters and a set of CLI flags. * result: This module is a helper for method that defines the result type. * method: This module defines the compression methods to test. It is what runs the regression test using the data and the config. It reports the total compressed size, or an error/skip. * test: This is the test binary that runs the tests for every (data, config, method) tuple, and prints the results to the output file and stderr. * results.csv: The results that the current commit is expected to produce.
-
- Nov 28, 2018
-
-
Yann Collet authored
Fix #1428 - zstdgrep now returns 1 on unmatch
-
Lzu Tao authored
-
Lzu Tao authored
-
Lzu Tao authored
-
Yann Collet authored
fixed a typo
-
Denis Ahrens authored
-
- Nov 27, 2018
-
-
Yann Collet authored
Update meson build system
-
Lzu Tao authored
NOTE: This commit only tested on Linux (Ubuntu 18.04). Windows build may not work as expected. * Use meson >= 0.47.0 cause we use install_man function * Add three helper Python script: * CopyFile.py: To copy file * CreateSymlink.py: To make symlink (both Windows and Unix) * GetZstdLibraryVersion.py: Parse lib/zstd.h to get zstd version These help emulating equivalent functions in CMake and Makefile. * Use subdir from meson to split meson.build * Add contrib build * Fix other build * Add new build options * build_programs: Enable programs build * build_contrib: Enable contrib build * build_tests: Enable tests build * use_static_runtime: Link to static run-time libraries on MSVC * zlib_support: Enable zlib support * lzma_support: Enable lzma support
-
Lzu Tao authored
-
Lzu Tao authored
-
Yann Collet authored
Fix feature detection with multiple -arch flags
-
- Nov 16, 2018
-
-
Nick Terrell authored
Fix i386 build failure "Junk character 13"
-
Ryan Schmidt authored
When multiple -arch flags are used, the compiler invokes itself once for each architecture. Apparently, input on stdin is consumed by the compilation of the first arch and is no longer available to the compilation of the second arch, which results in a build failure and the potentially incorrect determination that a feature is not available. So write the feature detection source to a file instead of using stdin.
-
Ryan Schmidt authored
-
- Nov 15, 2018
-
-
Nick Terrell authored
- Factor out LDM's hash function for reuse - Add rsyncable mode to zstdmt and expose it via the advanced API - Fix `-B`/`--block-size` to actually set the job size - Add rsyncable tests to `zstreamtest` and `playTests.sh` Tested by: ``` > cat A.100MB B.100MB C.100MB D.100MB E.100MB | zstd --rsyncable -fo src/file.zst /*stdin*\ : 48.22% (524288000 => 252837782 bytes, src/file.zst) > rsync -rc --stats src devbigvm:/data/users/terrelln/rsync-test Total bytes sent: 252868779 total size is 252837782 speedup is 1.00 > echo test > test > cat test A.100MB test B.100MB test C.100MB test D.100MB test E.100MB | zstd --rsyncable -fo src/file.zst /*stdin*\ : 48.23% (524288025 => 252838025 bytes, src/unicorn.tar.zst) > rsync -rc --stats src devbigvm:/data/users/terrelln/rsync-test Total bytes sent: 4605696 total size is 252838025 speedup is 53.60 ``` Close #1155.
-
Nick Terrell authored
-
Nick Terrell authored
-
Nick Terrell authored
-
Nick Terrell authored
- Add rsyncable mode to multithreaded mode - Factor out LDM's hash function for reuse
-
- Nov 14, 2018
-
-
Yann Collet authored
Separating benchfn and benchzstd
-
Yann Collet authored
because gcc considers that `void* const *` and `void**` are 2 different types ...
-
- Nov 13, 2018
-
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
to use structure for function parameters as it expresses much clearer than a long list of parameters, since each parameter can now be named.
-
Yann Collet authored
with BMK_extract_returnValue()
-
Yann Collet authored
it shall be possible to use benchfn without any dependency on zstd.
-
Yann Collet authored
Fix #1412: Perform Signed Comparison When Setting Attach Dict Param
-
Yann Collet authored
which is more accurate
-
- Nov 12, 2018
-
-
W. Felix Handte authored
-
W. Felix Handte authored
-
W. Felix Handte authored
-