- Jan 08, 2022
-
-
Yann Collet authored
for better cost estimation in the following series for matches. Unfortunately, this does not necessarily result in better compression. Results are all over the places, with best outcome observed for silesia/x-ray but most other files tend to get slightly worse after this change. It's strange because it seems that we are just providing more accurate information for the cost estimator. Anyway, as it also increases code complexity, it's probably not interesting enough for now.
-
- Jan 07, 2022
-
-
Nick Terrell authored
[license] Fix license header of huf_decompress_amd64.S
-
Nick Terrell authored
* Add the license header for `huf_decompress_amd64.S` * Add `.S` files to the `test-license.py` test
-
Nick Terrell authored
[opt] Fix oss-fuzz bug in optimal parser
-
Felix Handte authored
Remove Dependencies to Allow the Zstd Binary to Dynamically Link to the Library
-
Nick Terrell authored
oss-fuzz uncovered a scenario where we're evaluating the cost of litLength = 131072, which can't be represented in the zstd format, so we accessed 1 beyond LL_bits. Fix the issue by making it cost 1 bit more than litLength = 131071. There are still follow ups: 1. This happened because literals_cost[0] = 0, so the optimal parser chose 36 literals over a match. Should we bound literals_cost[literal] > 0, unless the block truly only has one literal value? 2. When no matches are found, the cost model isn't updated. In this case no matches were found for an entire block. So the literals cost model wasn't updated at all. That made the optimal parser think literals_cost[0] = 0, where it is actually quite high, since the block was entirely random noise. Credit to OSS-Fuzz.
-
- Jan 06, 2022
-
-
W. Felix Handte authored
xxHash symbols are present in `libzstd.so`, but they are local and therefore unavailable outside the lib. There are two possible solutions to the problem. We could make those symbols global, or we could remove the dependency. This commit chooses the latter approach. I suppose this comes at the cost of code size / build time. I'm open to comments on whether this is a good thing to do, especially since this will apply even when we are statically linking everything.
-
W. Felix Handte authored
-
Felix Handte authored
Improve Module Map File
-
- Jan 05, 2022
-
-
W. Felix Handte authored
-
W. Felix Handte authored
This commit makes several changes: 1. It adds modules for the dictionary builder and errors headers. 2. It captures all of the macros that are used to configure these headers. When the headers are imported as modules and one of these macros is defined the compiler issues a warning that it needs to be defined on the CLI. 3. It promotes the modulemap file into the root of the lib directory. Experimentation shows that clang's `-fimplicit-module-maps` will find the modulemap when placed here, but not when it's put in a subdirectory.
-
Felix Handte authored
Mark Huffman Decoder Assembly `noexecstack` on All Architectures
-
W. Felix Handte authored
-
W. Felix Handte authored
-
W. Felix Handte authored
Previously we did this only on Linux, which missed other Unices.
-
W. Felix Handte authored
Hopefully this marks the binary artifacts `noexecstack` even on platforms where binaries default to true.
-
Nick Terrell authored
[meson] Explicitly disable assembly for non clang/gcc copmilers
-
Yann Collet authored
fix performance issue in scenario #2966 (part 1)
-
Nick Terrell authored
After merging #2951 I realized that we will want to explicitly disable assembly when we aren't including the assembly source file. Otherwise, if some non clang/gcc compiler does support assembly, it would fail to build.
-
- Jan 04, 2022
-
-
Nick Terrell authored
meson: fix MSVC support
-
W. Felix Handte authored
-
Yann Collet authored
-
- Jan 02, 2022
-
-
Yann Collet authored
-
Yann Collet authored
-
- Dec 31, 2021
-
-
Yann Collet authored
When re-using a compression state, across multiple successive compressions, the state should minimize the amount of allocation and initialization required. This mostly matters in situations where initialization is an overwhelming task compared to compression itself. This can happen when the amount to compress is small, while the compression state was given the impression that it would be much larger, aka, streaming mode without providing a srcSize hint. This lean-initialization optimization was broken in 980f3bbf . This commit fixes it, making this scenario once again on par with v1.4.9. Note that this does not completely fix #2966, since another heavy initialization, specific to row mode, is also happening (and was not present in v1.4.9). This will be fixed in a separate commit.
-
- Dec 30, 2021
-
-
Yann Collet authored
-
Yann Collet authored
-
W. Felix Handte authored
Apparently, even when the assembly file is empty (because `ZSTD_ENABLE_ASM_X86_64_BMI2` is false), it still is marked as possibly needing an executable stack and so the whole library is marked as such. This commit applies a simple patch for this problem by moving the noexecstack indication outside the macro guard. This commit builds on #2857. This commit addresses #2963.
-
Yann Collet authored
Refactor offset+repcode sumtype
-
- Dec 29, 2021
-
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
Fix mini typo
-
- Dec 28, 2021
-
-
Yann Collet authored
storeSeq & mlBase : clarity refactoring
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
-
Yann Collet authored
the new contracts seems to make more sense : updateRep() updates an array of repeat offsets _in place_, while newRep() generates a new structure with the updated repeat-offset array. Most callers are actually expecting the in-place variant, and a limited sub-section, in `zstd_opt.c` mainly, prefer `newRep()`.
-
Yann Collet authored
-