Skip to content
Snippets Groups Projects
  1. Nov 20, 2023
    • Ben Dooks's avatar
      zstd: fix g_debuglevel export warning · 3f832dfb
      Ben Dooks authored
      
      The g_debuglevel variable in debug.c is only used when DEBUGLEVEL is
      defined to be above 2. This means by default there's no actual definition
      of this in the headers, so sparse is giving the following warning:
      
      lib/zstd/common/debug.c:24:5: warning: symbol 'g_debuglevel' was not declared. Should it be static?
      
      We can use the same check as in the header to remove this if it isn't
      going to be used, silencing the warning and removing a small bit of unused
      data.
      
      Signed-off-by: default avatarBen Dooks <ben.dooks@codethink.co.uk>
      Signed-off-by: default avatarNick Terrell <terrelln@fb.com>
      3f832dfb
    • Nick Terrell's avatar
      zstd: Backport Huffman speed improvement from upstream · 40eb0e91
      Nick Terrell authored
      Backport upstream commit c7269ad [0] to improve zstd decoding speed.
      
      Updating the kernel to zstd v1.5.5 earlier in this patch series
      regressed zstd decoding speed. This turned out to be because gcc was not
      unrolling the inner loops of the Huffman decoder which are executed a
      constant number of times [1]. This really hurts performance, as we expect
      this loop to be completely branch-free. This commit fixes the issue by
      unrolling the loop manually [2].
      
      The commit fixes one more minor issue, which is to mask a variable shift
      by 0x3F. The shift was guaranteed to be less than 64, but gcc couldn't
      prove that, and emitted suboptimal code.
      
      Finally, the upstream commit added a build macro
      `HUF_DISABLE_FAST_DECODE` which is not used in the kernel, but is
      maintained to keep a clean import from upstream.
      
      This commit was generated from upstream signed tag v1.5.5-kernel [3] by:
      
        export ZSTD=/path/to/repo/zstd/
        export LINUX=/path/to/repo/linux/
        cd "$ZSTD/contrib/linux-kernel"
        git checkout v1.5.5-kernel
        make import LINUX="$LINUX"
      
      I ran my benchmark & test suite before and after this commit to measure
      the overall decompression speed benefit. It benchmarks zstd at several
      compression levels. These benchmarks measure the total time it takes to
      read data from the compressed filesystem.
      
      Component,	Level,	Read time delta
      Btrfs    ,	    1,	-7.0%
      Btrfs    ,	    3,	-3.9%
      Btrfs    ,	    5,	-4.7%
      Btrfs    ,	    7,	-5.5%
      Btrfs    ,	    9,	-2.4%
      Squashfs ,	    1,	-9.1%
      
      Link: https://github.com/facebook/zstd/commit/c7269add7eaf028ed828d9af41e732cf01993aad
      Link: https://gist.github.com/terrelln/2e14ff1fb197102a08d7823d8044978d
      Link: https://gist.github.com/terrelln/a70bde22a2abc800691fb65c21eabc2a
      Link: https://github.com/facebook/zstd/tree/v1.5.5-kernel
      
      
      Signed-off-by: default avatarNick Terrell <terrelln@fb.com>
      40eb0e91
    • Nick Terrell's avatar
      zstd: import upstream v1.5.5 · 98988fc8
      Nick Terrell authored
      Import upstream zstd v1.5.5 to expose upstream's QAT integration.
      
      Import from upstream commit 58b3ef79 [0]. This is one commit before the
      tag v1.5.5-kernel [1], which is signed with upstream's signing key. The
      next patch in the series imports from v1.5.5-kernel, and is included in
      the series, rather than just importing directly from v1.5.5-kernel,
      because it is a non-trivial patch applied to improve the kernel's
      decompression speed. This commit contains 3 backported patches on top of
      v1.5.5: Two from the Linux copy of zstd, and one from upstream's `dev`
      branch.
      
      In addition to keeping the kernel's copy of zstd up to date, this update
      was requested by Intel to expose upstream zstd's external match provider
      API to the kernel, which allows QAT to accelerate the LZ match finding
      stage.
      
      This commit was generated by:
      
        export ZSTD=/path/to/repo/zstd/
        export LINUX=/path/to/repo/linux/
        cd "$ZSTD/contrib/linux-kernel"
        git checkout v1.5.5-kernel~
        make import LINUX="$LINUX"
      
      I tested and benchmarked this commit on x86-64 with gcc-13.2.1 on an
      Intel i9-9900K by running my benchmark scripts that benchmark zstd's
      performance in btrfs and squashfs compressed filesystems. This commit
      improves compression speed, especially for higher compression levels,
      and regresses decompression speed. But the decompression speed
      regression is addressed by the next patch in the series.
      
      Component,	Level,	C. time delta,	size delta,	D. time delta
      Btrfs    ,	    1,	        -1.9%,	     +0.0%,	        +9.5%
      Btrfs    ,	    3,	        -5.6%,	     +0.0%,	        +7.4%
      Btrfs    ,	    5,	        -4.9%,	     +0.0%,	        +5.0%
      Btrfs    ,	    7,	        -5.7%,	     +0.0%,	        +5.2%
      Btrfs    ,	    9,	        -5.7%,	     +0.0%,	        +4.0%
      Squashfs ,	    1,	          N/A,	      0.0%,	       +11.6%
      
      I also boot tested with a zstd compressed kernel on i386 and aarch64.
      
      Link: https://github.com/facebook/zstd/commit/58b3ef79eb9f1e6613684ea6e5b89720660ee8b6
      Link: https://github.com/facebook/zstd/tree/v1.5.5-kernel
      
      
      Signed-off-by: default avatarNick Terrell <terrelln@fb.com>
      98988fc8
  2. Nov 15, 2023
  3. Oct 30, 2023
  4. Oct 28, 2023
  5. Oct 27, 2023
  6. Oct 26, 2023
Loading