Otherwise MSan complains about use-of-uninitialized values in the window.
This happens in both regular deflate's longest_match and deflate_rle.

Before crrev.com/822755 we used to suppress those reports, but it seems
better to fix it properly. That will also allow us to catch other
potential issues with MSan in these functions.

The instances of this that we've seen only reproduce with
fill_window_sse(), not with the regular fill_window() function. Since
the former doesn't exist in upstream zlib, I'm not planning to send this
patch upstream.

Bug: 1137613, 1144420
Change-Id: I2b1801cd2a63fef48a0072b2d2c8fc1f8a7bb920
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2517520


Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Reviewed-by: Chris Blume <cblume@chromium.org>
Cr-Commit-Position: refs/heads/master@{#823845}
GitOrigin-RevId: 962cbbe81708214ff8e14e2bc8a07271cb15f1b9

prev_match can be set to -1 after sliding the window. In that case, the
window has slid past the first byte of the last match, which means it
cannot be compared in check_match.

This would cause zlib to crash on some inputs to deflate when built
with ZLIB_DEBUG enabled.

Check for this situation and avoid crashing by not trying to compare
the first byte.

Bug: 1113142
Change-Id: I26f006d81b91591e07fb02207e64b8440f7a5c50
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2426443


Commit-Queue: Hans Wennborg <hans@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Commit-Position: refs/heads/master@{#810195}
GitOrigin-RevId: c1560c99950947946ff9b5d5978f70bc975ab3af

When using CRC hashing, that assert doesn't hold, but a weaker variant
does.

Also disallow compiling with FASTEST defined, because I don't think
that longest_match variant would be safe with CRC hashing. It doesn't
guarantee that the fourth of the hashed bytes will be compared.

Bug: 1113596
Change-Id: I20ede29835b9c6b4bdc09fc1836864ffa2b10a97
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2401023


Commit-Queue: Hans Wennborg <hans@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Commit-Position: refs/heads/master@{#805690}
GitOrigin-RevId: 6d189bdd1c456fe0db3042b1ef3507ee96a0ff15

Otherwise a bad match can be chosen by longest_match, since it assumes
that match[2] and scan[2] are equal when the hashes and the other bytes
match. That assumption doesn't hold when using CRC hashing and a low
number of hash bits (which can be set by via the 'memLevel' parameter).

For example, the two hex byte sequences 2a 14 14 14 and 2a 14 db 14
have the same lower 9 bits CRC, and only differ in the third byte. This
could cause longest_match to consider them as matching.

This was already handled for x86; do the same for ARM and add a test.

Bug: 1113596
Change-Id: I251150594264f401e4d8bc6e91b44b39998ab029
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2392457


Reviewed-by: Chris Blume <cblume@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Commit-Queue: Chris Blume <cblume@chromium.org>
Auto-Submit: Hans Wennborg <hans@chromium.org>
Cr-Commit-Position: refs/heads/master@{#805076}
GitOrigin-RevId: 3fc5c74f812cda92f34897ed811694eaeb4f7654

Move the feature back to x86_cpu_enable_simd (was x86_cpu_enable_sse2)
to green waterfall android-pie-x86-rel bot, issue 1073809.

Failure to be investigated [1], potential change of feature name, etc,
but green android-pie-x86-rel waterfall bots come first.

[1] Failing tests were in net::WebSocketDeflaterTest (see CL:2161015),
and WebSocketDeflateParametersCompatibilityTest (also CL:2161015). Bot
failure details recorded in issue 1073809.

Tbr: cblume
Bug: 1072308, 1073809
Change-Id: Ib55cf308250c958a80c1a8dffe279767372b4048
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2162509


Commit-Queue: Noel Gordon <noel@chromium.org>
Reviewed-by: Benoit L <lizeb@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#761894}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ec864f4ecbc65d403b4e970c4411516e11e5d464

Add DEFLATE_FILL_WINDOW_SSE2 define for the fill_window_sse feature and
use it to guard the feature code (not ADLER32_SIMD_SSSE3).

The fill_window_sse feature only requires SSE2: use x86_cpu_enable_sse2
for the run-time check.

No-presubmit: true
Bug: 1072308
Change-Id: I770fffa619c998dae1c1f8da1cac8b973e43a509
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2156330


Commit-Queue: Noel Gordon <noel@chromium.org>
Auto-Submit: Noel Gordon <noel@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#761279}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: cd8dcf969915890e5b0d6aa5f9c35ec9a0363290

Due to its history, zlib has to support building for environments
without access to basic C functions (e.g. memcpy, memset, etc) or in
environments with odd memory management patterns (e.g. MSDOS).

Therefore, it is good form to use zmemset/zmemcpy/etc for handling
memory and the same applies for initializing a memory block to zero.

Bug: 1032721
Change-Id: I8d898872ca00197cc7e961d7c49e1912073895c2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2016772


Reviewed-by: Chris Blume <cblume@chromium.org>
Reviewed-by: vikas soni <vikassoni@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#734686}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: bcdbc75cd3f832ad0527a00f5bf56aa437abf1f6

Fix use of uninitialized memory reported by oss-fuzz
(and confirmed by valgrind@aarch64) by properly setting deflate_state
internal member (i.e. s->prev) to a valid value before use.

For details, see:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=11360

Bug: 1032721
Change-Id: I6c7b2e87e81b8ccc6c39298fd3c704befd797b96
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2015667


Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Reviewed-by: Chris Blume <cblume@chromium.org>
Reviewed-by: vikas soni <vikassoni@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#734278}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2d43e0d3369904d881e8519a69481226bba3394c

This will allow to remove some duplicated code (i.e. thread synchronization)
while at same time removing unnecessary use of inline ASM for Intel features
detection.

A few other advantages:
 - remove some extra logic (e.g. no need to test the platform to include the
   correct CPU detection header).
 - simplifies the buildsystem (i.e. we always include cpu_features.c)
 - get rid of the simd_stub file.

Bug: 1032721
Change-Id: I9427b34ec09dddc41925844a6ec4e6aa4d8f3207
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1987190


Reviewed-by: Mike Klein <mtklein@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#729515}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: d989ac2c596e88e7581eb93d053945a43c611124

This reverts commit 6f7e5e79cefe982ad84a88927565a88db2e592be.

Reason for revert: This broke the build, e.g.
https://ci.chromium.org/p/chromium/builders/ci/ios-device/144512

../../third_party/zlib/cpu_features.c:75:13: error: unused function '_cpu_check_features' [-Werror,-Wunused-function]
static void _cpu_check_features(void)
            ^

Original change's description:
> Unify CPU features detection code
> 
> This will allow to remove some duplicated code (i.e. thread synchronization)
> while at same time removing unnecessary use of inline ASM for Intel features
> detection.
> 
> A few other advantages:
>  - remove some extra logic (e.g. no need to test the platform to include the
>    correct CPU detection header).
>  - simplifies the buildsystem (i.e. we always include cpu_features.c)
>  - get rid of the simd_stub file.
> 
> Bug: 1032721
> Change-Id: Ic93472d3337bc2cbe092d4cf8fbe4b31b1ceca6d
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1976820


> Reviewed-by: Chris Blume <cblume@chromium.org>
> Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
> Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#727038}

TBR=cavalcantii@chromium.org,cblume@chromium.org,mtklein@chromium.org,adenilson.cavalcanti@arm.com

Change-Id: I20c5dedb98ba8b5d304ff1339042bcf243505e88
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1032721
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1979790


Reviewed-by: Hans Wennborg <hans@chromium.org>
Commit-Queue: Hans Wennborg <hans@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#727039}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 9d8f976414a7608c3361718462253104a761c6bb

This will allow to remove some duplicated code (i.e. thread synchronization)
while at same time removing unnecessary use of inline ASM for Intel features
detection.

A few other advantages:
 - remove some extra logic (e.g. no need to test the platform to include the
   correct CPU detection header).
 - simplifies the buildsystem (i.e. we always include cpu_features.c)
 - get rid of the simd_stub file.

Bug: 1032721
Change-Id: Ic93472d3337bc2cbe092d4cf8fbe4b31b1ceca6d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1976820


Reviewed-by: Chris Blume <cblume@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#727038}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 6f7e5e79cefe982ad84a88927565a88db2e592be

This change will unify the x86 and Arm optimized implementations
for insert_string (used for compression).

The objective here is two-fold:
a) Remove duplicated code.
b) Better insulate deflate.c divergence when compared to
vanilla zlib.

Bug: 1032721
Change-Id: Id2f65398aeb5a6384708493f0f6ae1fcd14022c2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1960893


Reviewed-by: Chris Blume <cblume@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#724325}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 9d4ec9349a1bf609eedb917c44c69eb0df9ff6bb

It seems that some years ago clang@Windows didn't have the
proper intrinsic required, which prompted the use of inline
ASM.

It has a side effect in that it will allow compilation of the
optimized function within the same compilation unit while using regular
compiler flags (i.e. 'crc32' instruction on x86 requires some special
compiler flags).

Main issue is that inline ASM is blocked on dependencies (e.g. 'base')
that will be linked to NaCl.

The main idea here is to allow the whole Chromium code base to use the
highly optimized checksums in zlib (e.g. crc32 and Adler-32), exported
through an interface (i.e. base::Crc32()).

This patch fixes this issue by removing the use of inline ASM.

The workaround is to use clang/gcc 'target attributes' to instruct the
backend to use different code generation options for the optimized
function, see:
https://clang.llvm.org/docs/AttributeReference.html#target

NOTE: While testing on my personal Windows PC, VS2019 including
smmintrin.h was insufficient. I needed to explicitly include either
immintrin.h or nmmintrin.h. I expected I would need that, but it seems
to be working with just smmintrin.h.

Bug: 902789
Change-Id: Id692fb839e20b26f9ba8b45538e652d5b140cd36
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1941688


Commit-Queue: Chris Blume <cblume@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#721425}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 1cebcd57bc3d09c39783395e6b173ff1f358a91b

This reverts commit ea6b9281bbf3ca08ccef8f5266f88de6f56c5ff6.

Reason for revert: It turns out the V8 team needs the MSVC build. :)

I tried installing the Clang compiler as part of Visual Studio 2017 and 2019 but neither of them became options in the toolchain. I'll need to spend more time figuring out how to get Clang on Windows.

Original change's description:
> Remove use of inline ASM in insert_string_sse
> 
> It seems that some years ago clang@Windows didn't have the
> proper intrinsic required, which prompted the use of inline
> ASM.
> 
> It has a side effect in that it will allow compilation of the
> optimized function within the same compilation unit while using regular
> compiler flags (i.e. 'crc32' instruction on x86 requires some special
> compiler flags).
> 
> Main issue is that inline ASM is blocked on dependencies (e.g. 'base')
> that will be linked to NaCl.
> 
> The main idea here is to allow the whole Chromium code base to use the
> highly optimized checksums in zlib (e.g. crc32 and Adler-32), exported
> through an interface (i.e. base::Crc32()).
> 
> This patch fixes this issue by removing the use of inline ASM.
> 
> The workaround is to use clang/gcc 'target attributes' to instruct the
> backend to use different code generation options for the optimized
> function, see:
> https://clang.llvm.org/docs/AttributeReference.html#target
> 
> Bug: 902789
> Change-Id: I0d139268aefb8335310c0e3f6533006be9af6470
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1931272


> Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
> Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#718788}

TBR=cavalcantii@chromium.org,cblume@chromium.org,mtklein@chromium.org,adenilson.cavalcanti@arm.com

Change-Id: I6b3fcce10197121b548300855710e99f7048f1ae
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 902789
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1936189


Reviewed-by: Chris Blume <cblume@chromium.org>
Commit-Queue: Chris Blume <cblume@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#719105}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 80c2a793b4ba20d9638fbdd030a1687dc26242a3

It seems that some years ago clang@Windows didn't have the
proper intrinsic required, which prompted the use of inline
ASM.

It has a side effect in that it will allow compilation of the
optimized function within the same compilation unit while using regular
compiler flags (i.e. 'crc32' instruction on x86 requires some special
compiler flags).

Main issue is that inline ASM is blocked on dependencies (e.g. 'base')
that will be linked to NaCl.

The main idea here is to allow the whole Chromium code base to use the
highly optimized checksums in zlib (e.g. crc32 and Adler-32), exported
through an interface (i.e. base::Crc32()).

This patch fixes this issue by removing the use of inline ASM.

The workaround is to use clang/gcc 'target attributes' to instruct the
backend to use different code generation options for the optimized
function, see:
https://clang.llvm.org/docs/AttributeReference.html#target

Bug: 902789
Change-Id: I0d139268aefb8335310c0e3f6533006be9af6470
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1931272


Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#718788}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ea6b9281bbf3ca08ccef8f5266f88de6f56c5ff6

While investigating the use of RAW mode, I noticed that it was *slower*
for compression than GZIP or ZLIB wrapper formats, which didn't make
sense (i.e. we don't calculate a data integrity check using RAW mode).

It turns out that the code was falling back to the portable implementation
of insert_string(), instead of using the optimized version that rely on 'crc32w'
as a hash function.

The reason is that CPU features detection is not triggered while using
RAW mode (i.e. we never call crc32() with a NULL buffer).

This patch fixes this issue ensuring that RAW mode is going to be faster
for both compression/decompression than either ZLIB or GZIP formats.

Bug: 833361
Change-Id: I285297f67ffc0114700ed03c2186ad21aab8b40e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1929634


Reviewed-by: Chris Blume <cblume@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#717877}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 171a0a69eb5d70f8a9f44000e26bc7dc65f1fd97

Faster symbol hashing has a positive impact on data compression speed
(around 20% on Intel and 36% on Arm Cortex big cores).

A misfeature is that the generated compressed output will differ from
vanilla zlib (even though it is still valid 'DEFLATE-d' content) and in
some cases has slightly better compression ratio.

We offer here a way to disable the optimization if there is the expectation
that compressed content should match when compared to vanilla zlib.

Bug: 990489
Change-Id: I910e140eb15dd334eb869b078f940581249a7e6d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1742297


Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Reviewed-by: Chris Blume <cblume@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#685041}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5b8477c39bb70cb1cbb8bbbc8149674fa65f1c91

It's possible to get here while building for ARM64.

Change-Id: Icf621059f4cfdec28d71437917ebb348369e6c26
Reviewed-on: https://chromium-review.googlesource.com/c/1368105


Reviewed-by: Leon Scroggins <scroggo@chromium.org>
Commit-Queue: Mike Klein <mtklein@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#614819}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 6ffae24943b91fd5ad63a5a6bec18c544244877b

When a payload is compressed via zlib using a known-in-advance
dictionary symbols that are distant can cause the compression output
buffer to be overwritten.

This patch applies a fix that comes from upstream:
https://github.com/madler/zlib/commit/5c44459c3b28a9bd3283aaceab7c615f8020c531
That fix hasn't yet landed in the master branch.

This will prevent us from generating those invalid payloads.

BUG=874550

Change-Id: Ic421d7242450f866388c90cee67cbc48b92ed83c
Reviewed-on: https://chromium-review.googlesource.com/1176284


Commit-Queue: Chris Blume <cblume@chromium.org>
Reviewed-by: Will Harris <wfh@chromium.org>
Reviewed-by: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#583975}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 777b3763b4f45e4e10f6ff3c6cd46c6e086dc243

Using a faster hash function yields a considerable performance boost
in compression (average 8% on A53 and 24% on A72).

This change was enabled by previous patch with optimized crc32 using
ARMv8-1 crypto extensions for performing CPU feature detection
(so won't help older ARMv7 SoCs).

Bug: 873759
Change-Id: I88ece549a63d923beef4f96a046acdf09e529784
Reviewed-on: https://chromium-review.googlesource.com/1173262


Reviewed-by: Chris Blume <cblume@chromium.org>
Reviewed-by: Mike Klein <mtklein@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#583113}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 1364a33fe0f2b9588a2d018f62ff4d966a525f37

This should help with compression of data, using NEON instructions
(therefore useful for ARMv7/ARMv8).

Average gains were around 4% to 5% in data compression, depending on the
data entropy.

Re-write of a patch contributed to Fedora Core, for reference:
https://src.fedoraproject.org/rpms/zlib/c/25e9802713484882c27c1f979a6610a42414ee13?branch=master

Bug: 863257
Change-Id: I76573d75843d6a63de75d8a9536da98515314543
Reviewed-on: https://chromium-review.googlesource.com/1136940


Reviewed-by: Mike Klein <mtklein@chromium.org>
Reviewed-by: Chris Blume <cblume@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#581241}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ddbbeb05cb3f0a4c27e7a5d5d0305462db373677

zlib.h includes a macro that renames read_buf->Cr_z_read_buf. Since
read_buf is a common name in other parts of the code, it causes
some random confusion depending on whether zlib.h has been
included or not.

The renaming macro is a side effect of the 0001-simd.patch that
exposes an internal read_buf method to other files in zlib. This
patch renames read_buf as it is exposed so that it has the less
common name deflate_read_buf.

Bug: 799448
Change-Id: Icdc4eba973891dfd28d82017415048eded62d577
Reviewed-on: https://chromium-review.googlesource.com/852257


Commit-Queue: Daniel Bratell <bratell@opera.com>
Reviewed-by: Chris Blume <cblume@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#528512}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2c709d38a1c6f812da205c03f5448fe4ac5679f3

This reverts commit 35988c821c051a57e30c76f9fcd87b7b677bd9bd.

Reason for revert: broke build ('cpu-features.h' not found)
https://uberchromegw.corp.google.com/i/internal.client.clank/builders/x64-builder/builds/13697

Original change's description:
> Using ARMv8 CRC32 specific instruction
> 
> CRC32 affects performance for both image decompression (PNG)
> as also in general browsing while accessing websites that serve
> content using compression (i.e. Content-Encoding: gzip).
> 
> This patch implements an optimized CRC32 function using the
> dedicated instruction available in ARMv8. This instruction is available
> in new Android devices featuring an ARMv8 SoC, like Nexus 5x and
> Google Pixel.
> 
> It should be between 6x (A53: 116ms X 22ms for a 4Kx4Kx4 buffer) to
> 10x faster (A72: 91ms x 9ms) than the C implementation currently used
> by zlib.
> 
> PNG decoding performance gains should be around 5-9%.
> 
> Finally it also introduces code to perform the ARM CPU features detection
> using getauxval()@Linux/CrOS or android_getCpuFeatures(). We pre-built
> and link the CRC32 instruction dependent code but will decide if to
> use it at run time.
> 
> If the feature is not supported, we fallback to the C implementation.
> 
> This approach allows to use the instruction in both 32bits and 64bits
> builds and works fine either in ARMv7 or ARMv8 processor. I tested the
> generated Chromium apk in both a ARMv7 (Nexus 4 and 6) and ARMv8 (Nexus 5x and
> Google Pixel).
> 
> Change-Id: I069408ebc06c49a3c2be4ba3253319e025ee09d7
> Bug: 709716
> Reviewed-on: https://chromium-review.googlesource.com/612629


> Reviewed-by: Chris Blume <cblume@chromium.org>
> Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#520377}

TBR=agl@chromium.org,noel@chromium.org,cavalcantii@chromium.org,cblume@chromium.org,mtklein@chromium.org,adenilson.cavalcanti@arm.com

Change-Id: Ief2c32df5c8a37635f937cd6a671f5574f5a53a3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 709716
Reviewed-on: https://chromium-review.googlesource.com/799930


Reviewed-by: Chris Blume <cblume@chromium.org>
Reviewed-by: Boris Sazonov <bsazonov@chromium.org>
Commit-Queue: Boris Sazonov <bsazonov@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#520497}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e7d9a4649bde6f047105d29f0026dd8c3d54143a

CRC32 affects performance for both image decompression (PNG)
as also in general browsing while accessing websites that serve
content using compression (i.e. Content-Encoding: gzip).

This patch implements an optimized CRC32 function using the
dedicated instruction available in ARMv8. This instruction is available
in new Android devices featuring an ARMv8 SoC, like Nexus 5x and
Google Pixel.

It should be between 6x (A53: 116ms X 22ms for a 4Kx4Kx4 buffer) to
10x faster (A72: 91ms x 9ms) than the C implementation currently used
by zlib.

PNG decoding performance gains should be around 5-9%.

Finally it also introduces code to perform the ARM CPU features detection
using getauxval()@Linux/CrOS or android_getCpuFeatures(). We pre-built
and link the CRC32 instruction dependent code but will decide if to
use it at run time.

If the feature is not supported, we fallback to the C implementation.

This approach allows to use the instruction in both 32bits and 64bits
builds and works fine either in ARMv7 or ARMv8 processor. I tested the
generated Chromium apk in both a ARMv7 (Nexus 4 and 6) and ARMv8 (Nexus 5x and
Google Pixel).

Change-Id: I069408ebc06c49a3c2be4ba3253319e025ee09d7
Bug: 709716
Reviewed-on: https://chromium-review.googlesource.com/612629


Reviewed-by: Chris Blume <cblume@chromium.org>
Commit-Queue: Adenilson Cavalcanti <cavalcantii@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#520377}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 35988c821c051a57e30c76f9fcd87b7b677bd9bd

Reapply and regenerate all local patches to upstream zlib 1.2.11

Explicitly specify 9 as the minimum windowBits value (representing a
512-byte window) during compression in net/websockets even when 8
(representing 256) is received. This was previously silently done during
compression. Because of how zlib's deflate is implemented, when
windowBits is 9, it will produce a stream that can be decompressed with
a 250-byte or larger window.

Changes in 1.2.9 (31 Dec 2016)
- Fix contrib/minizip to permit unzipping with desktop API [Zouzou]
- Improve contrib/blast to return unused bytes
- Assure that gzoffset() is correct when appending
- Improve compress() and uncompress() to support large lengths
- Fix bug in test/example.c where error code not saved
- Remedy Coverity warning [Randers-Pehrson]
- Improve speed of gzprintf() in transparent mode
- Fix inflateInit2() bug when windowBits is 16 or 32
- Change DEBUG macro to ZLIB_DEBUG
- Avoid uninitialized access by gzclose_w()
- Allow building zlib outside of the source directory
- Fix bug that accepted invalid zlib header when windowBits is zero
- Fix gzseek() problem on MinGW due to buggy _lseeki64 there
- Loop on write() calls in gzwrite.c in case of non-blocking I/O
- Add --warn (-w) option to ./configure for more compiler warnings
- Reject a window size of 256 bytes if not using the zlib wrapper
- Fix bug when level 0 used with Z_HUFFMAN or Z_RLE
- Add --debug (-d) option to ./configure to define ZLIB_DEBUG
- Fix bugs in creating a very large gzip header
- Add uncompress2() function, which returns the input size used
- Assure that deflateParams() will not switch functions mid-block
- Dramatically speed up deflation for level 0 (storing)
- Add gzfread(), duplicating the interface of fread()
- Add gzfwrite(), duplicating the interface of fwrite()
- Add deflateGetDictionary() function
- Use snprintf() for later versions of Microsoft C
- Fix *Init macros to use z_ prefix when requested
- Replace as400 with os400 for OS/400 support [Monnerat]
- Add crc32_z() and adler32_z() functions with size_t lengths
- Update Visual Studio project files [AraHaan]

Changes in 1.2.10 (2 Jan 2017)
- Avoid warnings on snprintf() return value
- Fix bug in deflate_stored() for zero-length input
- Fix bug in gzwrite.c that produced corrupt gzip files
- Remove files to be installed before copying them in Makefile.in
- Add warnings when compiling with assembler code

Changes in 1.2.11 (15 Jan 2017)
- Fix deflate stored bug when pulling last block from window
- Permit immediate deflateParams changes before any deflate input

BUG=691074, 691075

Review-Url: https://codereview.chromium.org/2690623003
Cr-Original-Commit-Position: refs/heads/master@{#450585}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 6d9a6251dfe87183075dc16cfa134e41dc4cee0d

With SPDY gone, this is no longer necessary. This change was done by
reversing simd.patch, reversing mixed-source.patch, reapplying
simd.patch, and then recreating simd.patch.

BUG=none

Review-Url: https://codereview.chromium.org/2669053004
Cr-Original-Commit-Position: refs/heads/master@{#447853}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: f2095b9da969dd7decf3224fc96ffe8ef5f5455b

merge the latest open source zlib into chromium external projects
Version 1.2.8 fixes a very rare bug in decompression. All users are encouraged to upgrade immediately. Version 1.2.8 also has these improvements:
- Add new inflateGetDictionary() function
- Fix bug where gzopen() immediately followed by gzclose() would write an empty file instead of an empty gzip stream.
- Fix bug in gzclose() when gzwrite() runs out of memory Version 1.2.7 has many portability improvements over 1.2.6, and has these enhancements:
- Fix bug in gzclose_w() when gzwrite() fails to allocate memory
- Add "x" (O_EXCL) and "e" (O_CLOEXEC) modes support to gzopen()
- Add gzopen_w() in Windows for wide character path names
- Fix type mismatch between get_crc_table() and crc_table Version 1.2.6 has many changes over 1.2.5, including these improvements:
- gzread() can now read a file that is being written concurrently
- gzgetc() is now a macro for increased speed
- Added a 'T' option to gzopen() for transparent writing (no compression)
- Added deflatePending() to return the amount of pending output
- Allow deflateSetDictionary() and inflateSetDictionary() at any time in raw mode
- deflatePrime() can now insert bits in the middle of the stream
- ./configure now creates a configure.log file with all of the results
- Added a ./configure --solo option to compile zlib with no dependency on any libraries
- Fixed a problem with large file support macros
- Fixed a bug in contrib/puff
- Many portability improvements

BUG=610615 R=gavinp@chromium.org

Review-Url: https://codereview.chromium.org/2084863002
Cr-Original-Commit-Position: refs/heads/master@{#401538}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ca1c569cff20aa43a42816791744d847c7d788de

Revert of Update Zlib to version 1.2.8 (patchset #10 id:220001 of https://codereview.chromium.org/1955383002/ )

Reason for revert:
Broke the Windows Debug GPU FYI builder:

https://build.chromium.org/p/chromium.gpu.fyi/builders/GPU%20Win%20Builder%20%28dbg%29/builds/41421

fx_zlib.gzlib.obj : error LNK2005: _gzopen_w already defined in zlib.gzlib.obj

initial\chrome.dll : fatal error LNK1169: one or more multiply defined symbols found

Possibly because they use shared components with a GYP build.

Original issue's description:
> Update Zlib to version 1.2.8
>
> merge the latest open source zlib into chromium external projects
>
>  Version 1.2.8 fixes a very rare bug in decompression. All users are encouraged to upgrade immediately. Version 1.2.8 also has these improvements:
>     - Add new inflateGetDictionary() function
>     - Fix bug where gzopen() immediately followed by gzclose() would write an empty file instead of an empty gzip stream.
>     - Fix bug in gzclose() when gzwrite() runs out of memory
>     Version 1.2.7 has many portability improvements over 1.2.6, and has these enhancements:
>     - Fix bug in gzclose_w() when gzwrite() fails to allocate memory
>     - Add "x" (O_EXCL) and "e" (O_CLOEXEC) modes support to gzopen()
>     - Add gzopen_w() in Windows for wide character path names
>     - Fix type mismatch between get_crc_table() and crc_table
>     Version 1.2.6 has many changes over 1.2.5, including these improvements:
>     - gzread() can now read a file that is being written concurrently
>     - gzgetc() is now a macro for increased speed
>     - Added a 'T' option to gzopen() for transparent writing (no compression)
>     - Added deflatePending() to return the amount of pending output
>     - Allow deflateSetDictionary() and inflateSetDictionary() at any time in raw mode
>     - deflatePrime() can now insert bits in the middle of the stream
>     - ./configure now creates a configure.log file with all of the results
>     - Added a ./configure --solo option to compile zlib with no dependency on any libraries
>     - Fixed a problem with large file support macros
>     - Fixed a bug in contrib/puff
>     - Many portability improvements
>
> BUG=610615
> R=agl@chromium.org
> R=gavinp@chromium.org
>
> Committed: https://crrev.com/a50849ae9ba1b2fad47905db521463b3bf085883
> Cr-Commit-Position: refs/heads/master@{#400647}

TBR=agl@chromium.org,gavinp@chromium.org,khasim.mohammed@linaro.org,jiadong.zhu@linaro.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=610615

Review-Url: https://codereview.chromium.org/2079313002
Cr-Original-Commit-Position: refs/heads/master@{#400670}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 43770253ccc0425c4edde966e56b3e68955cc4d3

merge the latest open source zlib into chromium external projects

 Version 1.2.8 fixes a very rare bug in decompression. All users are encouraged to upgrade immediately. Version 1.2.8 also has these improvements:
    - Add new inflateGetDictionary() function
    - Fix bug where gzopen() immediately followed by gzclose() would write an empty file instead of an empty gzip stream.
    - Fix bug in gzclose() when gzwrite() runs out of memory
    Version 1.2.7 has many portability improvements over 1.2.6, and has these enhancements:
    - Fix bug in gzclose_w() when gzwrite() fails to allocate memory
    - Add "x" (O_EXCL) and "e" (O_CLOEXEC) modes support to gzopen()
    - Add gzopen_w() in Windows for wide character path names
    - Fix type mismatch between get_crc_table() and crc_table
    Version 1.2.6 has many changes over 1.2.5, including these improvements:
    - gzread() can now read a file that is being written concurrently
    - gzgetc() is now a macro for increased speed
    - Added a 'T' option to gzopen() for transparent writing (no compression)
    - Added deflatePending() to return the amount of pending output
    - Allow deflateSetDictionary() and inflateSetDictionary() at any time in raw mode
    - deflatePrime() can now insert bits in the middle of the stream
    - ./configure now creates a configure.log file with all of the results
    - Added a ./configure --solo option to compile zlib with no dependency on any libraries
    - Fixed a problem with large file support macros
    - Fixed a bug in contrib/puff
    - Many portability improvements

BUG=610615
R=agl@chromium.org
R=gavinp@chromium.org

Review-Url: https://codereview.chromium.org/1955383002
Cr-Original-Commit-Position: refs/heads/master@{#400647}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a50849ae9ba1b2fad47905db521463b3bf085883

This version uses a "pthread_once" implementation, using Windows
synchronisation primitives, imported from tcmalloc.

Previous CLs:
https://codereview.chromium.org/677713002/
https://codereview.chromium.org/552123005

This version of the CL also runs fine on Windows Server 2003.

These optimisations have been published on zlib mailing list and at
https://github.com/jtkukunas/zlib/

This change merges the following optimisation patches:
- "For x86, add CPUID check."
- "Adds SSE2 optimized hash shifting to fill_window."
- "add SSE4.2 optimized hash function"
- "add PCLMULQDQ optimized CRC folding"

From Jim Kukunas <james.t.kukunas@linux.intel.com>; and adapts them to the
current zlib version in Chromium.

The optimisations are enabled at runtime if all the necessary CPU features are
present. As the optimisations require extra cflags to enable the compiler to
use the instructions the optimisations are held in their own static library
with a stub implementation to allow linking on other platforms.

TEST=net_unittests(GZipUnitTest) passes, Chrome functions and performance
improvement seen on RoboHornet benchmark on Linux Desktop
BUG=401517

Review URL: https://codereview.chromium.org/678423002

Cr-Original-Commit-Position: refs/heads/master@{#302799}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 02a95e3084f979084fa8586e1718a6e6dd4c22da

Revert of Reland "Integrate SIMD optimisations for zlib" (patchset #2 id:40001 of https://codereview.chromium.org/677713002/)

Reason for revert:
Speculatively reverting because XP Tests (1) is having failures.

https://build.chromium.org/p/chromium.win/builders/XP%20Tests%20(1)

Original issue's description:
> Reland "Integrate SIMD optimisations for zlib"
>
> This reland adds an MSan suppression entry to work around gaps in MSan's
> support for some of the intrinsics this patch uses. This version also inlines
> the insert_string_sse function as it uses inline assembly and therefore does
> not need to be in the static library.
>
> Original CL: https://codereview.chromium.org/552123005
>
> These optimisations have been published on zlib mailing list and at
> https://github.com/jtkukunas/zlib/
>
> This change merges the following optimisation patches:
> - "For x86, add CPUID check."
> - "Adds SSE2 optimized hash shifting to fill_window."
> - "add SSE4.2 optimized hash function"
> - "add PCLMULQDQ optimized CRC folding"
>
> From Jim Kukunas <james.t.kukunas@linux.intel.com>; and adapts them to the
> current zlib version in Chromium.
>
> The optimisations are enabled at runtime if all the necessary CPU features are
> present. As the optimisations require extra cflags to enable the compiler to
> use the instructions the optimisations are held in their own static library
> with a stub implementation to allow linking on other platforms.
>
> TEST=net_unittests(GZipUnitTest) passes, Chrome functions and performance
> improvement seen on RoboHornet benchmark on Linux Desktop
> BUG=401517
>
> Committed: https://crrev.com/a5022d5eab6f77889aceed6ab0ccaf44a657ffc4
> Cr-Commit-Position: refs/heads/master@{#301162}

TBR=agl@chromium.org,hans@chromium.org,robert.bradford@intel.com
NOTREECHECKS=true
NOTRY=true
BUG=401517

Review URL: https://codereview.chromium.org/665203006

Cr-Original-Commit-Position: refs/heads/master@{#301221}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5d38e0bd32f9a7e4766b877711c710df986d74ed

This reland adds an MSan suppression entry to work around gaps in MSan's
support for some of the intrinsics this patch uses. This version also inlines
the insert_string_sse function as it uses inline assembly and therefore does
not need to be in the static library.

Original CL: https://codereview.chromium.org/552123005

These optimisations have been published on zlib mailing list and at
https://github.com/jtkukunas/zlib/

This change merges the following optimisation patches:
- "For x86, add CPUID check."
- "Adds SSE2 optimized hash shifting to fill_window."
- "add SSE4.2 optimized hash function"
- "add PCLMULQDQ optimized CRC folding"

From Jim Kukunas <james.t.kukunas@linux.intel.com>; and adapts them to the
current zlib version in Chromium.

The optimisations are enabled at runtime if all the necessary CPU features are
present. As the optimisations require extra cflags to enable the compiler to
use the instructions the optimisations are held in their own static library
with a stub implementation to allow linking on other platforms.

TEST=net_unittests(GZipUnitTest) passes, Chrome functions and performance
improvement seen on RoboHornet benchmark on Linux Desktop
BUG=401517

Review URL: https://codereview.chromium.org/677713002

Cr-Original-Commit-Position: refs/heads/master@{#301162}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a5022d5eab6f77889aceed6ab0ccaf44a657ffc4

Revert of Integrate SIMD optimisations for zlib (patchset #14 id:280001 of https://codereview.chromium.org/552123005/)

Reason for revert:
Strong candidate for: multiple Linux MSan test failures with:

SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 ??

Including: CompressionUtilsTest.GzipCompression

Original issue's description:
> Integrate SIMD optimisations for zlib
>
> These optimisations have been published on zlib mailing list and at
> https://github.com/jtkukunas/zlib/
>
> This change merges the following optimisation patches:
> - "For x86, add CPUID check."
> - "Adds SSE2 optimized hash shifting to fill_window."
> - "add SSE4.2 optimized hash function"
> - "add PCLMULQDQ optimized CRC folding"
>
> From Jim Kukunas <james.t.kukunas@linux.intel.com> and adapts them to the
> current zlib version in Chromium.
>
> The optimisations are enabled at runtime if all the necessary CPU features are
> present. As the optimisations require extra cflags to enable the compiler to
> use the instructions the optimisations are held in their own static library
> with a stub implementation to allow linking on other platforms.
>
> TEST=net_unittests(GZipUnitTest) passes, Chrome functions and performance
> improvement seen on RoboHornet benchmark on Linux Desktop
> BUG=401517
>
> Committed: https://crrev.com/e045ec106de29562ae94eafccde49a7b73848471
> Cr-Commit-Position: refs/heads/master@{#300866}

TBR=agl@chromium.org,nathan.d.ciobanu@intel.com
NOTREECHECKS=true
NOTRY=true
BUG=401517

Review URL: https://codereview.chromium.org/671163003

Cr-Original-Commit-Position: refs/heads/master@{#300889}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ca54b3014721cdc27e4a17dd2c82fe5811fa39cc

These optimisations have been published on zlib mailing list and at
https://github.com/jtkukunas/zlib/

This change merges the following optimisation patches:
- "For x86, add CPUID check."
- "Adds SSE2 optimized hash shifting to fill_window."
- "add SSE4.2 optimized hash function"
- "add PCLMULQDQ optimized CRC folding"

From Jim Kukunas <james.t.kukunas@linux.intel.com> and adapts them to the
current zlib version in Chromium.

The optimisations are enabled at runtime if all the necessary CPU features are
present. As the optimisations require extra cflags to enable the compiler to
use the instructions the optimisations are held in their own static library
with a stub implementation to allow linking on other platforms.

TEST=net_unittests(GZipUnitTest) passes, Chrome functions and performance
improvement seen on RoboHornet benchmark on Linux Desktop
BUG=401517

Review URL: https://codereview.chromium.org/552123005

Cr-Original-Commit-Position: refs/heads/master@{#300866}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e045ec106de29562ae94eafccde49a7b73848471

Previously, SPDY cookies would look like:

  "FOO=BAR;BAR=BOO;WIBBLE=WOBBLE;"

With this patch they will look like:

  "FOO=BAR; BAR=BOO; WIBBLE=WOBBLE"

This will hurt compression a little as cookies are only matched completely so:

  Cookie: FOO=BAR
          |-----|

won't match against the next Cookie header with an additional cookie:

  Cookie: FOO=BAR; BAR=BOO
          |------||------|

However, once the set of cookies is stable, it does fine.

BUG=151433


Review URL: https://chromiumcodereview.appspot.com/10957038

Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a19cb733bc10273d8e31c6a3e71d4d8f0b6b60f2

(This is a reland of r151502, which was reverted in r151517 because it
broke the Linux Official build.)

This may break the Official build for a brief window while a two-sided
patch lands.

BUG=139744

Review URL: https://chromiumcodereview.appspot.com/10837057

Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 9523c88db869fc1df3b95e29a6d6a8e2fa8bad1f

This reverts r151502.

Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: c84b42002c558e2eaff044167b6c22a090c74bfc

BUG=139744

Review URL: https://chromiumcodereview.appspot.com/10837057

Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 1f73d5478f1a940cefb256d1d6aef9694532fb65

Another step towards a script that does the zlib upgrades for us...

Change-Id: I940b5b4e79d2a8daf7a47129bbea7bdaffd4140c

Change-Id: Id2bcb4a91755b39521c1151ebdcae557a4a783ce