67 Commits

Author SHA1 Message Date
James Zern
45129ee027 Revert "Check all the rows."
This reverts commit ee26766a89a149afe5f73fdcb8f2493ec808f2b7.

This change also reverts the parent.

Revert "Increase the transform bits if possible."

This reverts commit 7ec51c591608333fd5e6fdddbee16e1d65bef3db.

These changes result in non-lossless encodes.

Bug: oss-fuzz:69231, oss-fuzz:69109, oss-fuzz:69208
Bug: b:341475869, b:342743143
Change-Id: Ia28f558992e0aa6f024af1ff66da52e0a5e26fa3
2024-05-25 11:00:32 -07:00
Vincent Rabaud
7ec51c5916 Increase the transform bits if possible.
This brings minor size improvements because repetitive values in
the transform images are easily explainable through LZ77. Still,
it makes an upcoming pull request a bit more stable.

This is 971a03d8204c3ab1e12db8ae68ba6f919c615ade with a fix to
not forget to analyze the end of the line.
A const has also been added to match VP8LColorSpaceTransform's
signature.

Change-Id: Iae03216fef298c7abc96a766f8a799552b05ade5
2024-05-23 14:04:34 +02:00
James Zern
3cd16fd3e2 Revert "Increase the transform bits if possible."
This reverts commit 971a03d8204c3ab1e12db8ae68ba6f919c615ade.

Reason for revert:
This creates non-lossless encodes.

Original change's description:
> Increase the transform bits if possible.
>
> This brings minor size improvements because repetitive values in
> the transform images are easily explainable through LZ77. Still,
> it makes an upcoming pull request a bit more stable.
>
> Change-Id: I1c7135675cb59b5e27ca960738d74465f10d0deb

Bug: oss-fuzz:69109, b:341475869
Change-Id: I3b9f21a5498735eb3681e62fb35bf9f9c2ed4f9f
2024-05-20 22:25:57 +00:00
Vincent Rabaud
971a03d820 Increase the transform bits if possible.
This brings minor size improvements because repetitive values in
the transform images are easily explainable through LZ77. Still,
it makes an upcoming pull request a bit more stable.

Change-Id: I1c7135675cb59b5e27ca960738d74465f10d0deb
2024-05-17 15:19:03 +02:00
Vincent Rabaud
1bf198a22b Allow transform_bits to be different during encoding.
The spec allows it but it is currently forced to the same value for simplicity.

Change-Id: I26197dbf3342f7a72115cc7f7805c154313a2afb
2024-05-13 16:56:19 +02:00
Vincent Rabaud
1e462ca80e Define MAX_TRANSFORM_BITS according to the specification.
Change-Id: I0d575aa84e143bea56b55deb8f42b44e13dd5f1e
2024-05-07 09:16:02 +02:00
Vincent Rabaud
64d1ec23ac Use (MIN/NUM)_(TRANSFORM/HUFFMAN)_BITS where appropriate
Change-Id: I849ff8864f7abcc723dfe2b7ee0f290c8ee89c3f
2024-05-06 22:46:44 +02:00
Vincent Rabaud
a7aa7525b8 Fix some function declarations
- fix some function declarations to match the implementation
- fix some consts

Change-Id: I8c89f49ec68d3dd1db7f8ee5cac73777f52d2576
2024-05-03 20:10:46 +02:00
Vincent Rabaud
34c8074915 Remove alpha encoding pessimization.
This was created to have outputs readable with libwebp 0.4.x
Jelly-Bean and Kit-Kat are not supported anymore:
https://android-developers.googleblog.com/2021/07/google-play-services-discontinuing-jelly-bean.html
https://android-developers.googleblog.com/2023/07/google-play-services-discontinuing-updates-for-kitkat.html

Change-Id: I0b9ff3091412ce12eb50da301d7dacace9b54cc3
2023-11-14 14:18:12 +01:00
Vincent Rabaud
13d9c30b2b Add a WEBP_NODISCARD
Change-Id: Ice66f2aa6358474d728fb19c571edc86ed139a49
2023-11-10 13:57:07 +01:00
Vincent Rabaud
c1ffd9ac75 Merge "vp8l_enc: fix non-C90 code" into main 2023-09-15 06:45:52 +00:00
James Zern
f80e9b7e4c vp8l_enc: fix non-C90 code
Change-Id: I14b5288c1b9392dcb64db09b9852f1d7d47b3200
2023-09-14 16:21:04 -07:00
Vincent Rabaud
433c7dca11 Fix static analyzer warnings.
Change-Id: I45f0db2310b1188809963af93240e3d438f807b8
2023-09-14 22:43:45 +02:00
Vincent Rabaud
1432ebbadb Refactor palette sorting computation.
This will ease the integration of new methods.

Change-Id: Icec3eddaa4ab9030d28ccfe579eb6dc13ded2f02
2023-07-26 10:25:14 +02:00
Vincent Rabaud
eac3bd5c53 Have the palette code be in its own file.
Change-Id: I099a342effedd9f451c94d00a14aead27079e6cc
2023-07-06 22:09:24 +02:00
James Zern
4339332038 enc/*: normalize WebPEncodingSetError() calls
return WebPEncodingSetError() directly where possible (it returns 0 /
failure for convenience).

Change-Id: I89a937778886c15ecbbe078af5b4c8c89ef0f9af
2023-06-21 16:43:00 -07:00
James Zern
287fdefe95 enc/*: add missing WebPEncodingSetError() calls
Users of the encoder (including anim_encode.c) and areas of the encoder
itself rely on the status returned via WebPPicture.

Change-Id: Id786176b8ac3b2329d1e41b9dacbb8dcc5d822e4
2023-06-21 16:43:00 -07:00
James Zern
d49cfbb348 vp8l_enc,WriteImage: add missing error check
VP8LBitWriterFinish() may cause the VP8LBitWriter's buffer to be grown.
If that allocation fails, VP8LBitWriterNumBytes() will return a size
larger than the current allocation resulting in a heap overwrite of the
missing bytes.

==3531848==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x61d000000880 at pc 0x556eddfa1007 bp 0x7ffe434c7a90 sp 0x7ffe434c7260
READ of size 2052 at 0x61d000000880 thread T0
    #0 0x556eddfa1006 in __asan_memcpy
    #1 0x556eddfeeccf in WebPMemoryWrite src/enc/picture_enc.c:220:5
    #2 0x556ede0f9f87 in WriteImage src/enc/vp8l_enc.c:1454:8

Found by Nallocfuzz (https://github.com/catenacyber/nallocfuzz).

Change-Id: Ib1c9454c2c51849b0ba58c5347e6bd5b02a12fbe
2023-06-15 15:09:10 -07:00
James Zern
0edbb6ea71 PaletteSortModifiedZeng: fix leak on error
Change-Id: I462bd9a3bc4670efdf251c295f6771a38c08a6ce
2023-02-23 22:03:31 -08:00
James Zern
f853685e13 lossless: SUBTRACT_GREEN -> SUBTRACT_GREEN_TRANSFORM
this makes the name of the TransformType enum value match the other
members

Bug: webp:448
Change-Id: I85b2f615f97b40fc6d544197cccfb7189dcf4fc0
2022-11-21 16:48:51 -08:00
James Zern
00ff988a59 vp8l_enc,AddSingleSubGreen: clear int sanitizer warnings
this localizes the conversion to int in the function; the parameter was
previously changed in:
6ab496ed fix some 'unsigned integer overflow' warnings in ubsan

implicit conversion from type 'uint32_t' (aka 'unsigned int') of value
2350919223 (32-bit, unsigned) to type 'int' changed the value to
-1944048073 (32-bit, signed)

Bug: b/229626362
Change-Id: I589eec11c0dabaeba99e153e705f956181c570d2
2022-08-08 17:41:25 -07:00
Vincent Rabaud
a19a25bb03 Replace doubles by floats in lossless misc cost estimations.
Doubles are slower and use more RAM for no benefit.

Change-Id: I05b313576f9b33388c7c39d7fed8de84170c3753
2022-04-17 21:07:54 +02:00
James Zern
2de4b05a56 Merge changes Id9890a60,I376d81e6,I1c958838 into main
* changes:
  GetBackwardReferences: fail on alloc error
  BackwardReferencesHashChainDistanceOnly: fix segfault on OOM
  VP8LEncodeStream: fix segfault on OOM
2022-04-06 20:04:25 +00:00
James Zern
10c5084842 normalize label indent
1 space is most common in the source; this fixes some mixed cases within
lossless files, likely from clang-format

Change-Id: I504206d5bf418781d4131ee73570ecee4e0a69a1
2022-04-04 12:47:41 -07:00
James Zern
fe153fae98 VP8LEncodeStream: fix segfault on OOM
initialize bw_side before calling EncoderAnalyze() & EncoderInit() which
may fail; previously this would cause a free of an invalid pointer in
VP8LBitWriterWipeOut().

since at least:
v0.6.0-120-gf8c2ac15 Multi-thread the lossless cruncher.

Tested:
for i in `seq 1 639`; do
  export MALLOC_FAIL_AT=$i
  ./examples/cwebp -m 6 -q 100 -lossless jpeg_file
done

Bug: webp:565
Change-Id: I1c95883834b6e4b13aee890568ce3bad0f4266f0
2022-04-04 11:21:30 -07:00
Yannis Guyon
ec178f2c7f Add progress hook granularity in lossless
A WebPPicture instance is necessary to call WebPReportProgress() which
sets WebPPicture::error_code so as well use WebPEncodingSetError() to
record errors too, instead of functions returning a WebPEncodingError.
However there must be one WebPPicture instance per thread, with error
codes merged at sync time. A mutex could simplify that but it is not
the objective of this change.

https://groups.google.com/a/webmproject.org/g/webp-discuss/c/yOiP8APubgc/m/vCTvxl6ODgAJ

Change-Id: Ia1a8f9d1199202e1c88484ce719b0180a80447ce
2022-02-07 11:00:09 +01:00
James Zern
b60869a18e Revert "Do not use a palette for one color images."
This reverts commit b6513fbaa8a3e21d1d726f6c63bbfa76f157b9a2.

This change can produce files that can cause decode failures in some
versions of chrome and safari/ios/macos.

https://chromium-review.googlesource.com/c/chromium/src/+/2876279

The chrome fix will be available in M92. This change can be revisited
after it and the mac updates are more widely deployed.

Bug: b/186640109,b/188702956
Change-Id: I296b8fe88c6c48219e3edf532226c4f972f1605b
2021-05-21 10:26:37 -07:00
James Zern
05b72d4205 vp8l_enc.c: normalize index types
fixes conversion warnings in visual studio after:
b1674240 Add modified Zeng's method to palette sorting.

src\enc\vp8l_enc.c(296) : warning C4244: '=' : conversion from 'const
uint16_t' to 'uint8_t', possible loss of data
src\enc\vp8l_enc.c(299) : warning C4244: '=' : conversion from 'const
uint16_t' to 'uint8_t', possible loss of data

Change-Id: I981b1ba4912edbbafbd49f1f5b1043bf12266920
2021-04-28 15:09:34 -07:00
Vincent Rabaud
b6513fbaa8 Do not use a palette for one color images.
1 color images now always take 30 bytes.

Change-Id: Ifa86bc5320362c659672b3836160353d63576467
2021-04-28 13:14:28 +02:00
Vincent Rabaud
98bbe35b51 Fix multi-threading with palettes.
Change-Id: Ifa23378c1f9c489d5963b4928781e104a14eb01a
2021-04-27 16:54:03 +02:00
Vincent Rabaud
b1674240f9 Add modified Zeng's method to palette sorting.
Also add palette sorting to crunch configurations.

Change-Id: I010a8bf8f1921279db6e9c7209307d8d19a4d105
2021-04-27 13:11:18 +02:00
Vincent Rabaud
8fdaecb09d Disable cross-color when palette is used.
With palette+predictors, cross-color was forced (because of predictors).
No need for cross-color for palettes as R/B==0.
This saves 10 bytes per image that uses palette+predictors.

Change-Id: If2184d16cdabe1e8498009062284ad3e37ef1342
2021-04-23 17:43:36 +02:00
Vincent Rabaud
cf2f88b38f Add palette and spatial for q >= 75 and -m 5
Change-Id: I12198b7eb82a4247e606bc60342595abf4d6eee0
2020-06-17 12:51:40 +02:00
Vincent Rabaud
f0110baec0 Add no-color cache configuration to the cruncher
Fix another pessimization found by the pingo image compressor.

Refactoring is necessary to make LZ77 computation
common to cache or no-cache analysis.
Slower by 1.7x instead of 2x

Change-Id: I396701ea6e88543dbfe9471eb552877f6c8ce1e3
2020-06-09 19:04:44 +02:00
Vincent Rabaud
4f9f00ccf4 Use spatial predictors on top of palette no matter what.
This is fixing another inefficiency found by the pingo image optimizer.

Change-Id: Icecb0d39fcbd17b403667e8e2095c7705b1dd493
2020-06-04 18:03:57 +02:00
Vincent Rabaud
7658c68613 Add spatial prediction on top of palette in cruncher.
Change-Id: I3765ab628ef915eedf2e541a80c65ce9880dff36
2020-06-04 14:38:08 +02:00
Vincent Rabaud
f95a996c64 Simpler histogram clustering.
Instead of re-organizing the list of histograms, set
the unused ones to NULL.

Change-Id: I8d25e1bb8f78ae9486ff358cc647ba1821cd5fcf
2018-11-11 10:59:34 +01:00
Vincent Rabaud
6752904b2f Speed-up: Make sure we only initialize histograms when needed.
Also, histograms in a HistogramSet can be initialized all
at once.

Change-Id: Ibbfa6034dce58dca8bb9113487e2ae507222ce7d
2018-10-31 11:54:09 +00:00
Pascal Massimino
92dc0f0937 clean-up MakeInputImageCopy()
use pointer increments.

Change-Id: I269412d41a58ab9ffd7fc0f3d479fe73a3d07b9e
2018-10-02 14:46:35 +00:00
Pascal Massimino
994be82d00 Merge "Remove some very hard TODOs." 2018-04-03 14:10:50 +00:00
Vincent Rabaud
4033e1d70d Remove some very hard TODOs.
Change-Id: I3d1b0072e0ac9125840fbbd76e91d151c82489ec
2018-04-03 15:17:21 +02:00
Vlad Tsyrklevich
978eec2507 [CFI] Remove function pointer casts
Control Flow Integrity [1] indirect call checking verifies that function
pointers only call valid functions with a matching type signature. This
change eliminates function pointer casts that were causing cfi-icall
failures.

[1] https://www.chromium.org/developers/testing/control-flow-integrity

BUG=chromium:827826

Change-Id: I5db021d06390a6cefd670fdd2f0d34c9e530465e
2018-04-02 16:04:47 -07:00
Vincent Rabaud
551948e45f Remove unused argument in VP8LBitsEntropy.
The function is only used once and does not use the extra argument.

Change-Id: I9735383784746cb02b5a643b7a4a2037f2874bf9
2018-02-16 16:05:28 +01:00
James Zern
f4dd92565e remove WEBP_EXPERIMENTAL_FEATURES
the webp bitstream is considered stable at this point

Change-Id: I4b13f9ed4c45f63785474b097e96cb7bf651be7b
2018-02-09 10:25:11 -08:00
James Zern
083507f244 right-size some tables
Change-Id: I5e894fd3f0f4b666512108495d8cecb34f65e119
2017-11-23 08:40:58 +00:00
James Zern
f593d71a64 enc: disable pic->stats/extra_info w/WEBP_DISABLE_STATS
Change-Id: I4ca3fa45710bd6bbe131b2ae047b1e268241657c
2017-11-22 17:04:01 -08:00
Vincent Rabaud
c462cd0065 Remove useless code.
The casts are to the same type and the #define not used.

Change-Id: I8d69c3b9dde7a1c53c2ba5a026a653d8c2e1d2a7
2017-11-08 10:52:49 +01:00
James Zern
a439972175 WIP: list includes as descendants of the project dir
#include "(.|..)/..." -> #include "src/..."

Change-Id: I772880aa097a770722043c8a4393552ba38a89b6
2017-10-10 23:04:05 -07:00
Vincent Rabaud
a80fcc4ae1 ifdef code not used by Chrome/Android.
Change-Id: Id086f6fd602b1fe3dc9034764b6a920a696ff1d2
2017-08-31 14:02:05 +02:00
Vincent Rabaud
e132072f0b Re-use the transformed image when trying several LZ77 in lossless.
Change-Id: If879b960131ba26859f8e4719a4c74cec1e0aaa8
2017-06-19 17:30:01 +02:00