114 Commits

Author SHA1 Message Date
Vincent Rabaud
8874b16275 Fix a non-deterministic color cache size computation.
In case of impossible allocation, some value was returned while
computation should be stopped.

Change-Id: I5f85e264575be825e4261ab6fa63840c157cf5c2
2017-01-10 18:53:19 +01:00
Vincent Rabaud
11bc423ae5 MIN_LENGTH cleanups.
No change in logic so no change in speed or compression.

Change-Id: I744161978c7d058c9b58450f330cba11731530c6
2016-10-10 15:37:45 +02:00
Vincent Rabaud
5f1caf2987 Small LZ77 speedups.
The most common conditions are re-ordered and cached.

iter_min was recently introduced to make sure enough iterations
are made in cases where there are many matches (mostly uniform regions).
Now that those are properly analyzed, it becomes useless.

Change-Id: Id3010ee4ec66b84d602fcb926f91eb9155ad27f4
2016-09-22 14:03:25 +02:00
Vincent Rabaud
c9b45863e2 Split off common lossless dsp inline functions.
Change-Id: I64f96897b11d1c21f033c7e47b21edccb5c68738
2016-09-12 17:35:08 +02:00
Vincent Rabaud
85cd5d061c Smarter LZ77 for uniform regions.
No need to find backward references for pixels in uniform regions
by looking at all pixels.
Only pixels at the same distance from the end need to be compared to.

Change-Id: I4f187e965f0667d3a929775726a412f7e69f6473
2016-08-26 09:53:49 +02:00
skal
6ab496ed22 fix some 'unsigned integer overflow' warnings in ubsan
I couldn't find a safe way of fixing VP8GetSigned() so i just
used the big-hammer.

Change-Id: I1039bc00307d1c90c85909a458a4bc70670e48b7
2016-08-16 23:18:27 -07:00
James Zern
8a4ebc6ab0 Revert "fix 'unsigned integer overflow' warnings in ubsan"
This reverts commit e44f5248ff4b9d27d76edaff93128046a517b5e8.

contains unintentional changes in quant.c

Change-Id: I1928f072566788b0c9ea80f6fbc9e571061f9b3e
2016-08-16 16:55:56 -07:00
skal
e44f5248ff fix 'unsigned integer overflow' warnings in ubsan
I couldn't find a safe way of fixing VP8GetSigned() so i just
used the big-hammer.

Change-Id: I1039bc00307d1c90c85909a458a4bc70670e48b7
2016-08-16 15:04:41 -07:00
hui su
1269dc7cfb Refactor VP8LColorCacheContains()
Return key/index if the query is found, and -1 otherwise.
The benefit of this is to save a hashing computation.

Change-Id: Iff056be330f5fb8204011259ac814f7677dd40fe
2016-08-12 15:16:06 -07:00
Vincent Rabaud
2a5c417c68 Apply the RLE heuristic to LZ77.
Change-Id: I7317eed7e017ee8981f40fcf1737f97e0e3a238c
2016-07-14 20:12:48 +02:00
Vincent Rabaud
c7eb06f737 Fix corner case in CostManagerInit.
Change-Id: I91795d05eb78816d6d9a8cadc64d3814650d2aee
2016-06-27 20:01:44 +02:00
Vincent Rabaud
319e37be13 Improve lossless compression.
This is essentially a revert of a3611513d2bf465fd282d9dc45b3f72c79c232ad
and cfbcc5ece022fc74ae9b987e05c2807df0d82ec5.
Here is what happened: there was a corruption bug that eventually
got fixed by 0174d18d8b51f6c9228c70066a987c30a8132995.
But before finding the root, a3611513d2bf465fd282d9dc45b3f72c79c232ad
and cfbcc5ece022fc74ae9b987e05c2807df0d82ec5 hid the bug
by not imposing length of 1 when it was actually 2 or 3 (which does help
compression as a litteral is more efficient than an offset and a length
of size 2 or 3).

Change-Id: I6f18fc1f583a51ac9d8aab2508458264047cd493
2016-06-24 16:11:25 +02:00
Vincent Rabaud
7d58d1b7b9 Speed-up uniform-region processing.
Change-Id: I9a88d0ac97c31d19323c9505ebe21f375d2e96b8
2016-06-21 15:45:46 +02:00
Vincent Rabaud
0174d18d8b Fix a boundary case in BackwardReferencesHashChainDistanceOnly.
The optimization for (len != MIN_LENGTH) actually only holds for
(len > MIN_LENGTH) but (len < MIN_LENGTH) can now happen as len can
be changed in the loop before.

Change-Id: I3f9f91a540206c80385c5fba96c3d64ab9536752
2016-06-16 19:22:28 +02:00
Vincent Rabaud
cfbcc5ece0 Make sure to consider small distances in LZ77.
This could corrupt certain images since commit
a3611513d2bf465fd282d9dc45b3f72c79c232ad

Change-Id: Ifbe43abaafe8efb27c62af18039fea5a9dc4e062
2016-06-16 09:14:11 +00:00
Pascal Massimino
f2a0946a7a add some asserts to delimit the perimeter of CostManager's operation
a small protection in a fairly complex code.

Change-Id: I920e10e1fc1c35da2cf486349417048d516ff2b9
2016-06-15 20:55:32 +02:00
James Zern
6fda58f137 backward_references: quiet double->int warning
since:
059aab4 Fix a compression regression for images with long uniform
regions.

Change-Id: I1783a74220961e8bc3bb42696e3412fe4bfc4ddb
2016-06-14 15:27:58 -07:00
Pascal Massimino
a48cc9d201 Merge "Fix a compression regression for images with long uniform regions." into 0.5.1 2016-06-14 21:26:06 +00:00
Pascal Massimino
cc2720c1d5 Merge "Revert an LZ77 boundary constant." into 0.5.1 2016-06-14 21:25:08 +00:00
Vincent Rabaud
059aab4fa1 Fix a compression regression for images with long uniform regions.
Change-Id: Id87a4ac2a22daaa71e8f3132e69703b9b3ddd752
2016-06-14 21:51:10 +02:00
Vincent Rabaud
b0c7e49e58 Check more backward matches with higher quality.
Change-Id: I3f0887b0b9b7f0e69758f51783807e1583b74be2
2016-06-14 21:50:03 +02:00
Vincent Rabaud
a3611513d2 Revert an LZ77 boundary constant.
This is getting back to the old behavior which is actually better for
compression and speed with the latest patches.

Change-Id: I35884bab02589297c25d6e1e66dc5f13e05f7aa7
2016-06-14 21:42:45 +02:00
Pascal Massimino
7685123a7f fix comment typos
Change-Id: I2a55e371dbf7e62b446f6bb732c8913b85633c49
2016-06-10 13:29:07 +00:00
Vincent Rabaud
a246b921a0 Speedup backward references.
In case where the same offset is found in consecutive pixels,
the cost computation from one pixel can be re-used for the next.

Change-Id: Ic03c7d4ab95f3612eafc703349cfefd75273c3d7
2016-06-09 20:05:15 +02:00
Pascal Massimino
76d73f1835 Merge "CostManager: introduce a free-list of ~10 intervals" 2016-06-09 15:58:38 +00:00
Pascal Massimino
eab39d8147 CostManager: introduce a free-list of ~10 intervals
and also recycle the malloc'd intervals
This avoids quite some malloc/free cycles during interval managment.

Change-Id: Ic2892e7c0260d0fca0e455d4728f261fb4c3800e
2016-06-09 01:50:50 -07:00
Vincent Rabaud
0ba7fd70c6 Improve speed and compression in backward reference for lossless.
Change-Id: I664c5e68b036a2d424192962dbad873a2c70b826
2016-06-08 21:13:33 +02:00
Pascal Massimino
0481d42ad8 CostManager: cache one interval and re-use it when possible
In a lot of cases, only one interval is used. This can cause
a lot of malloc/free cycles for only 56 bytes. By caching this
single interval and re-using it, we remove this cycle in most
frequent cases.

Change-Id: Ia22d583f60ae438c216612062316b20ecb34f029
2016-06-08 14:10:06 +00:00
Vincent Rabaud
d963775859 Compute the hash chain once and for all for lossless compression.
In some cases, the hash chain for a function is filled several
times:
- GetBackwardReferences -> CalculateBestCacheSize ->
BackwardReferencesLz77 that computes the hash chain
- GetBackwardReferences ->
(not always) BackwardReferencesTraceBackwards ->
BackwardReferencesHashChainDistanceOnly that computes the hash
chain in a slightly different way

Speed and compression performance are slightly changed (+ or -)
but will be homogneized in a later patch.

Change-Id: I43f0ecc7a9312c2ed6cdba1c0fabc6c5ad91c953
2016-06-03 11:42:13 +02:00
Vincent Rabaud
3e023c17cd Speed-up BackwardReferencesHashChainDistanceOnly.
Instead of comparing all the following pixels over len (which can
frequently reach the maximum MAX_LENGTH=4096 for some images),
intervals are stored and compared.

Change-Id: I0dafef6cc988dde3c1c03ae07305ac48901d60ee
2016-05-19 04:51:13 +00:00
Vincent Rabaud
8ce975ac82 SSE optimization for vector mismatch.
Change-Id: I564b822033b59d86635230f29ed6197e306a2c4f
2016-01-07 18:23:45 +01:00
Vincent Rabaud
6c702b81ac Speed up hash chain initialization using memset.
That gains 1% on lossy compression.

Change-Id: Ib9aa210194ed2f17eaff85b499b55cc4eb99ff11
2015-12-07 11:54:50 +01:00
Vincent Rabaud
010ca3d10d Fix FindMatchLength with non-aligned buffers.
The 32-bit buffers are actually rarely 64-bit aligned.
The new solution uses memcmp and is alignment agnostic.
It is also slightly faster.

Change-Id: I863003e9ee4ee8a3eed25b7b2478cb82a0ddbb20
2015-12-04 10:19:58 +01:00
Scott Hancher
5ae220bef6 backward_references.c: Fixed compiler warning
"Implicit conversion loses integer precision: 'long' to 'int'."

Change-Id: I1aec7431f84123e5280447883eb80b84a3821d91
2015-12-02 23:51:06 -08:00
Vincent Rabaud
a141178255 Optimization in hash chain comparison for 64 bit
Arrays were compared 32 bits at a time, it is now done 64 bits at a time.
Overall encoding speed-up is only of 0.2% on @skal's small PNG corpus.
It is of 3% on my initial 1.3 Mp desktop screenshot image.

Change-Id: I1acb32b437397a7bf3dcffbecbcd4b06d29c05e1
2015-12-01 13:01:57 +01:00
Jyrki Alakuijala
90fcfcd905 Insert less hash chain entries from the beginnings of long copies.
This makes the chains more efficient and a larger variety of data is tested.

0.02 % compression gain at q 100, 0.05 % at default quality. 0.8 % speedup by
callgrind.

0.16 % compression gain for lossy alpha ?!

Change-Id: I888120133352799eb14f5f602c7f40ab404bd665
2015-08-18 18:44:03 -07:00
Jyrki Alakuijala
01d61fd9c6 lossless: ~20 % speedup
0.28 % byte size increase on lossless, 0.18 % increase on lossy alpha

Change-Id: I1e001a56831a8f996ac522aa646f9ae587c80d12
2015-07-20 17:13:44 -07:00
Jyrki Alakuijala
f722c8f0bd lossless: Speed up ComputeCacheEntropy by 40 %
a total impact of 1 % on encoding speed

This allows for performance neutral removal of the binary search
in cache bits selection. This will give a small improvement in
compression density.

Change-Id: If5d4d59460fa1924ce71af977320834a47c2054a
2015-07-20 17:13:44 -07:00
Jyrki Alakuijala
17eb609916 lossless: Allow copying from prev row in rle-mode.
0.21 % compression density improvement for 1000 png corpus in
lossless mode

0.50 % compression density improvement for 1000 png corpus in
lossy mode

Change-Id: I14ee8c427ae5d3e116b0ee6695fcdea3321a319d
2015-07-20 17:13:43 -07:00
Jyrki Alakuijala
c4855ca249 lossless: Inlining add literal
this is a simple speedup of about 1-2 %

Change-Id: I0c7b01c0a69f4aeaf363ffda05a28871f1def696
2015-07-07 20:24:28 -07:00
Jyrki Alakuijala
8e9c94dedb lossless: simplify HashChainFindCopy heuristics
for small speedup
0.0003 % worse compression

Change-Id: Ic4b6b21e5279231c6321f2cec1c79f7e17e56afa
2015-07-07 20:24:27 -07:00
Jyrki Alakuijala
888429f409 lossless: 0.5 % compression density improvement
do not do length 2 matches far away

speedup for non compressible data by inserting two literals at a time
when no matches are found

Change-Id: Ia8e033071f4186bb8148bb2bf13ca37586734aa3
2015-07-07 20:24:27 -07:00
Jyrki Alakuijala
5e75642efd lossless: rle mode not to accept lengths smaller than 4.
Gives a compression gain of 0.22 %

Change-Id: I0f3b8dad6b4c1bfb16eab095a467f34466b9e3b7
2015-07-07 20:24:25 -07:00
Pascal Massimino
7fa67c9b9e change GetPixPairHash64() return type to uint32_t
Change-Id: Ibb61c1631d7a4bcda5417b5a85864d5e2c3f3858
2015-04-16 00:55:25 -07:00
Pascal Massimino
7fe357b8c0 split 64-mult hashing into two 32-bit multiplies
Speed-wise equivalent on x86 and ARM (maybe a tad faster, hard to tell).

Note that the two 32-bit multiples are not strictly equivalent
to the 64-bit one, since we're missing one carry propagation.
In practice, no observable difference was seen because of this
slightly different hashing result.

Change-Id: I8f2381175eae1cb20dabf149e6b27e1768fba6ab
2015-04-15 17:45:19 +02:00
Vikas Arora
4d6d7285b0 Simplify backward refs calculation for low-effort.
Simplify and speedup backward references for low-effort settings by evaluating
LZ77 references only. This change speeds up compression by 10-25% at lower
(q <= 25) quality range with a slight drop (0.2%) in the compression density.

Change-Id: Ibd6f03b1a062d8ab9191786c2a425e9132e4779f
2015-01-27 09:36:14 -08:00
Pascal Massimino
0d5b334ee8 BackwardReferencesHashChainFollowChosenPath: remove unused variable
Change-Id: I8dc4622dbacca03a7876f8856a0db5b9b9ec2fbd
2015-01-22 23:22:58 -08:00
Pascal Massimino
cb4a18a7ba rename HashChainInit into HashChainReset
this avoids the confusion with "VP8LHashChainInit"

Change-Id: Ia1686828c138729e5bda3cc5c8246d99c80915ef
2015-01-20 00:38:07 -08:00
Pascal Massimino
f079e487ae use uint16_t for chosen_path[]
len is MAX_LENGTH (4096) at max. This reduce memory for path by a half.

Change-Id: I399fda4093d93b1e9d956397b7b210956c5b948f
2015-01-20 00:34:09 -08:00
Vikas Arora
b9e356b998 Disable costly TraceBackwards for method=0.
Disable costly TraceBackwards heuristic for computing the backward references
for low_effort (method=0) compression.

The TraceBackwards heuristic is already disabled for lower (q < 25) quality
range. Following is the compression data for 1000 image corpus for q >= 25.

This speeds up compression (q >= 25) by a factor of 2.5-3X with slight loss of
compression density (0.7% for lower quality range and 1.2% for higher qualities).

Change-Id: I256c9e2137c7de4083f423ea32ee12d3b0f46253
2015-01-15 09:01:40 -08:00