Commit Graph

3946 Commits

Author SHA1 Message Date
James Zern
e38602d2ad Merge branch 'lossless_encoder'
* lossless_encoder: (46 commits)
  split StoreHuffmanCode() into smaller functions
  more consolidation: introduce VP8LHistogramSet
  big code clean-up and refactoring and optimization
  Some cosmetics in histogram.c
  Approximate FastLog between value range [256, 8192]
  Forgot to update out_bit_costs to symbol_bit_costs at one instance.
  Evaluate output cluster's bit_costs once in HistogramRefine.
  Simple Huffman code changes.
  Lossless decoder: remove an unneeded param in ReadHuffmanCodeLengths().
  Reducing emerging palette size from 11 to 9 bits.
  Move GetHistImageSymbols to histogram.c
  Improve predict vs no-predict heuristic.
  code-moving and clean-up
  reduce memory usage by allocating only one histo
  Restrict histo_bits to ensure histo_image size is under 32MB
  further simplification for the meta-Huffman coding
  A quick pass of cleanup in backward reference code
  Make transform bits a function of encode method (-m).
  introduce -lossless option, protected by USE_LOSSLESS_ENCODER
  Run TraceBackwards for higher qualities.
  ...

Conflicts:
	src/enc/webpenc.c

Change-Id: I9a5d98cba0889ea91d10699466939cc283da345a
2012-05-07 14:27:17 -07:00
Pascal Massimino
e8d3d6a018 split StoreHuffmanCode() into smaller functions
Change-Id: Iaa715f4997505eebabee1e92e964a5d7ee6f3e7d
2012-05-07 14:25:32 -07:00
Pascal Massimino
d0d88990d8 more consolidation: introduce VP8LHistogramSet
VP8LHistogramSet is container for pointers to histograms that
we can shuffle around. Allocation is one big chunk of memory.
Downside is that we don't de-allocate memory on-the-go during
HistogramRefine().

+ renamed HistogramRefine() into HistogramRemap(), so we don't
confuse with "HistogramCombine"
+ made VP8LHistogramClear() static.

Change-Id: Idf1a748a871c3b942cca5c8050072ccd82c7511d
2012-05-07 14:25:30 -07:00
Pascal Massimino
1a210ef1a9 big code clean-up and refactoring and optimization
* de-inline some function
* make VP8LBackwardRefs be more like a vectorwith max capacity
* add bit_cost_ field to VP8LHistogram
* general code simplifications
* remove some memmov() from HistogramRefine
* simplify HistogramDistance()
...

Change-Id: I16904d9fa2380e1cf4a3fdddf56ed1fcadfa25dc
2012-05-07 14:25:29 -07:00
Urvang Joshi
41b5c8ff71 Some cosmetics in histogram.c
Change-Id: I5d5872a793759fad593dba88c3f593f72b328b0c
2012-05-07 14:25:28 -07:00
Vikas Arora
ada6ff77df Approximate FastLog between value range [256, 8192]
Profiled data: Profiled few images and found that in the function VP8LFastLog,
90% of time table lookup is performed, while rest of time (10%) call to log
function is made. Typical lookup accounts for 10 CPU instructions and call to
log 200 instruction counts. The weighted average comes out to be 30
instructions per call. For mid qualities (25-75), this function (VP8LFastLog)
accounts for 30-50% of total CPU cycles (via call path: VP8LCOlorSpaceTransform
-> PredictionCostCrossColor -> ShannonEntropy). After this change, the log is
called less that 1% of time, with average instructions being 15 per call.
Measured the performance over 1000 files for various qualities and found
overall compression speedup between 10-15% (in quality range [0, 75]). The
compression density loss is around 0.5% (though at some qualities, compression
is little better as well).

Change-Id: I247bc6a8d4351819c871f19d65455dc23aea8650
2012-05-07 14:25:26 -07:00
Vikas Arora
ec123ca3f6 Forgot to update out_bit_costs to symbol_bit_costs at one instance.
Change-Id: Iaf952c0cb6e3fe35257d2503a16a437c6f2eb3aa
2012-05-07 14:25:25 -07:00
Vikas Arora
cf33ccd160 Evaluate output cluster's bit_costs once in HistogramRefine.
Avoid bit_costs evaluated every time in function HistogramDistance. Also moved
VP8LInitBackwardRefs and VP8LClearBackwardRefs to backward_references.h

Change-Id: Id507f164d0fc64480aebc4a3ea3e6950ed377a60
2012-05-07 14:25:23 -07:00
Vikas Arora
781c01f421 Simple Huffman code changes.
No empty trees are codified with the simple Huffman code. The simple Huffman
code is simplified to be either a 1-bit code or 8-bit code for symbols.

Change-Id: I3e2813027b5a643862729339303d80197c497aff
2012-05-07 14:25:21 -07:00
Urvang Joshi
a2849bc502 Lossless decoder: remove an unneeded param in ReadHuffmanCodeLengths().
Change-Id: I279452fdf38b680737d5ba6e868a219281bc8962
2012-05-07 14:25:20 -07:00
Vikas Arora
b39e7487a7 Reducing emerging palette size from 11 to 9 bits.
This is required to reduce memory used to construct histo_image.

Change-Id: I491a06e10a3e3f3d8a00ecec286394378283ffea
2012-05-07 14:25:19 -07:00
Vikas Arora
bfc73db4a8 Move GetHistImageSymbols to histogram.c
Planning to revisit memory allocation scheme at several instanaces (next CL).

Change-Id: Id7b9f4854e9577e10b53d5b1d8595b7d862e6e01
2012-05-07 14:25:18 -07:00
Urvang Joshi
889a578681 Improve predict vs no-predict heuristic.
This improves compression density. For example, at quality 95 on 1000 PNGs:
bpp(before) = 2.447 and bpp(after) = 2.412

Change-Id: I19c343ba05cca48a6940293721066502a5c3d693
2012-05-07 14:25:16 -07:00
Pascal Massimino
01f50663dc code-moving and clean-up
* removed use_lz77_ field, and added cache_bits_ one.
* use more BackwardRefs params
* move code around to organize more logically
* reduce memory use on histo
...

Change-Id: I833217a1b950189cf486704049e3fe28382ce335
2012-05-07 14:25:14 -07:00
Pascal Massimino
31035f3b49 reduce memory usage by allocating only one histo
instead of lz77+rle

* introduce VP8LBackwardRefs structure and simplify the code by not passing
  around {PixOrCopy/int} pairs.

More functions should be turned into using this struct (TODO(later)).

Change-Id: I69c5c9fa61dddd61a2abc2824d70b8606a1c55b6
2012-05-07 14:25:12 -07:00
Vikas Arora
fbb501b8ee Restrict histo_bits to ensure histo_image size is under 32MB
Change-Id: I75ccb65d56ee060b649de714287e71611a92c8e9
2012-05-07 14:25:10 -07:00
Pascal Massimino
8415ddf3be further simplification for the meta-Huffman coding
* don't transmit the number of Huffman tree group explicitly
* move color-cache information before the meta-Huffman block

* also add a check that color_cache_bits is in [1..11] range, as per spec.

Change-Id: I81d7711068653b509cdbc1151d93e229c4254580
2012-05-07 14:25:07 -07:00
Urvang Joshi
e491729905 A quick pass of cleanup in backward reference code
const correctness, renaming, cosmetics etc.

Change-Id: I432befbb22f0eafd9a613f5f632398b6ef03c0f6
2012-05-07 14:25:05 -07:00
Vikas Arora
83332b3c16 Make transform bits a function of encode method (-m).
Change-Id: Idc392f7cba6e160ea068eacd7f82be4ebc971eaa
2012-05-07 14:25:03 -07:00
Pascal Massimino
72920caa61 introduce -lossless option, protected by USE_LOSSLESS_ENCODER
Change-Id: Ic5707082fefd964b9bcab0c9f9aa276992c22b06
2012-05-07 14:25:02 -07:00
Vikas Arora
c6ac4dfbb4 Run TraceBackwards for higher qualities.
Also reduce the iteration count in function VP8LHashChain_FindCopy.

Change-Id: I73e3811e142e81314515587fd655ab3bfa74d099
2012-05-07 14:25:00 -07:00
Vikas Arora
412222c88c Make histo_bits and transform_bits function of quality.
Change-Id: Ic34e40853604811abc63a38e09d6a01961649efc
2012-05-07 14:24:58 -07:00
Urvang Joshi
149b5098a9 Update lossless encoder strategy:
Don't use any other transform when using palette.

Change-Id: I488ac546869677f1b6e4eed80e973569c757e997
2012-05-07 14:24:56 -07:00
Urvang Joshi
0e6fa06595 cache_bits passed to EncodeImageInternal()
should be 0 when not using color cache.

Change-Id: Id15c9b6bfbeb7c69a0189fa4c411df2763f9dead
2012-05-07 14:24:54 -07:00
Vikas Arora
e38b40a996 Factorize code for clearing HtreeGroup.
Change-Id: I29de6dab7383b8cf071eec155e01340d5fdadee5
2012-05-07 14:24:53 -07:00
Vikas Arora
6f4a16ea00 Removing the indirection of meta-huffman tables.
Now, the indexing refers directly to 5 huffman codes that must be encoded
separately.

Change-Id: I92e10ccf8ca464aa7259867d5fae2869343e3b3c
2012-05-07 14:24:51 -07:00
Urvang Joshi
3d33ecd12b Some renaming/comments related to palette in lossless encoder.
Change-Id: Iaab32912f4c31e809d7a49fd748099d8c0c3e7d9
2012-05-07 14:24:49 -07:00
Urvang Joshi
4d02d5863f Lossless encoder: correction in Palette storage
(Essentially, there was no need of a separate 'argb_palette' array. And
argb_palette[0] was never being set).

Change-Id: Id0a8c7e063d3af41e39fc9b8661611b51ccc55cd
2012-05-07 14:24:47 -07:00
Urvang Joshi
4a6362357a fix a memleak in EncodeImageInternal()
Change-Id: I55cd013211f192188b54c694ef0837af0c01b53c
2012-05-07 14:24:45 -07:00
Urvang Joshi
0993a611cd Full and final fix for prediction transform
use (tile_size + 1) rows of scratch area.

Change-Id: I06d612fff1794fc045ba76275e94e7210802c332
2012-05-07 14:24:43 -07:00
Urvang Joshi
afd2102f43 Fix cross-color transform in lossless encoder
make elements of "Multiplier" struct unsigned, so that any negative values are
automatically converted to "mod 256" values.

Change-Id: Iab4f9bacc50dcd94a557944727d9338dbb0982f7
2012-05-07 14:24:41 -07:00
Urvang Joshi
b96d8740c9 Need to write a '0' bit at the end of transforms.
Also convert an 'if' to 'assert'.

Change-Id: Ia1129ad9ddb027c27b4f4fc1da4bbaf53a0a8f76
2012-05-07 14:24:39 -07:00
Urvang Joshi
54dad7e553 Color cache size should be counted as 0 when cache bits = 0
Change-Id: I1d05e0561a92aebaf62162fe11ffc4b12496d698
2012-05-07 14:24:37 -07:00
Urvang Joshi
4f0c5caf67 Fix prediction transform in lossless encoder.
(Keep one tile as a scratch buffer).

Change-Id: If112ada29bfd0bdc81b82e849a566b30dd331d2f
2012-05-07 14:24:35 -07:00
Vikas Arora
36dabdadf8 Fix memory leak in method EncodeImageInternal for histogram_image.
Change-Id: Ia1cfb96d9e6c120630732e2b5f39688376d1d208
2012-05-07 14:24:33 -07:00
Urvang Joshi
352a4f49ab Get rid of PackLiteralBitLengths()
[and in turn a malloc]. Also, a few related const fixes.

Change-Id: I229519b1c34d41c78d9ad2403f1e25feab3c9d93
2012-05-07 14:24:31 -07:00
Vikas Arora
d673b6b9a0 Change the predictor function to pass left pixel
instead of pointer to the source.

Change-Id: Ia2c8e17c3140709a825c2f85a88c5e31bd6e462f
2012-05-07 14:24:29 -07:00
Urvang Joshi
b2f99465a7 Fix CopyTileWithPrediction()
so that it uses original values of left, top etc for prediction rather than the
predicted values of the same. Also, do some renaming in the same to make it
more readable.

Change-Id: I2fe94e35a6700bd437f5c601e2af12323bf32445
2012-05-07 14:24:27 -07:00
Vikas Arora
84547f540c Add EncodeImageInternal() method.
Most of changes in enc/vp8l.c is cherry-picked from src/lossless/encode.c

Change-Id: I27938cb2590eccbfe1db0a454343e856bd483e75
2012-05-07 14:24:25 -07:00
Urvang Joshi
6b38378acb Guard the lossless encoder (in flux) under a flag
Change-Id: I6dd8fd17089c199001c06b1afde14233dc3e3234
2012-05-07 14:24:23 -07:00
Vikas Arora
09f7532cce Fix few nits (const qualifiers)
Change-Id: I527e82af49956b695ab18625d34e143854067421
2012-05-07 14:24:21 -07:00
Vikas Arora
648be3939f Added implementation for various lossless functions
- VP8LEncAnalyze, EvalAndApplySubtractGreen, ApplyPredictFilter,
  ApplyCrossColorFilter
- Added palette handling and transform buffer management in VP8LEncodeImage()
- Add Transforms (subtract Green, Predict, cross_color) to dsp/lossless.c.

These are more-or-less copied from src/lossless code.

After this Change, will implement the EncodeImageInternal() method.

Change-Id: Idf71f803c24b3b5ae3b5079b15e019721784611d
2012-05-07 14:24:19 -07:00
Vikas Arora
32714ce3be Add VP8L prefix to backward ref & histogram methods.
Change-Id: I8c14fb219a1d7830d3244aa780c91c9964867330
2012-05-07 14:24:17 -07:00
Vikas Arora
fcba7be2d3 Fixed header file tag (WEBP_UTILS_HUFFMAN_ENCODE_H_)
Change-Id: I7ccd00361b1b0347639b05ee494e8e701c95cfe3
2012-05-07 14:24:16 -07:00
Vikas Arora
bc7037465d Add backward_ref, histogram & huffman encode modules from lossless.
Change-Id: Iac056d27972956782defa182caa3ea400cdb77f8
2012-05-07 14:24:14 -07:00
Vikas Arora
fdccaaddcf Fixing nits
- Const Handling of picture object, removed bitwriter from encoder.

Change-Id: Id943854de09324de81cca615ada960390c4b8152
2012-05-07 14:24:12 -07:00
Vikas Arora
227110c4c3 libwebp interface changes for lossless encoding.
Change-Id: I703a1a18347acf78378cb23fddc6e5ca6dc6a0bb
2012-05-07 14:24:09 -07:00
Pascal Massimino
50679acf25 minor style fixes
Change-Id: I78a6fa19a799c41d6536631f547279692b658c4d
2012-05-04 19:01:11 -07:00
Pascal Massimino
b38dfccf8d remove unneeded reference to NUM_LITERAL_CODES
Change-Id: I3e98acce3a69fa45054ffcf77644fcbbc04bd366
2012-05-04 19:01:09 -07:00
Pascal Massimino
8979675bdf harmonize header description
Change-Id: Ie0256794e663b58557e7234f08af3d516071001e
2012-05-04 19:01:06 -07:00