On some dataset, this was taking 2.5%. 2% when switching to
_mm_maskmoveu_si128. 1.7% when using _mm_loadu_si128
Confirmed by IACA: going from throughput of 4.26 to 3.5 and then
to 6.26 for twice the input.
Change-Id: I409f901aaad9d39bf55a1aac28cc25f126876b01
This restores the use of the function after
980b708e enc_neon: fix build w/aarch64 gcc < 9.4.0
The intrinsic was added to llvm for aarch64 in:
5e4ce1ae9dad Implement the newly added AArch64 ACLE functions for
ld1/st1 with 2/3/4 vectors. The functions are like:
vst1_s8_x2 ...
llvmorg-3.4.0-rc1~101
https://github.com/llvm/llvm-project/commit/5e4ce1ae9dad
Visual Studio 2019 and 2022 also support the function (2017 is still
disabled for this path due to it relying on arm64_neon.h).
Change-Id: I6ff10e22deb3968a48738a4458d2d3d55410b5ec
The values for the R/G/B floating point formulas resembled
https://fourcc.org/fccyvrgb.php and Video Demystified, but the fixed
point values are more closely aligned to rounded values from
https://en.wikipedia.org/wiki/YCbCr and BT.601.
The R/G/B formulas with the values prior to this change are added to
sharpyuv_csp.c as they align with the fixed values. The origin of those
coefficients is unclear. For consistency between library versions we'll
leave them as is.
Bug: webp:375011696
Change-Id: Id3f2a57530eee700cc52a899b32b25b5c015e89b
Take advantage of the known sizes used by VP8LHistogramAdd() and
remove loop for the remainder. The loop was being auto-vectorized making
the code larger and slower than the vectorized C code.
For larger sizes the new code is ~3-4.5% faster than the old code with
about the same improvement against the vectorized C code. For the
minimal size (40), the new code is ~30% faster than the C and old SSE2
code.
The LINE_SIZE==8 option is removed with this change. It had been set
to 16 for its entire life and clang-16 was unrolling the LINE_SIZE==8
case by 2 in any case; they both profile similarly.
Change-Id: I6dfedfd57474f44d15e2ce510a48e5252221077a
Take advantage of the known sizes used by VP8LHistogramAdd() and remove
loop for the remainder. The loop was being auto-vectorized making the
code larger and slower than the vectorized C code.
For larger sizes the new code is ~4-7% faster than the old code with
about the same improvement against the vectorized C code. For the
minimal size (40), the new code is ~30% faster than the C and old SSE2
code.
The LINE_SIZE==8 option is removed with this change. It had been set to
16 for its entire life and clang-16 was unrolling the LINE_SIZE==8 case
by 2 in any case; they both profile similarly.
Change-Id: I2376e2dca3bffa38477b4a432f4c533419e3be0e
Extend VP8EncIterator::i4_boundary_ by 3 bytes to avoid Intra4Preds_NEON
reading deeper into the struct (likely padding) when top is positioned
at offset 29. This data is memset with MSan to prevent a warning due to
its incorrect modeling of tbl instructions.
Prior to:
169dfbf9 disable Intra4Preds_NEON
there was a mismatch in the preprocessor checks for enabling the
function in NEON and removing the C version; NEON used `BPS == 32` while
the C code was removed unconditionally when building for aarch64. This
patch also normalizes those checks to look for `BPS == 32` and `BPS !=
32` as appropriate.
Bug: b:366668849,webp:372109644
Change-Id: Ic9e6ad4b2d844cb446decd63aec0b2676a89c8d0
These appear as warnings under VS15 (16 and 17 are silent) and were
missed in:
a32b436b dsp/lossless*: use WEBP_RESTRICT qualifier
Change-Id: Ia7cffafc166f2da93b51714363558798cda71b67
* changes:
dsp/yuv*: use WEBP_RESTRICT qualifier
dsp/upsampling*: use WEBP_RESTRICT qualifier
dsp/rescaler*: use WEBP_RESTRICT qualifier
dsp/lossless*: use WEBP_RESTRICT qualifier
dsp/filters*: use WEBP_RESTRICT qualifier
dsp/enc*: use WEBP_RESTRICT qualifier
dsp/dec*: use WEBP_RESTRICT qualifier
dsp/cost*: use WEBP_RESTRICT qualifier
The load of the `top` parameter may over read causing MSan errors:
==7373==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0xfff891d52ad4 in Intra4Preds_NEON src/dsp/enc_neon.c:1003:12
#1 0xfff892d87618 in MakeIntra4Preds src/enc/quant_enc.c:484:3
Bug: b:366668849
Change-Id: I29cf3b2f402ee79ea93c1ee2a4fdd95083aeed68
Better vectorization in the C code, fewer instructions / comparisons in
NEON, and fewer reloads in SSE2/SSE4 w/ndk r27/gcc-13/clang-16.
This only affects non-vector pointers; any vector pointers are left as a
follow up.
Change-Id: I07a7e36a2dce8632c71c0fbbeef94dc51453eaf7
Better vectorization in the C code, fewer instructions in NEON, and some
code reordering / better register usage in SSE2/SSE4 w/ndk
r27/gcc-13/clang-16.
This only affects non-vector pointers; any vector pointers are left as a
follow up.
Change-Id: Ib29980f778ad3dbb952178ad8dee39b8673c4ff8
Some improvement in the C code. No changes in NEON or SSE2 w/ndk
r27/gcc-13/clang-16.
This only affects non-vector pointers; any vector pointers are left as a
follow up.
Change-Id: I2316122db893f48f0afda90a147c83cac7f07526
lossless_enc: better vectorization, most benefits seen in AddVector/Eq
w/ndk r27/gcc-13/clang-16
lossless: minor reordering and some improvement to PredictorAdd5_SSE2
w/gcc-13
This only affects non-vector pointers; any vector pointers are left as a
follow up.
Change-Id: I2356e314f391ee2f2c71f00bc6ee10097d3881e7
Better stack/register usage in SSE2/NEON code and improved vectorization
of the C code with ndk r27/gcc-13/clang-16.
This only affects non-vector pointers; any vector pointers are left as a
follow up.
Change-Id: I32b53dd38bfc7e2231d875409e7dfda7c513cfb6
This allows for better vectorization of the C code, inlining of
TrueMotion_SSE2, better load usage in aarch64 and other minor
reordering with ndk r27/gcc-13/clang-16.
This only affects non-vector pointers; any vector pointers are left as a
follow up.
Change-Id: I07e9944d5c0aa5a079b22883ac5a2d649695e4a0
A minor improvement for arm targets with ndk r27/gcc-13 in H/VFilter8 (a
couple fewer moves w/aarch64) and much better vectorization of
DitherCombine8x8_C in most targets.
This only affects non-vector pointers; any vector pointers are left as a
follow up.
Change-Id: I03e73e6d6404261bb8408a9ae76a4b6ef142f8f0
on SetResidualCoeffs_*. This results in some minor code reordering when
targeting arvm7 with ndk r27 and other recent versions of clang. No
changes in the x86 compilations with clang-16 / gcc-13.
This only affects non-vector pointers; any vector pointers are left as a
follow up.
Change-Id: I7c3554ece848fafbc5ac9c4944f1dc85129f6fd8
The row parameter became a constant in:
2102ccd update the Unfilter API in dsp to process one row independently
num_rows is always equal to height.
Change-Id: Ie43dc5ef222e442ce8c92766da0b9824ccbca236
The inverse parameter became a constant in:
2102ccd update the Unfilter API in dsp to process one row independently
The row parameter to these functions is in a similar state; it will be
removed in a follow up.
Change-Id: I94cd8babe0e42474ff794ba5fa29dd48039de5f8
Replace vmovl_u8 -> s16 + signed vaddq with unsigned vaddw.
No change in assembly with clang-16 (armv7 & aarch64) and gcc-13
(aarch64). armv7 gcc-13 had kept the vmovl instructions, those are now
gone.
Change-Id: Ibb4fbdd5680d3e9dd06933c100528a6f363de472
This needs to be done with signed saturation as the sum may be negative.
fixes mismatch with C code after:
3bfb05e3 Add AArch64 Neon implementation of Intra16Preds
Change-Id: I017e939d7155cc3489ceb76fc8ad50ac9917f23d
This needs to be done with signed saturation as the sum may be negative.
fixes mismatch with C code after:
baa93808 Add AArch64 Neon implementation of Intra4Preds
Change-Id: I190c3d7f78cfd2c7ae83fb7059de41e307abda36
* changes:
Use QuantizeBlock_NEON for VP8EncQuantizeBlockWHT on Arm
Add AArch64 Neon implementation of Intra16Preds
Add AArch64 Neon implementation of Intra4Preds
Add a Neon implementation of Intra16Preds for use on 64-bit Arm
platforms. (This implementation cannot be used on 32-bit Arm
platforms as it makes use of a number of AArch64-only Neon
instructions.)
Change-Id: I24c67cd54b66307e3924fd332c2795fd7422f082
Add Neon implementation of Intra4Preds for use on 64-bit Arm
platforms. (The same implementation cannot be used for 32-bit Arm
platforms as it uses a number of AArch64-only Neon instructions.)
Change-Id: Id781e7614f4e8e876dfeecd95cfc85e04611d8c6
The lossless encoding speed-ups are:
- up to 1% with default parameters
- up to 4% in cruncher mode: -q 100 -m 6
Change-Id: Id92d4bad0b0a2c28c8aa9ff5280eea5717017f30
Replace the 2d histograms with uint32_t 1d versions (to avoid
pointer casting and to use the optimized VP8LAddVectorEq).
Change-Id: I90b0fe98390b49e3fd03e3484289571cf7ae6eca
fixes warnings of the form:
/src/dsp/upsampling_sse41.c:170:1: runtime error: implicit conversion
from type 'int' of value -16 (32-bit, signed) to type 'uintptr_t' (aka
'unsigned long') changed the value to 18446744073709551600 (64-bit,
unsigned)
this is the same change as was done previously in upsampling_sse2.c:
2ee786c7 upsampling_sse2.c: clear int sanitizer warnings
Change-Id: I36064d837ad1a7a118918c16a5551fc732dec2ff
This is available with clang. Clears warnings of the form:
warning: empty expression statement has no effect; remove unnecessary
';' to silence this warning [-Wextra-semi-stmt]
As a side-effect it also clear a few -Wpedantic warnings with gcc:
warning: ISO C does not allow extra ';' outside of a function
[-Wpedantic]
Change-Id: I9295c767aad475c68b1fbbdff855b0d6650a25f5
Initially added to workaround gcc implementation issues that clang
does not have. (gcc hardcodes rbx as the PIC register, clang uses a
virtual register)
Change-Id: I1a3277abf02b1ff437b4aea4d28f4cb1c0176b80
According to https://gcc.gnu.org/onlinedocs/gcc/extensions-to-the-c-language-family/how-to-use-inline-assembly-language-in-c-code.html
For the C language, the asm keyword is a GNU extension. When
writing C code that can be compiled with -ansi and the -std options
that select C dialects without GNU extensions, use __asm__ instead
of asm (see Alternate Keywords). For the C++ language, asm is a
standard keyword, but __asm__ can be used for code compiled with
-fno-asm.
Change-Id: I4af950e67c857c890290c1e3d9cc886da0748784
