If the first assembler to be probed is an old nasm build, X86ASM_DEPFLAGS
will be set and remain so after yasm is ultimately used as fallback.
This results in yasm being called with said nasm specific flags and failing
during actual object assembly but not with configure sanity checks.

Regression since 5cae5a1dReviewed-by: NMichael Niedermayer <michael@niedermayer.cc>
Signed-off-by: NJames Almer <jamrial@gmail.com>

Fixes framesync filters with shortest option enabled.

This reverts commit 0ff5567a.

Checks for NULL before using object.
Signed-off-by: NRick Kern <kernrj@gmail.com>

Signed-off-by: NRick Kern <kernrj@gmail.com>

set_ue_golomb just support 2^16 - 2 at most, becase this function call
put_bits, and put_bits just support write up to 31 bits, when write 32
bit in put_bits, it's will overwrite the bit buffer, and the default
assert level is 0, the av_assert2(n <= 31 && value < (1U << n)) in
put_bits can not be trigger runtime.
Signed-off-by: NJun Zhao <jun.zhao@intel.com>
Signed-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: signed integer overflow: 227511904 + 1964113935 cannot be represented in type 'int'
Fixes: 2331/clusterfuzz-testcase-minimized-6182185830711296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: signed integer overflow: 512 + 2147483146 cannot be represented in type 'int'
Fixes: 2314/clusterfuzz-testcase-minimized-4519333877252096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: signed integer overflow: 538976288 * 32 cannot be represented in type 'int'
Fixes: 2310/clusterfuzz-testcase-minimized-4534784887881728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Signed-off-by: NRostislav Pehlivanov <atomnuker@gmail.com>

Fixes a NULL pointer derefence when ogg_init() returns a failure and
a stream's private data was not yet allocated.

This is a regression since 3c5a53cdReviewed-by: NPaul B Mahol <onemda@gmail.com>
Signed-off-by: NJames Almer <jamrial@gmail.com>

Avoids overriding the v0 register (which containins the result of the
tested function) in checkasm_call_checked.

Fixes: runtime error: shift exponent 32 is too large for 32-bit type 'int'
Fixes: 2306/clusterfuzz-testcase-minimized-5002997392211968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: left shift of negative value -1
Fixes: 2303/clusterfuzz-testcase-minimized-5529675273076736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

This allows for more graceful fallback from NASM to Yasm if the available
NASM version is too old.

(Cherry-picked from libav commit adfd7892)
Signed-off-by: NJames Almer <jamrial@gmail.com>

NASM is more actively maintained and permits generating dependency information
as a sideeffect of assembling, thus cutting build times in half.

(Cherry-picked from libav commit 57b753b4)
Signed-off-by: NJames Almer <jamrial@gmail.com>

"nasm --version" works on nasm since around version 2.11 and as
such configure assumed it was yasm.

Based on libav commit f54037da by
Diego Biurrun.
Signed-off-by: NJames Almer <jamrial@gmail.com>

Consistently use object format names with "32" suffix and set object format
to "win64" on Windows x86_64, which fixes assembling with nasm.

(Cherry-picked from libav commit 808ef435)
Signed-off-by: NJames Almer <jamrial@gmail.com>

(Cherry-picked from libav commit 3c0efbd0)
Signed-off-by: NJames Almer <jamrial@gmail.com>

None of them are specific to the YASM assembler.

(Cherry-picked from libav commit 39e208f4)
Signed-off-by: NJames Almer <jamrial@gmail.com>

Signed-off-by: NPaul B Mahol <onemda@gmail.com>

This was added in e3e3c825, probably
as a workaround for the fact that the quant table was not permutated
and the IDCT coefficients are, meaning that you'd only get correct
reconstruction if the IDCT permutation was an identity matrix, which
happens to be the case when you use the simple IDCT. The quant table
permutation bug was fixed in 42dd1434,
meaning this workaround is no longer necessary.

In practical terms, before 42dd1434,
the PSNR between decodes of the fate-mdec using simple (C) or simplemmx
IDCTs was 35. After 42dd1434, it's 90.

Properly use the b.eq form instead of the nonstandard form (which
both gas and newer clang accept though), and expand the register
lists that used a range (which the Xcode 6.2 clang, based on clang
3.5 svn, didn't support).
Signed-off-by: NMartin Storsjö <martin@martin.st>

Properly use the b.eq/b.ge forms instead of the nonstandard forms
(which both gas and newer clang accept though), and expand the
register list that used a range (which the Xcode 6.2 clang, based
on clang 3.5 svn, didn't support).

This is cherrypicked from libav commit
a970f9de.
Signed-off-by: NMartin Storsjö <martin@martin.st>

Reviewed-by: NJames Darnley <jdarnley@obe.tv>
Signed-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: left shift of negative value -1

Fixes: 2299/clusterfuzz-testcase-minimized-4843509351710720

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Signed-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: signed integer overflow: 262144 * -16120 cannot be represented in type 'int'
Fixes: 2292/clusterfuzz-testcase-minimized-6156080415506432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: signed integer overflow: -1575818955 + -915383657 cannot be represented in type 'int'
Fixes: 2224/clusterfuzz-testcase-minimized-6208559949807616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

The early check for inconsistent in-source vs out-of-source build
cannot generate a config.log otherwise.

(Cherry-picked from libav commit 0ee78020)
Signed-off-by: NLuca Barbato <lu_zero@gentoo.org>
Signed-off-by: NJames Almer <jamrial@gmail.com>

Use avci->last_pkt_props to get the side data. Using |pkt| doesn't work
when FF_API_MERGE_SD is set, as the compressed side data is expanded into
|tmp|, leaving the original |pkt| unchanged.
Signed-off-by: NJames Almer <jamrial@gmail.com>

Use named arguments for the functions so we can remove a define.  The
stride/linesize argument is now ptrdiff_t type so we no longer need to
sign extend the register.

(cherry picked from commit 2f792cb6)

It is wrong/incorrect in two aspects:
1. The pixel format is not enough to guarantee that the resulting file
   will be any more compatible with media players.
2. Media players not supporting higher profiles are not necessarily
   outdated (in fact this is simply an arrogant statement that
   libavcodec can handle these particular features).

You could add that there are plenty of other ways to produce widely
incompatible files with ffmpeg, and these don't show any warnings.

What we really want to do here is defaulting to codec profiles that
have wide compatibility, such as main/high for h264. Also, if an
encoder does not accept certain pixfmts, we should automatically
convert them to a pixfmt the encoder can accept. But the existing
message certainly is not appropriate.

It also works for 2 specific encoders only. Extending it for other
cases would result in a lot of special cases, so this is not the
right place.

Signed-off-by: NMichael Niedermayer <michael@niedermayer.cc>