When making a new P-frame when B-frames are present the previous P-frame
is normally in the DPB because it will be referred to by subsequent
B-frames.  However, this is not true if there are no B-frames, or in edge
cases where a GOP ends with two P-frames.  Fix this by adding the direct
ref pics to the RPS explicitly.

Fixes #7699.
Tested-by: NUllysses A Eoff <ullysses.a.eoff@intel.com>

The reference picture sets are now constructed directly from the DPB
information.

This attaches the logic of picking the mode of for the next picture to
the output, which simplifies some choices by removing the concept of
the picture for which input is not yet available.  At the same time,
we allow more complex reference structures and track more reference
metadata (particularly the contents of the DPB) for use in the
codec-specific code.

It also adds flags to explicitly track the available features of the
different codecs.  The new structure also allows open-GOP support, so
that is now available for codecs which can do it.

Matching previous commit for H.264.

Fixes CID #1439657.

Sets the level based on the stream properties if it is not explicitly
set by the user.  Also add a tier option to set general_tier_flag, since
that affects the level choice.

Set profile compatibility/constraint flags properly (including the
constraint flags used for RExt profiles, as all streams we can currently
generate are RExt-compatible), and use that to add support for the "Main
Intra" and "Main 10 Intra" RExt subprofiles (for which we can re-use the
existing Main and Main10 VAAPI profiles).

Add a larger warning more clearly explaining the consequences of missing
packed header support in the driver.  Also only write the extradata if the
user actually requests it via the GLOBAL_HEADER flag.

Choose what types of reference frames will be used based on what types
are available, and make the intra-only mode explicit (GOP size one,
which must be used for MJPEG).

This was added in libva 2.1.0 (VAAPI 1.1.0).  Use AVCodecContext.qmax,
matching the existing behaviour for qmin, and clean up the defaults so
that we only pass min/max when explicitly set.

Query which modes are supported and select between VBR and CBR based
on that - this removes all of the codec-specific rate control mode
selection code.

The only common option here is low_power - it was previously supported
for H.264 only, that specific option is removed.

Previously there was one fixed choice for each codec (e.g. H.265 -> Main
profile), and using anything else then required an explicit option from
the user.  This changes to selecting the profile based on the input format
and the set of profiles actually supported by the driver (e.g. P010 input
will choose Main 10 profile for H.265 if the driver supports it).

The entrypoint and render target format are also chosen dynamically in the
same way, removing those explicit selections from the per-codec code.

The codec-specific context now contains both the common context and the
codec-specific options directly.

Matching previous commit for H.264.

Signed-off-by: NHaihao Xiang <haihao.xiang@intel.com>

'-sei xxx' is added to control SEI insertion, so far only mastering
display colour volume is available for testing.
Signed-off-by: NHaihao Xiang <haihao.xiang@intel.com>

This makes it easier for users of the CBS API to get alloc/free right -
all subelements use the buffer API so that it's clear how to free them.
It also allows eliding some redundant copies: the packet -> fragment copy
disappears after this change if the input packet is refcounted, and more
codec-specific cases are now possible (but not included in this patch).

The iHD driver looks at entries beyond num_ref_idx_l[01]_active_minus1
for unknown reasons.

... instead of making callers allocate it themselves.  This is more
consistent with other APIs in libav.

Explicitly identify decoder/encoder wrappers with a common name. This
saves API users from guessing by the name suffix. For example, they
don't have to guess that "h264_qsv" is the h264 QSV implementation, and
instead they can just check the AVCodec .codec and .wrapper_name fields.

Explicitly mark AVCodec entries that are hardware decoders or most
likely hardware decoders with new AV_CODEC_CAPs. The purpose is allowing
API users listing hardware decoders in a more generic way. The proposed
AVCodecHWConfig does not provide this information fully, because it's
concerned with decoder configuration, not information about the fact
whether the hardware is used or not.

AV_CODEC_CAP_HYBRID exists specifically for QSV, which can have software
implementations in case the hardware is not capable.

Based on a patch by Philip Langdale <philipl@overt.org>.

Merges Libav commit 47687a2f.

Explicitly identify decoder/encoder wrappers with a common name. This
saves API users from guessing by the name suffix. For example, they
don't have to guess that "h264_qsv" is the h264 QSV implementation, and
instead they can just check the AVCodec .codec and .wrapper_name fields.

Explicitly mark AVCodec entries that are hardware decoders or most
likely hardware decoders with new AV_CODEC_CAPs. The purpose is allowing
API users listing hardware decoders in a more generic way. The proposed
AVCodecHWConfig does not provide this information fully, because it's
concerned with decoder configuration, not information about the fact
whether the hardware is used or not.

AV_CODEC_CAP_HYBRID exists specifically for QSV, which can have software
implementations in case the hardware is not capable.

Based on a patch by Philip Langdale <philipl@overt.org>.
Signed-off-by: NLuca Barbato <lu_zero@gentoo.org>

Also fixes the default, which previously contained a nonsense value.

Also fixes the default, which previously contained a nonsense value.

To match vaapi_h264.

From ffmpeg commit 385cafb0.
Signed-off-by: NMark Thompson <sw@jkqxz.net>

Matching the H.264 encoder.

(cherry picked from commit e3e8eab3)

Also improves the metadata and generally makes the configuration
a bit cleaner.

(cherry picked from commit ac124867)

This reverts commit a14a12ca.

The CTU size is always 32x32; the surface size is what actually sets
the desired property, and it is already correct.

Follow vaapi_h264 style, enable the VBR mode.
Signed-off-by: NJun Zhao <jun.zhao@intel.com>
Signed-off-by: NMark Thompson <sw@jkqxz.net>

It is not necessary to pad to the CTU size.  The CB size of 8x8 should be
sufficient, but due to constraints in the Intel driver (the one usable
implementation of this) it has to be padded to 16x16 like in H.264.

Matching the H.264 encoder.

Also improves the metadata and generally makes the configuration
a bit cleaner.

the VAEncSliceParameterBufferHEVC in libva have support this field,
so remove the duplicate field in VAAPIEncodeH265MiscSliceParams.
Signed-off-by: NJun Zhao <jun.zhao@intel.com>
Signed-off-by: NMark Thompson <sw@jkqxz.net>

The non-H.26[45] codecs already use this form.  Since we don't
currently generate I frames for codecs which support them separately
to IDR, the p_per_i variable is set to infinity by default so that it
doesn't interfere with any other calculation.  (All the code for I
frames still exists, and it works for H.264 if set manually.)

(cherry picked from commit 6af014f4)

The non-H.26[45] codecs already use this form.  Since we don't
currently generate I frames for codecs which support them separately
to IDR, the p_per_i variable is set to infinity by default so that it
doesn't interfere with any other calculation.  (All the code for I
frames still exists, and it works for H.264 if set manually.)

Same issue as 17a0f948.

(cherry picked from commit 7d81698b)

(cherry picked from commit 5a5df90d)
(cherry picked from commit d08e02d9)

10-bit surface support was added in libva 1.6.2, earlier versions
support H.265 encoding in 8-bit only.