Currently, SPS.mb_height is actually what the spec calls
PicHeightInMapUnits, which is half the frame height when interlacing is
allowed. Calling this 'mb_height' is quite confusing, and there are at
least two associated bugs where this field is treated as the actual
frame height - in the h264 parser and in the code computing maximum
reordering buffer size for a given level.

Fix those issues (and avoid possible future ones) by exporting the real
frame height in this field.

This field (which the spec calls max_num_ref_frames) must be less than
or equal to MaxDpbFrames, which is at most 16.

The PS parsing code is independent from the decoder, so it makes more
sense for it to have its own separate header.

This is more consistent with the naming of other decoders.

Signed-off-by: NAnton Khirnov <anton@khirnov.net>

Signed-off-by: NDiego Biurrun <diego@biurrun.de>

Make the SPS/PPS parsing independent of the H264Context, to allow
decoupling the parser from the decoder. The change is modelled after the
one done earlier for HEVC.

Move the dequant buffers to the PPS to avoid complex checks whether they
changed and an expensive copy for frame threads.

Fall back to maximum DPB size if the level is unknown.

This should be more spec-compliant and does not depend on the caller
setting has_b_frames before opening the decoder.

The old behaviour, when the delay is supplied by the caller setting
has_b_frames, can still be obtained by setting strict_std_compliance
below normal.

Convert doxygen to multiline and express bitfields more simply.
Signed-off-by: NVittorio Giovara <vittorio.giovara@gmail.com>

Bug-Id: CID 1026763

There is in general no reason for the currently active SPS to be the one
referenced by the PPS being parsed.

CC: libav-stable@libav.org

Signed-off-by: NDiego Biurrun <diego@biurrun.de>

This avoids compiler warnings about argument and specifier type mismatch.

Also comment all previous profiles.

In preparation for MVC support.

Also make error style consistent and drop redundant information.

An invalid VUI is not considered a fatal error, so the SPS containing it
may still be used. Leaving an invalid value of num_reorder_frames there
can result in writing over the bounds of H264Context.delayed_pic.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

There is no point in delaying the check and it avoids bugs with a
half-initialized context.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

A VUI doesn't contain anything strictly necessary for decoding.
Apparently there are many samples with truncated VUIs in the wild, this
commit should allow decoding them.

Based on a patch by Vittorio Giovara <vittorio.giovara@gmail.com>

Fixes Bug 378.

Most of the changes are just trivial are just trivial replacements of
fields from MpegEncContext with equivalent fields in H264Context.
Everything in h264* other than h264.c are those trivial changes.

The nontrivial parts are:
1) extracting a simplified version of the frame management code from
   mpegvideo.c. We don't need last/next_picture anymore, since h264 uses
   its own more complex system already and those were set only to appease
   the mpegvideo parts.
2) some tables that need to be allocated/freed in appropriate places.
3) hwaccels -- mostly trivial replacements.
   for dxva, the draw_horiz_band() call is moved from
   ff_dxva2_common_end_frame() to per-codec end_frame() callbacks,
   because it's now different for h264 and MpegEncContext-based
   decoders.
4) svq3 -- it does not use h264 complex reference system, so I just
   added some very simplistic frame management instead and dropped the
   use of ff_h264_frame_start(). Because of this I also had to move some
   initialization code to svq3.

Additional fixes for chroma format and bit depth changes by
Janne Grunau <janne-libav@jannau.net>
Signed-off-by: NAnton Khirnov <anton@khirnov.net>

Some 3D systems overload the meaning of the field for other purposes.
Signed-off-by: NLuca Barbato <lu_zero@gentoo.org>

Fixes bug 396.

CC: libav-stable@libav.org

Introduced in d7d6efe4.

Fixes a crash in the fuzzed sample sample_varPAR.avi_s26638 with
alternating bit depths.

Fixes infinite or long taking loop in frame num gap code in
the fuzzed sample bipbop234.ts_s223302.

CC: libav-stable@libav.org