Fixes: runtime error: left shift of negative value -967831544
Fixes: 2815/clusterfuzz-testcase-minimized-6062914471460864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: left shift of 1965559808 by 4 places cannot be represented in type 'int'
Fixes: 2333/clusterfuzz-testcase-minimized-5223935677300736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegReviewed-by: NPaul B Mahol <onemda@gmail.com>
Signed-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Those values are decoder-independent and are also use by the VA-API
encoder.

This is more consistent with the rest of libav and frees up the hevc.h
name for decoder-independent shared declarations.

Fixes: 04ec80eefa77aecd7a49a442cc02baea/asan_heap-oob_19544fa_3303_1905796cd9d8e15f86d664332caabc00.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: NMichael Niedermayer <michael@niedermayer.cc>

Factor out the parameter sets into a separate struct and use it instead.

This will allow us to reuse this code in the parser.

avcodec/hevc_cabac: Rename ff_hevc_transform_skip_flag_decode() to hevc_transform_skip_flag_decode()

The function is static and used once in the file its defined in
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Reviewed-by: NMichael Niedermayer <michaelni@gmx.at>
Signed-off-by: NJames Almer <jamrial@gmail.com>

hevc seems to be the only place where the C implementation
of the av_clip function is explicitly selected, precluding
platform-specific optimizations
Signed-off-by: NPeter Meerwald <pmeerw@pmeerw.net>
Signed-off-by: NAnton Khirnov <anton@khirnov.net>

Prevents shifts with undefined behavior, as no syntax element has a valid
value greater than 1 << 31.

Bug-Id: CID 1206635

Kind of hackish but...
Reviewed-by: NMickael Raulet <Mickael.Raulet@insa-rennes.fr>
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

-new rext bitstreams:
PERSIST_RPARAM_A_RExt_Sony_1.bit           ok    =
QMATRIX_A_RExt_Sony_1.bit                  ok    =
SAO_A_RExt_MediaTek_1.bit                  ok    =

(cherry picked from commit cdea029d452c521f8e5bcbe589f44b13a4011604)
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

SPS features/flags:
- transform_skip_rotation_enabled_flag
- transform_skip_context_enabled_flag
- implicit_rdpcm_enabled_flag
- explicit_rdpcm_enabled_flag
- intra_smoothing_disabled_flag
- persistent_rice_adaptation_enabled_flag

PPS features/flags:
- log2_max_transform_skip_block_size
- cross_component_prediction_enabled_flag
- chroma_qp_offset_list_enabled_flag
- diff_cu_chroma_qp_offset_depth
- chroma_qp_offset_list_len_minus1
- cb_qp_offset_list
- cr_qp_offset_list
- log2_sao_offset_scale_luma
- log2_sao_offset_scale_chroma
(cherry picked from commit 005294c5b939a23099871c6130c8a7cc331f73ee)
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

- support for 4:2:2 and 4:4:4 up to 12 bits
- add a new profile for range extension
(cherry picked from commit d3c067fa65bbc871758d28aa07f54123430ca346)
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

(cherry picked from commit 6b3856ef57d66f2e59ee61fd2eb5f83b6d0d7d4a)
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

hevc/cabac: add new context for new syntax elements related to Rext(cherry picked from commit 6d71e2394f52679cfc8b86fb5880f89e6bd311d4)
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

hevc: cleaning up, remove unused constants(cherry picked from commit 7eed32d076c57aa03011d65a64903e8bdb633978)
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Only its size is actually used. Silence a clang warning.

Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

pretty print C
Reviewed-by: N"Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Prevents shifts with undefined behavior

Fixes CID1206634
Fixes CID1206635
Fixed CID1206636

Reviewed-by: smarter
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Initially written by Guillaume Martres <smarter@ubuntu.com> as a GSoC
project. Further contributions by the OpenHEVC project and other
developers, namely:

Mickaël Raulet <mraulet@insa-rennes.fr>
Seppo Tomperi <seppo.tomperi@vtt.fi>
Gildas Cocherel <gildas.cocherel@laposte.net>
Khaled Jerbi <khaled_jerbi@yahoo.fr>
Wassim Hamidouche <wassim.hamidouche@insa-rennes.fr>
Vittorio Giovara <vittorio.giovara@gmail.com>
Jan Ekström <jeebjp@gmail.com>
Anton Khirnov <anton@khirnov.net>
Martin Storsjö <martin@martin.st>
Luca Barbato <lu_zero@gentoo.org>
Yusuke Nakamura <muken.the.vfrmaniac@gmail.com>
Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Diego Biurrun <diego@biurrun.de>
Signed-off-by: NAnton Khirnov <anton@khirnov.net>

Decreases the difference to Anton Khirnovs patch v5
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Decreases the difference to Anton Khirnovs patch v5
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

cherry picked from commit b971f2c8fdc60f8bab605a6e8060492eb548a53a
cherry picked from commit e57b0a2c915ce6b8a9d57b8292f6581f0680842e
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

hevc: inline cabac in hls_mvd_coding(cherry picked from commit ad387195ad04e8a005a1bfd509e9e4f827e68fa9)
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

hevc: inline cabac in residual coding(cherry picked from commit 17d7a880445b72feb36d684ae1f0597195811e97)
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>

Initially written by Guillaume Martres <smarter@ubuntu.com> as a GSoC
project. Further contributions by the OpenHEVC project and other
developers, namely:

Mickaël Raulet <mraulet@insa-rennes.fr>
Seppo Tomperi <seppo.tomperi@vtt.fi>
Gildas Cocherel <gildas.cocherel@laposte.net>
Khaled Jerbi <khaled_jerbi@yahoo.fr>
Wassim Hamidouche <wassim.hamidouche@insa-rennes.fr>
Vittorio Giovara <vittorio.giovara@gmail.com>
Jan Ekström <jeebjp@gmail.com>
Anton Khirnov <anton@khirnov.net>
Martin Storsjö <martin@martin.st>
Luca Barbato <lu_zero@gentoo.org>
Yusuke Nakamura <muken.the.vfrmaniac@gmail.com>
Signed-off-by: NAnton Khirnov <anton@khirnov.net>
Signed-off-by: NMichael Niedermayer <michaelni@gmx.at>