floating point precision will cause rgb*max generate different value on
x86_32 and x86_64. have pass fate test on x86_32 and x86_64 by using
lrintf to get the nearest integral value for rgb * max before av_clip.
Reviewed-by: NPaul B Mahol <onemda@gmail.com>
Signed-off-by: NLimin Wang <lance.lmwang@gmail.com>

Signed-off-by: NAndreas Rheinhardt <andreas.rheinhardt@gmail.com>

Tested on x86-32/64, mingw32/64, arm & mips qemu
Tested-by: NMichael Niedermayer <michael@niedermayer.cc>
Signed-off-by: NLimin Wang <lance.lmwang@gmail.com>