avcodec/vc1: fix out-of-bounds reference pixel replication

Out-of-bounds reference pixel replication should take into account the frame
coding mode of the reference frame(s), not the frame coding mode of the
current frame.
Signed-off-by: NJerome Borsboom <jerome.borsboom@carpalis.nl>