If the width and height are 0xffff (or close), the result of a
multiplication will overflow.
Cast to a larger type to avoid undefined behavior.
Such values are possible in CODA7, but unlikely.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
Fixes:
918c66fd4126 ("[media] coda: add CODA7541 decoding support")
Signed-off-by: Anastasia Belova <abelova@astralinux.ru>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
if (!ctx->slicebuf.vaddr && q_data->fourcc == V4L2_PIX_FMT_H264) {
/* worst case slice size */
- size = (DIV_ROUND_UP(q_data->rect.width, 16) *
+ size = (unsigned long)(DIV_ROUND_UP(q_data->rect.width, 16) *
DIV_ROUND_UP(q_data->rect.height, 16)) * 3200 / 8 + 512;
ret = coda_alloc_context_buf(ctx, &ctx->slicebuf, size,
"slicebuf");