Random crashes due to accessing memory beyond allocated size

[mikulas@…]

Bit operations in libavcodec require that every pointer has additional
FF_INPUT_BUFFER_PADDING_SIZE bytes allocated because optimized routines touch
memory after end. Unfortunatelly sometimes pointer is allocated without this
additional memory.

To trigger the bug, link mplayer with electric fence (you can additionally set
shell variable "EF_ALIGNMENT=1" to get all 1-byte overruns) and play avi files
--- it will crash in UPDATE_CACHE macro in libavcodec.

Without electric fence this bug happens too --- but randomly with very low
probability.

Fix is not trivial, because you have to check all places where init_get_bits is
called and all places that pass pointers to libavcodec, all demuxers etc. that
they pad the stream with FF_INPUT_BUFFER_PADDING_SIZE bytes.

[diego@…]

Forwarded here from Michael Niedermayer:

why not provide some gdb output (backtrace mainly) and say which codec is
used? both audio & video or provide the full and uncut output of mplayer

[mikulas@…]

first proposed patch for overflow problems

[mikulas@…]

The bug happens when playing a file using h263 codec in asf stream. (I won't
post it publicly because of copyright issues)

I tried to spot the buggy places in mplayer and created this patch.

The extradata allocations violate the rules libavcodec/avcodec.h at line 672
---
i.e. they all should be padded with FF_INPUT_BUFFER_PADDING_SIZE bytes. This
causes real bug in h263.c codec.

Another problem is in demux_asf.c file --- when FF_INPUT_BUFFER_PADDING_SIZE
bytes were added to allocation, the problem with bit reader in h263.c went away

and the file played fine.

Other FF_INPUT_BUFFER_PADDING_SIZE changes in demuxers in my patch are just
guesses --- I haven't seen it on real file (I have only mpeg/avi/asf/mov
files).

The if (!trak->stdata) return; in demux_mov.c fixes crash after "* constant
samplesize & variable duration not yet supported! *
Contact the author if you have such sample file!" message (if anyone wants to
debug it further and add support for this file, contact me).

The other part in demux_mov.c fixes another reading beyond end of allocated
area
found with electric fence --- this happens on files that are normally playable
without efence.

[diego@…]

This patch is not against CVS and thus useless, since it does not apply
cleanly. If you wish to see this included you should best submit it yourself
on the ffmpeg-devel and mplayer-dev-eng mailing lists.

[mikulas@…]

patch another reading beyond allocated memory

[mikulas@…]

Even if it applied correctly against CVS mplayer, someone still should check
that padding is made on all paths to libavcodec --- I don't claim I fixed
everything, I fixed only what I hit or what I thought could be bugs.

BTW. --- here is another patch for crash with -lefence when playing mov files
with proprietary audio codec --- function expCreatePalette reads too much data
from buffer provided by the codec and crashes with ElectricFence.

[diego@…]

The first patch has been applied by Jindrich Mackovicka. The second patch is
still pending.

[mans@…]

The first patch has been applied. The other one belongs to mplayer. Reassigning.