-vo pp and Theora cause segmentation fault

[nicolas.george@…]

The ​http://mirror.fluendo.com:8800/ Theora stream causes mplayer to segfault.
Other Theora files do the same.

Details:

All tests are done on Debian Sarge.

• mplayer 1.0pre5 on an Athlon with libtheora 0.0.0.alpha3-1 from Debian: crash;
• mplayer 1.0pre6 on the same Athlon with the same libtheora: crash;
• mplayer 1.0pre6 (same binary) with a libtheora alpha4 from the sources: crash;
• mplayer 1.0pre5try2 on a Pentium MMX: ok (either on local display or using X11

over ssh).

The crash occurs even with -vo null.

libc 2.3.2
gcc 3.3.4
binutils 2.15
processor : 0
vendor_id : AuthenticAMD
cpu family : 6
model : 4
model name : AMD Athlon(tm) Processor
stepping : 2
cpu MHz : 1200.523
cache size : 256 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 mmx fxsr syscall mmxext 3dnowext 3dnow
bogomips : 2359.29

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1085748672 (LWP 27798)]
0x080d634b in fast_memcpy_MMX2 (to=0x87ba000, from=0x40dd8b38, len=0)

at aclib_template.c:259

259 asm volatile(
(gdb) bt
#0 0x080d634b in fast_memcpy_MMX2 (to=0x87ba000, from=0x40dd8b38, len=0)

at aclib_template.c:259

#1 0x080d6502 in fast_memcpy (to=0x8728c40, from=0x480, len=1152)

at aclib.c:134

#2 0x0837422b in postProcess_MMX2 (

src=0x40d5beb8 <repeats 39 times>, "", <repeats 40 times>, "&#1786;",

<repeats 38 times>, "\233\217\224\231\225\223", '\225' <repeats 32 times>,
"\226\226\225\225\230\230\214uge", 'i' <repeats 15 times>...,

srcStride=-352, dst=0x879f300 "", dstStride=320, width=320, height=240,
QPs=0x86b9178 '\001' <repeats 20 times>, "\002", QPStride=0, isColor=0,
c2=0x87271c0) at postprocess_template.c:3564

#3 0x083792ca in pp_postprocess (src=0x879b8a8, srcStride=0x879b8b8,

dst=0x86b9908, dstStride=0x86b9918, width=320, height=240,
QP_store=0x86b9178 '\001' <repeats 20 times>, "\002", QPStride=0,
vm=0x86b8aa0, vc=0x87271c0, pict_type=0) at libpostproc/postprocess.c:703

#4 0x0813ba1e in put_image (vf=0x86b88c8, mpi=0x879b888) at vf_pp.c:126
#5 0x0810de10 in decode_video (sh_video=0x86b5b88,

start=0x480 <Address 0x480 out of bounds>, in_size=1152, drop_frame=0)
at dec_video.c:332

#6 0x08096981 in main (argc=2, argv=0xbffff8b4) at mplayer.c:2302

[nicolas.george@…]

Update: there was a configuration difference I did not remember. The
segmentation fault is triggered by the -vf pp option, on both systems.

Valgrind shows:

==28615== Invalid write of size 1
==28615== at 0x1B906AEE: memcpy (mac_replace_strmem.c:286)
==28615== by 0x8367851: postProcess_C (postprocess_template.c:3564)
==28615== by 0x836A529: pp_postprocess (postprocess.c:711)
==28615== by 0x8131E9D: put_image (vf_pp.c:126)
==28615== Address 0x1C96C72E is 2 bytes before a block of size 12288 alloc'd
==28615== at 0x1B907AD5: memalign (vg_replace_malloc.c:217)
==28615== by 0x8369E99: reallocAlign (postprocess.c:956)
==28615== by 0x8369F3A: reallocBuffers (postprocess.c:968)
==28615== by 0x836A1B4: pp_get_context (postprocess.c:1013)

(BTW, running mplayer in valgrind is frightening...)

[reimar]

(In reply to comment #1)

Update: there was a configuration difference I did not remember. The
segmentation fault is triggered by the -vf pp option, on both systems.

I guess you already tried CVS? Some problems with -vf pp have been fixed some
months ago...
Also please post a full mplayer -v log, it will allow us to see things like e.g.
what resolution the video has...
I fear I won't be able to help much as I don't have the theora decoder installed
and am not motivated to download it, but it could be some obvious problem after
all (hey, you can always hope ;-) )

Valgrind shows:

==28615== Invalid write of size 1
==28615== at 0x1B906AEE: memcpy (mac_replace_strmem.c:286)
==28615== by 0x8367851: postProcess_C (postprocess_template.c:3564)
==28615== by 0x836A529: pp_postprocess (postprocess.c:711)
==28615== by 0x8131E9D: put_image (vf_pp.c:126)
==28615== Address 0x1C96C72E is 2 bytes before a block of size 12288 alloc'd
==28615== at 0x1B907AD5: memalign (vg_replace_malloc.c:217)
==28615== by 0x8369E99: reallocAlign (postprocess.c:956)
==28615== by 0x8369F3A: reallocBuffers (postprocess.c:968)
==28615== by 0x836A1B4: pp_get_context (postprocess.c:1013)

A few callers more might be helpful, too...

(BTW, running mplayer in valgrind is frightening...)

Why? Last time I check almost everything reported was caused by ioctls it
couldn't handle and the alsa ao (where I am still unable to determine whether it
is MPlayer or alsa lib or both that is broken as hell).

[nicolas.george@…]

I guess you already tried CVS? Some problems with -vf pp have been fixed some
months ago...

I only browsed the logs for Theora-related changed, and did not find anything. I
just built the 20050109 snapshot, here are the results.

Also please post a full mplayer -v log, it will allow us to see things like e.g.
what resolution the video has...

The problem occurs with various video, on various systems (some friends of mine
have tested), and with -vo null, sot it is probably irrelevant. Nonetheless,
here is the full log:

MPlayer dev-CVS-050109-06:00-3.3.4 (C) 2000-2005 MPlayer Team
CPU: Advanced Micro Devices Athlon Thunderbird (Family: 6, Stepping: 2)
Detected cache-line size is 64 bytes
CPUflags: MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 0 SSE2: 0
Compiled for x86 CPU with extensions: MMX MMX2 3DNow 3DNowEx

CommandLine: '-vf' 'pp' '-v' 'log' 'theora.ogg'
init_freetype
get_path('font/font.desc') -> '/dev/null/.mplayer/font/font.desc'
font: can't open file: /dev/null/.mplayer/font/font.desc
font: can't open file: /opt/mplayer-20050109/share/mplayer/font/font.desc
Using MMX (with tiny bit MMX2) Optimized OnScreenDisplay
Linux RTC init error in ioctl (rtc_irqp_set 1024): Permission denied
Try adding "echo 1024 > /proc/sys/dev/rtc/max-user-freq" to your system startup
scripts.
Using nanosleep() timing
get_path('input.conf') -> '/dev/null/.mplayer/input.conf'
Can't open input config file /dev/null/.mplayer/input.conf: Not a directory
Can't open input config file /opt/mplayer-20050109/etc/mplayer/input.conf: No
such file or directory
Falling back on default (hardcoded) input config
get_path('log.conf') -> '/dev/null/.mplayer/log.conf'
Playing log.
File not found: 'log'
Failed to open log

get_path('theora.ogg.conf') -> '/dev/null/.mplayer/theora.ogg.conf'
Playing theora.ogg.
[file] File size is 309345 bytes
STREAM: [file] theora.ogg
STREAM: Description: File
STREAM: Author: Albeu
STREAM: Comment: based on the code from ??? (probably Arpi)
Checking for YUV4MPEG2
ASF_check: not ASF guid!
Checking for Nullsoft Streaming Video
Checking for NuppelVideo
Checking for REAL
Checking for SMJPEG
Searching demuxer type for filename theora.ogg ext: .ogg
Trying demuxer 18 based on filename extension
==> Found video stream: 0
Ogg : stream 0 is theora v3.2.0 320:240, 5.000 FPS, aspect 0:0
======= VIDEO Format ======

biSize 40
biWidth 320
biHeight 240
biPlanes 3
biBitCount 24
biCompression 65532=
biSizeImage 230400

===========================
Ogg stream length (granulepos): 69479004
Ogg demuxer : found 0 audio stream, 1 video stream and 0 text stream
Ogg file format detected.
VIDEO: [] 320x240 24bpp 5.000 fps 0.0 kbps ( 0.0 kbyte/s)
[V] filefmt:18 fourcc:0xFFFC size:320x240 fps: 5.00 ftime:=0.2000
get_path('sub/') -> '/dev/null/.mplayer/sub/'
get_path('default.sub') -> '/dev/null/.mplayer/default.sub'
X11 opening display: :0.0
vo: X11 color mask: FFFFFF (R:FF0000 G:FF00 B:FF)
vo: X11 running at 1600x1200 with depth 24 and 32 bpp (":0.0" => local display)
[x11] Detected wm supports layers.
[x11] Detected wm supports NetWM.
[x11] Detected wm supports ABOVE state.
[x11] Detected wm supports BELOW state.
[x11] Detected wm supports FULLSCREEN state.
[x11] Detected wm supports STAYS_ON_TOP state.
[x11] Current fstype setting honours LAYER FULLSCREEN STAYS_ON_TOP ABOVE BELOW X
atoms
Opening video filter: [pp]
[vo] query(Planar YV12) -> 3
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Could not open codec.
VDecoder init failed :(
Opening video decoder: [theora] Theora/VP3
INFO: Theora video init ok!
VDec: vo config request - 320 x 240 (preferred csp: Planar YV12)
[PP] Using external postprocessing filter, max q = 6.
Trying filter chain: pp vo
VDec: using Planar YV12 as output csp (no 0)
Movie-Aspect is undefined - no prescaling applied.
VO Config (320x240->320x240,flags=0,'MPlayer',0x32315659)
REQ: flags=0x437 req=0x0
VO: [xv] 320x240 => 320x240 Planar YV12
VO: Description: X11/Xv
VO: Author: Gerd Knorr <kraxel@…> and others
Xvideo image format: 0x41424752 (RGBA) packed
Xvideo image format: 0x0 ( ) packed
Xvideo image format: 0x54424752 (RGBT) packed
Xvideo image format: 0x32424752 (RGB2) packed
Xvideo image format: 0x32595559 (YUY2) packed
Xvideo image format: 0x59565955 (UYVY) packed
Xvideo image format: 0x32315659 (YV12) planar
Xvideo image format: 0x30323449 (I420) planar
using Xvideo port 61 for hw scaling
[xv] dx: 0 dy: 0 dw: 320 dh: 240
Selected video codec: [theora] vfm:theora (Theora (free, reworked VP3))
==========================================================================
Audio: no sound
Freeing 0 unused audio chunks.
Starting playback...
Ogg : bad packet in stream 0
* [pp] Exporting mp_image_t, 320x240x12bpp YUV planar, 115200 bytes
* [vo] Allocating mp_image_t, 320x240x12bpp YUV planar, 115200 bytes

MPlayer interrupted by signal 11 in module: decode_video

• MPlayer crashed by bad usage of CPU/FPU/RAM. Recompile MPlayer with --enable-debug and make a 'gdb' backtrace and disassembly. Details in DOCS/HTML/en/bugreports_what.html#bugreports_crash.
• MPlayer crashed. This shouldn't happen. It can be a bug in the MPlayer code _or_ in your drivers _or_ in your gcc version. If you think it's MPlayer's fault, please read DOCS/HTML/en/bugreports.html and follow the instructions there. We can't and won't help unless you provide this information when reporting a possible bug.

uninit video: theora
zsh: segmentation fault HOME=/dev/null ./mplayer -vf pp -v log theora.ogg

I fear I won't be able to help much as I don't have the theora decoder installed
and am not motivated to download it, but it could be some obvious problem after
all (hey, you can always hope ;-) )

I can tell that the Theora decoder is quite painless to install on an up-to-date
Linux box at least, and apparently on Solaris/Sparc too. The result is a 120ko
.so and a 20ko .h.

A few callers more might be helpful, too...

I re-tried with the CVS version, here is the result. First, the version without
-vf pp:

HOME=/dev/null valgrind --num-callers=50 ./mplayer -really-quiet -vo null
-nosound theora.ogg
==8534== Memcheck, a memory error detector for x86-linux.
==8534== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al.
==8534== Using valgrind-2.2.0, a program supervision framework for x86-linux.
==8534== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al.
==8534== For more details, rerun with: -v
==8534==
MPlayer dev-CVS-050109-06:00-3.3.4 (C) 2000-2005 MPlayer Team
CPU: Advanced Micro Devices Athlon Thunderbird (Family: 6, Stepping: 2)
Detected cache-line size is 64 bytes
CPUflags: MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 0 SSE2: 0
Compiled for x86 CPU with extensions: MMX MMX2 3DNow 3DNowEx

==8534== Invalid free() / delete / delete[]
==8534== at 0x1B907460: free (vg_replace_malloc.c:153)
==8534== by 0x81EBCB5: av_freep (utils.c:148)
==8534== by 0x81EC5FC: avcodec_open (utils.c:508)
==8534== by 0x81050AC: init (vd_ffmpeg.c:370)
==8534== by 0x8101A76: init_video (dec_video.c:237)
==8534== by 0x8101D23: init_best_video_codec (dec_video.c:283)
==8534== by 0x808FE45: main (mplayer.c:2012)
==8534== Address 0x865E090 is not stack'd, malloc'd or (recently) free'd
==8534==
==8534== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 87 from 1)
==8534== malloc/free: in use at exit: 181800 bytes in 6705 blocks.
==8534== malloc/free: 12022 allocs, 5318 frees, 4149795 bytes allocated.
==8534== For a detailed leak analysis, rerun with: --leak-check=yes
==8534== For counts of detected errors, rerun with: -v

Now, the same with -vf pp:

HOME=/dev/null valgrind --num-callers=50 ./mplayer -vf pp -really-quiet -vo null
-nosound theora.ogg
==8536== Memcheck, a memory error detector for x86-linux.
==8536== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al.
==8536== Using valgrind-2.2.0, a program supervision framework for x86-linux.
==8536== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al.
==8536== For more details, rerun with: -v
==8536==
MPlayer dev-CVS-050109-06:00-3.3.4 (C) 2000-2005 MPlayer Team
CPU: Advanced Micro Devices Athlon Thunderbird (Family: 6, Stepping: 2)
Detected cache-line size is 64 bytes
CPUflags: MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 0 SSE2: 0
Compiled for x86 CPU with extensions: MMX MMX2 3DNow 3DNowEx

==8536== Invalid free() / delete / delete[]
==8536== at 0x1B907460: free (vg_replace_malloc.c:153)
==8536== by 0x81EBCB5: av_freep (utils.c:148)
==8536== by 0x81EC5FC: avcodec_open (utils.c:508)
==8536== by 0x81050AC: init (vd_ffmpeg.c:370)
==8536== by 0x8101A76: init_video (dec_video.c:237)
==8536== by 0x8101D23: init_best_video_codec (dec_video.c:283)
==8536== by 0x808FE45: main (mplayer.c:2012)
==8536== Address 0x865E090 is not stack'd, malloc'd or (recently) free'd
==8536==
==8536== Invalid write of size 8
==8536== at 0x836E127: postProcess_MMX2 (postprocess_template.c:1186)
==8536== by 0x8372949: pp_postprocess (postprocess.c:703)
==8536== by 0x812F99D: put_image (vf_pp.c:126)
==8536== by 0x8101DCF: decode_video (dec_video.c:332)
==8536== by 0x8094658: main (mplayer.c:2318)
==8536== Address 0x52BFB7D8 is just below %esp. Possibly a bug in GCC/G++
==8536== v 2.96 or 3.0.X. To suppress, use: --workaround-gcc296-bugs=yes
==8536==
==8536== Invalid write of size 8
==8536== at 0x836E208: postProcess_MMX2 (postprocess_template.c:1186)
==8536== by 0x8372949: pp_postprocess (postprocess.c:703)
==8536== by 0x812F99D: put_image (vf_pp.c:126)
==8536== by 0x8101DCF: decode_video (dec_video.c:332)
==8536== by 0x8094658: main (mplayer.c:2318)
==8536== Address 0x52BFB7E0 is just below %esp. Possibly a bug in GCC/G++
==8536== v 2.96 or 3.0.X. To suppress, use: --workaround-gcc296-bugs=yes
==8536==
==8536== Invalid read of size 8
==8536== at 0x836E20C: postProcess_MMX2 (postprocess_template.c:1186)
==8536== by 0x8372949: pp_postprocess (postprocess.c:703)
==8536== by 0x812F99D: put_image (vf_pp.c:126)
==8536== by 0x8101DCF: decode_video (dec_video.c:332)
==8536== by 0x8094658: main (mplayer.c:2318)
==8536== Address 0x52BFB7D8 is just below %esp. Possibly a bug in GCC/G++
==8536== v 2.96 or 3.0.X. To suppress, use: --workaround-gcc296-bugs=yes
==8536==
<snip a lot of invalid read/write of size 8>

Why? Last time I check almost everything reported was caused by ioctls it
couldn't handle and the alsa ao (where I am still unable to determine whether it
is MPlayer or alsa lib or both that is broken as hell).

There is the invalid free reported earlier in this post, and I see quite a few
off-by-one accesses. In fact, if I try to read some DivX 3 video under valgrind,
mplayer segfaults in init_video_codec. The same video is ok without valgrind, so
I expect it does not matter for the moment.

[makovick@…]

Theora decoder produces _negative_ strides, which -vf pp (and probably other
filters) cannot handle. It can be solved by copying each frame to a temporary
buffer with some performance penalty. I am not sure if it's feasible, maybe
there is a better solution.

Also, I currently have the whole Theora decoder in MPlayer build tree, so there
is no need to compile an additional library for Theora. Maybe it could be
eventually committed as a whole - it would also eliminate problems with Theora
(in)compatibility.

[makovick@…]

fix for negative strides & aspect ratio bug

[makovick@…]

set correct disp_w & disp_h

[makovick@…]

set correct disp_w & disp_h