Signal 11 in mpeg2_set_buf ( libmpeg2/decode.c ) with patch

[dax@…]

Under heavy system load, or while rapid seeking within an MPEG2 video the following occurs on an intermittent basis:

[daxbert@daxhome2 ~]$ gdb /usr/local/bin/gmplayer.bad
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
(gdb) run
Starting program: /usr/local/bin/gmplayer.bad
[New LWP 103326]
[New Thread 0xf01120 (LWP 103326)]
MPlayer 1.0rc2-4.2.1 (C) 2000-2007 MPlayer Team
CPU: AMD Athlon(tm) 64 Processor 3700+ (Family: 15, Model: 39, Stepping: 1)
CPUflags: MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 1 SSE2: 1
Compiled for x86 CPU with extensions: MMX MMX2 3DNow 3DNowEx SSE SSE2

Playing /home/daxbert/Test.mpg.
MPEG-PS file format detected.
VIDEO: MPEG1 352x288 (aspect 8) 25.000 fps 1150.0 kbps (143.8 kbyte/s)
==========================================================================
Trying to force video codec driver family no...
Opening video decoder: [mpegpes] MPEG 1/2 Video passthrough
VDec: vo config request - 352 x 288 (preferred colorspace: Mpeg PES)
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
The selected video_out device is incompatible with this codec.
Try appending the scale filter to your filter list,
e.g. -vf spp,scale instead of -vf spp.
VDecoder init failed :(
Opening video decoder: [libmpeg2] MPEG 1/2 Video decoder libmpeg2-v0.4.0b
Selected video codec: [mpeg12] vfm: libmpeg2 (MPEG-1 or 2 (libmpeg2))
==========================================================================
==========================================================================
Forced audio codec: hwac3
Forced audio codec: a52
Trying to force audio codec driver family hwac3...
Opening audio decoder: [mp3lib] MPEG layer-2, layer-3
AUDIO: 44100 Hz, 2 ch, s16le, 224.0 kbit/15.87% (ratio: 28000->176400)
Selected audio codec: [mp3] afm: mp3lib (mp3lib MPEG layer-2, layer-3)
==========================================================================
AO: [oss] 44100Hz 2ch s16le (2 bytes per sample)
Starting playback...
VDec: vo config request - 352 x 288 (preferred colorspace: Planar YV12)
VDec: using Planar YV12 as output csp (no 0)
Movie-Aspect is 1.30:1 - prescaling to correct movie aspect.
VO: [xv] 352x288 => 376x288 Planar YV12
A: 69.5 V: 63.2 A-V: 6.269 ct: 0.100 87/ 87 ??% ??% ??,?% 0 0
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xf01120 (LWP 103326)]
0x000000000098a26c in mpeg2_set_buf (mpeg2dec=0x1117040, buf=0xf08b50, id=0xf08b30) at decode.c:360
360 if (mpeg2dec->custom_fbuf && !mpeg2dec->fbuf[1]->buf[0]){
(gdb) bt
#0 0x000000000098a26c in mpeg2_set_buf (mpeg2dec=0x1117040, buf=0xf08b50, id=0xf08b30) at decode.c:360
#1 0x000000000051b49d in decode (sh=0xf4f480, data=0x1700000, len=4804, flags=0) at vd_libmpeg2.c:213
#2 0x00000000004ce7ed in decode_video (sh_video=0xf4f480, start=0x1700000 "", in_size=4800, drop_frame=0, pts=69.289756774902344) at dec_video.c:366
#3 0x0000000000442435 in update_video (blit_frame=0x7fffffffe1f4) at mplayer.c:2043
#4 0x00000000004466ec in main (argc=1, argv=0x7fffffffe388) at mplayer.c:3393
(gdb) quit
The program is running. Exit anyway? (y or n) y

I'm not sure why mpeg2dec->fbuf[1] is invalid, but the following patch resolves the issue:

---

PATCH

---

--- libmpeg2/decode.c.orig 2008-02-13 19:56:25.520080427 -0800
+++ libmpeg2/decode.c 2008-02-13 20:23:50.298807026 -0800
@@ -356,7 +356,7 @@

fbuf->buf[2] = buf[2];
fbuf->id = id;
HACK! FIXME! At first I frame, copy pointers to prediction frame too!

• if (mpeg2dec->custom_fbuf && !mpeg2dec->fbuf[1]->buf[0]){

+ if (mpeg2dec->custom_fbuf && mpeg2dec->fbuf[1] && !mpeg2dec->fbuf[1]->buf[0]){

mpeg2dec->fbuf[1]->buf[0]=buf[0];
mpeg2dec->fbuf[1]->buf[1]=buf[1];
mpeg2dec->fbuf[1]->buf[2]=buf[2];

[dax@…]

Patch for Signal 11 in decode.c

[dax@…]

Adding patch as attachment

[compn]

ffmpeg12 is now default, closing libmpeg2 bugs