mencoder and scaling.

[bpringlemeir@…]

The issue I am experiencing is a bunch of flashing garbage when playing back a scaled mpegts stream. I guess I don't know if this is a player or encoder problem. I suspect it is encoding as I have never seen mplayer exhibit this problem on any other files.

The captured *source* stream has been edited with the xtscut tool found in the atscap package. It seems that upgrading from atscap-1.1rc9u to atscap-1.1rc9t may have increased the frequency of this encode issue? However, it was occurring with previous versions. I recently upgraded from SVN and the issues seems to have increased in frequency as well (make distclean, configure, make). I have also had issues with the SATA controller on this system, but I think that the issue was a cable that has been replaced (and if that was the issue, I think I would have different problems).

The files have been uploaded to 'Incoming' under the title WoodsmithShop-1212-1100.* My system is a P4 with an SMP 2.6.31 kernel. I have several captured streams which I have seen the same problems with (the encoded file is WoodsmithShop-1212-1100.ts, which is actually an avi).

Thanks.

[bpringlemeir@…]

I forgot that the issue occurs with/without yadif and I have moved yadif from before the scale to after on advice from the Interlaced Videos section of "​http://hubpages.com/hub/1001-mencoder-tips".

[bpringlemeir@…]

I trimmed the source material and recoded it with commands like this,

nice -n+10 /home/src/mplayer/mencoder -alang en -vf crop=704:480:0:0,scale=416:-11,hqdn3d=2:1:2 -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=1000:mbd=2:trell:v4mv:psnr:turbo:vpass=1 -ofps 24000/1001 -oac copy -o /dev/null source.ts

nice -n+10 /home/src/mplayer/mencoder -alang en -vf crop=704:480:0:0,scale=416:-11,hqdn3d=2:1:2 -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=1000:mbd=2:trell:v4mv:psnr:turbo:vpass=2 -ofps 24000/1001 -oac copy -o destination.avi source.ts

Hmm, I still can not upload to incoming. I think that the quota has been reached. I hope the mpegts I uploaded is not the cause. The smaller files still exhibit the same behaviour.

[bpringlemeir@…]

I have tried various lavc encoding paramaters with the following filter

crop=704:352:0:66,dsize=704:352,scale=-11:240

It seems that scaling this size video to '240' causes the encoding issue. If I play back the source with the same parameters using mplayer, there are no artifacts. I have tried scaling with/without the dsize and used '-10', '-11' and a fixed size. I have also tried scaling to different dimensions such as '-10:320' and the same thing occurred. If I just crop and do not scale, the black band artifacts go away. These are occuring at about 6seconds in the sample I uploaded. I still can not upload any new files to incoming.

I guess I will try a dumpstream with mplayer and encode that.

[bpringlemeir@…]

dumpstream will not work. Apparently that is done before any filters (which makes sense). I don't know if the issues is an inter-play between the scaling filter and mpeg-ts stream with mencoder. I guess I can try to re-code the stream to an avi and/or mjpeg (with the original size and scaled) and see which if any versions have this issue.

[bpringlemeir@…]

I encoded the scaled video to mjpeg. I then used this as the source to a 2-pass encode to lavc with mpeg4 target. The same black bands appear at the same point in the sequence. I think that this is an issue with the lavc mpeg4 encoder with certain size video.

The mjpeg and original mpeg-ts files are about 200Mb.

[bpringlemeir@…]

Arrgh. The mjpeg exhibits the black bands at a different point in the video, but it does occur in the mjpeg version. I encoded this (mjpeg) with lavc as well.

[bpringlemeir@…]

I tried without cropping,

set args -vf scale=-10:224 -ovc lavc -lavcopts vcodec=mjpeg -oac copy -o foobar.mjpeg foobar.ts

This results in a segv.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb706a6c0 (LWP 2645)]
0x087209b0 in hScale_MMX2 ()
Current language: auto; currently asm
(gdb) where
#0 0x087209b0 in hScale_MMX2 ()
#1 0xb68a044c in ?? ()

This might be a seperate issue. I guess I can recompile without MMX2 and try the same commands to see if this is the issue (or something completely different).

[bpringlemeir@…]

I also have crashing with mplayer if I change the 'sws' value. I am recompiling with debug.

[bpringlemeir@…]

Here is the crash in mplayer. Note the SEGV appears in the video in about the same location that the mencoder starts to show garbage when transcoding.

(gdb) bt
#0 0x0864611c in hScale_MMX2 (dst=0xaedcc70, dstW=320, src=0xb57de440 <Address 0xb57de440 out of bounds>, srcW=704, xInc=144179,

filter=0xaebab80, filterPos=0xae5af50, filterSize=0) at swscale_template.c:2157

#1 0x0864ffd3 in swScale_MMX2 (c=0xadf11a0, src=0xbff96640, srcStride=0xbff96610, srcSliceY=224, srcSliceH=16, dst=0xbff96630,

dstStride=0xbff96620) at swscale_template.c:2280

#2 0x08647304 in sws_scale (c=0xadf11a0, src=0xbff966cc, srcStride=0xae45750, srcSliceY=224, srcSliceH=16, dst=0xaf33904,

dstStride=0xaf33914) at swscale.c:3090

#3 0x0816b825 in scale (sws1=0xadf11a0, sws2=0x0, src=<value optimized out>, src_stride=0xae45750, y=224, h=16, dst=0xaf33904,

dst_stride=0xaf33914, interlaced=0) at libmpcodecs/vf_scale.c:350

#4 0x0824145a in draw_slice (s=0xae12970, src=0xae45740, offset=0xbff9677c, y=224, type=3, height=16) at libmpcodecs/vd_ffmpeg.c:481
#5 0x083897f1 in ff_draw_horiz_band (s=0xae12d00, y=224, h=16) at mpegvideo.c:2081
#6 0x0844feb2 in mpeg_decode_slice (s1=0xae12d00, mb_y=14, buf=0xbff96b64, buf_size=2119) at mpeg12.c:1798
#7 0x0845463c in decode_chunks (avctx=0xae12970, picture=0xae12890, data_size=0xbff96cb4, buf=0xb5b93008 "", buf_size=3084)

at mpeg12.c:2465

#8 0x08454c5d in mpeg_decode_frame (avctx=0xae12970, data=0xae12890, data_size=0xbff96cb4, avpkt=0xbff96c60) at mpeg12.c:2267
#9 0x083573a1 in avcodec_decode_video2 (avctx=0xae12970, picture=0xae12890, got_picture_ptr=0xbff96cb4, avpkt=0xbff96c60) at utils.c:584
#10 0x08240984 in decode (sh=0xade3960, data=0xb5b93008, len=3084, flags=0) at libmpcodecs/vd_ffmpeg.c:787
#11 0x08143810 in decode_video (sh_video=0xade3960, start=0xb5b93008 "", in_size=3084, drop_frame=0, pts=36290.91015625)

at libmpcodecs/dec_video.c:365

#12 0x080a4095 in main (argc=6, argv=0xbff9b034) at mplayer.c:2357

(gdb) disass $pc-32 $pc+32
Dump of assembler code from 0x86460fc to 0x864613c:
0x086460fc <hScale_MMX2+82>: add %al,(%eax)
0x086460fe <hScale_MMX2+84>: add %al,(%eax)
0x08646100 <hScale_MMX2+86>: mov 0x20(%ebp),%ecx
0x08646103 <hScale_MMX2+89>: movzwl (%ecx,%esi,1),%eax
0x08646107 <hScale_MMX2+93>: movzwl 0x2(%ecx,%esi,1),%edx
0x0864610c <hScale_MMX2+98>: mov 0x10(%ebp),%ecx
0x0864610f <hScale_MMX2+101>: pxor %mm4,%mm4
0x08646112 <hScale_MMX2+104>: pxor %mm5,%mm5
0x08646115 <hScale_MMX2+107>: movq (%edi),%mm1
0x08646118 <hScale_MMX2+110>: movq (%edi,%ebx,1),%mm3
0x0864611c <hScale_MMX2+114>: movd (%ecx,%eax,1),%mm0
0x08646120 <hScale_MMX2+118>: movd (%ecx,%edx,1),%mm2
0x08646124 <hScale_MMX2+122>: punpcklbw %mm7,%mm0
0x08646127 <hScale_MMX2+125>: punpcklbw %mm7,%mm2
0x0864612a <hScale_MMX2+128>: pmaddwd %mm1,%mm0
0x0864612d <hScale_MMX2+131>: pmaddwd %mm2,%mm3
0x08646130 <hScale_MMX2+134>: paddd %mm3,%mm5
0x08646133 <hScale_MMX2+137>: paddd %mm0,%mm4
0x08646136 <hScale_MMX2+140>: add $0x8,%edi
0x08646139 <hScale_MMX2+143>: add $0x4,%ecx
End of assembler dump.

(gdb) info all-registers
eax 0x0 0
ecx 0xb57de440 -1250040768
edx 0x0 0
ebx 0x18 24
esp 0xbff96298 0xbff96298
ebp 0xbff962b8 0xbff962b8
esi 0xfffffd80 -640
edi 0xaebab80 183217024
eip 0x864611c 0x864611c <hScale_MMX2+114>
eflags 0x10202 [ IF RF ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0 0
gs 0x33 51
st0 -nan(0xffc6ffc6ffc6ffc6) (raw 0xffffffc6ffc6ffc6ffc6)
st1 -nan(0xfeb00b4f1b051cf3) (raw 0xfffffeb00b4f1b051cf3)
st2 -nan(0xfffafffafffafff8) (raw 0xfffffffafffafffafff8)
st3 -nan(0x1c8c15d104a9fc8a) (raw 0xffff1c8c15d104a9fc8a)
st4 -inf (raw 0xffff0000000000000000)
st5 -inf (raw 0xffff0000000000000000)
st6 -nan(0x2b002c002d002e) (raw 0xffff002b002c002d002e)
st7 -inf (raw 0xffff0000000000000000)
fctrl 0x37f 895
fstat 0x420 1056
ftag 0xaaaa 43690
fiseg 0x73 115
fioff 0x81437c4 135542724
foseg 0x7b 123
fooff 0xbff96ce0 -1074172704
fop 0x1c9 457
xmm0 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0,

0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}

xmm1 {v4_float = {0xc6a00000, 0x3de0b10, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x0}, v16_int8 = {0x58, 0x4e, 0x35, 0xd4,

0xc4, 0x82, 0x77, 0x4c, 0xf1, 0xfa, 0xee, 0xb6, 0x89, 0x75, 0x3e, 0xba}, v8_int16 = {0x4e58, 0xd435, 0x82c4, 0x4c77, 0xfaf1, 0xb6ee,
0x7589, 0xba3e}, v4_int32 = {0xd4354e58, 0x4c7782c4, 0xb6eefaf1, 0xba3e7589}, v2_int64 = {0x4c7782c4d4354e58, 0xba3e7589b6eefaf1},

uint128 = 0xba3e7589b6eefaf14c7782c4d4354e58}

xmm2 {v4_float = {0xd10320c0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x7d, 0xf3, 0x3b, 0xce, 0x9b, 0xa9, 0xe9, 0x29,

0x24, 0x10, 0x2, 0x73, 0xd7, 0xd4, 0xf0, 0x3d}, v8_int16 = {0xf37d, 0xce3b, 0xa99b, 0x29e9, 0x1024, 0x7302, 0xd4d7, 0x3df0},

v4_int32 = {0xce3bf37d, 0x29e9a99b, 0x73021024, 0x3df0d4d7}, v2_int64 = {0x29e9a99bce3bf37d, 0x3df0d4d773021024},
uint128 = 0x3df0d4d77302102429e9a99bce3bf37d}

xmm3 {v4_float = {0x0, 0x0, 0x0, 0x1e698b20}, v2_double = {0x0, 0x8000000000000000}, v16_int8 = {0x3f, 0x6a, 0x52, 0x94, 0xd9,

0x8, 0x3, 0x93, 0x18, 0x7, 0x72, 0xa4, 0x59, 0x4c, 0xf3, 0x4d}, v8_int16 = {0x6a3f, 0x9452, 0x8d9, 0x9303, 0x718, 0xa472, 0x4c59,
0x4df3}, v4_int32 = {0x94526a3f, 0x930308d9, 0xa4720718, 0x4df34c59}, v2_int64 = {0x930308d994526a3f, 0x4df34c59a4720718},

---Type <return> to continue, or q <return> to quit---

uint128 = 0x4df34c59a4720718930308d994526a3f}

xmm4 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0,

0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}

xmm5 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0,

0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}

xmm6 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0,

0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}

xmm7 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0,

0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}

mxcsr 0x1fa0 [ PE IM DM ZM OM UM PM ]
mm0 {uint64 = 0xffc6ffc6ffc6ffc6, v2_int32 = {0xffc6ffc6, 0xffc6ffc6}, v4_int16 = {0xffc6, 0xffc6, 0xffc6, 0xffc6}, v8_int8 = {

0xc6, 0xff, 0xc6, 0xff, 0xc6, 0xff, 0xc6, 0xff}}

mm1 {uint64 = 0xfeb00b4f1b051cf3, v2_int32 = {0x1b051cf3, 0xfeb00b4f}, v4_int16 = {0x1cf3, 0x1b05, 0xb4f, 0xfeb0}, v8_int8 = {

0xf3, 0x1c, 0x5, 0x1b, 0x4f, 0xb, 0xb0, 0xfe}}

mm2 {uint64 = 0xfffafffafffafff8, v2_int32 = {0xfffafff8, 0xfffafffa}, v4_int16 = {0xfff8, 0xfffa, 0xfffa, 0xfffa}, v8_int8 = {

0xf8, 0xff, 0xfa, 0xff, 0xfa, 0xff, 0xfa, 0xff}}

mm3 {uint64 = 0x1c8c15d104a9fc8a, v2_int32 = {0x4a9fc8a, 0x1c8c15d1}, v4_int16 = {0xfc8a, 0x4a9, 0x15d1, 0x1c8c}, v8_int8 = {

0x8a, 0xfc, 0xa9, 0x4, 0xd1, 0x15, 0x8c, 0x1c}}

mm4 {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0, 0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm5 {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0, 0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm6 {uint64 = 0x2b002c002d002e, v2_int32 = {0x2d002e, 0x2b002c}, v4_int16 = {0x2e, 0x2d, 0x2c, 0x2b}, v8_int8 = {0x2e, 0x0,

0x2d, 0x0, 0x2c, 0x0, 0x2b, 0x0}}

mm7 {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0, 0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}

[bpringlemeir@…]

I enabled the DEBUG_SWSCALE_BUFFERS and re-ran mplayer over the section causing issues.

[swscaler @ 0x88c8e40]dstY: 103
[swscaler @ 0x88c8e40] firstLumSrcY: 206 lastLumSrcY: 207 lastInLumBuf: 205
[swscaler @ 0x88c8e40] firstChrSrcY: 102 lastChrSrcY: 103 lastInChrBuf: 103
[swscaler @ 0x88c8e40] lumBufIndex 2: lastInLumBuf: 205
[swscaler @ 0x88c8e40] lumBufIndex 3: lastInLumBuf: 206
[swscaler @ 0x88c8e40]dstY: 104
[swscaler @ 0x88c8e40] firstLumSrcY: 208 lastLumSrcY: 207 lastInLumBuf: 207
[swscaler @ 0x88c8e40] firstChrSrcY: 104 lastChrSrcY: 103 lastInChrBuf: 103
[swscaler @ 0x88c8e40]swScale() 0xb5797040[704] 0xb57fe240[352] 0xb57e9840[352] (nil)[0] -> 0x91a8340[320] 0x91bfa40[160] 0x91baf40[160] (nil)[0]
[swscaler @ 0x88c8e40]srcSliceY: 80 srcSliceH: 16 dstY: 104 dstH: 240
[swscaler @ 0x88c8e40]vLumFilterSize: 2 vLumBufSize: 2 vChrFilterSize: 2 vChrBufSize: 2
[swscaler @ 0x88c8e40]dstY: 104
[swscaler @ 0x88c8e40] firstLumSrcY: 208 lastLumSrcY: 95 lastInLumBuf: 207
[swscaler @ 0x88c8e40] firstChrSrcY: 104 lastChrSrcY: 47 lastInChrBuf: 103
[swscaler @ 0x88c8e40]swScale() 0xb5797040[704] 0xb57fe240[352] 0xb57e9840[352] (nil)[0] -> 0x91a8340[320] 0x91bfa40[160] 0x91baf40[160] (nil)[0]
[swscaler @ 0x88c8e40]srcSliceY: 224 srcSliceH: 16 dstY: 104 dstH: 240
[swscaler @ 0x88c8e40]vLumFilterSize: 2 vLumBufSize: 2 vChrFilterSize: 2 vChrBufSize: 2
[swscaler @ 0x88c8e40]dstY: 104
[swscaler @ 0x88c8e40] firstLumSrcY: 208 lastLumSrcY: 209 lastInLumBuf: 207
[swscaler @ 0x88c8e40] firstChrSrcY: 104 lastChrSrcY: 105 lastInChrBuf: 103
[swscaler @ 0x88c8e40] lumBufIndex 2: lastInLumBuf: 207

MPlayer interrupted by signal 11 in module: decode_video

• MPlayer crashed by bad usage of CPU/FPU/RAM. Recompile MPlayer with --enable-debug and make a 'gdb' backtrace and disassembly. Details in DOCS/HTML/en/bugreports_what.html#bugreports_crash.
• MPlayer crashed. This shouldn't happen. It can be a bug in the MPlayer code _or_ in your drivers _or_ in your gcc version. If you think it's MPlayer's fault, please read DOCS/HTML/en/bugreports.html and follow the instructions there. We can't and won't help unless you provide this information when reporting a possible bug.

[bpringlemeir@…]

I have observed that there are 'ac-tex damaged' errors around the same time that these problems occur. Perhaps the mpegts decoder has changed and cause this problem (passing garbage off to the scaling module). However, the black band sequence persist for many frames (~10-30 seconds). If there is a segv, this may be due to corruption.

Encoding the unscaled, uncrop image to mjpeg avi and then proceeding to encode with scaling and cropping works.

[bpringlemeir@…]

see also,

​http://bugzilla.mplayerhq.hu/show_bug.cgi?id=1039

[bpringlemeir@…]

I have observed problems in many capture mpeg-ts streams. The atscap has a log of errors that occur. This is a 'DVB' capture of an ATSC stream. Some of the captures have a very good signal (being less than 10km from the CN tower with a CM-4221 antenna). In some cases, atscap indicates no errors. This doesn't preclude something being corrupt/non-standard in the mpegts source. Playing channels direct from v4l2 devices with mplayer results in eventual playback oddities (some sort of buffer under-run where the video only displays about 1-2 frames per second). However, if I do this with the atscap this event never happens. I mention just as a clue that perhaps the mpeg-ts demux (or the mpeg video decoder) is at fault. This might be a completely unrelated issue.

[bpringlemeir@…]

Another possibly related bug,

​http://bugzilla.mplayerhq.hu/show_bug.cgi?id=1588

[bpringlemeir@…]

Add the additional,

-lavdopts er=0:threads=2

Makes the black bands go away, using all scaling and original commands in the corrupt sequence. There are a variety of places where the individual sequence flashes, but no persistent. I think that lavc decode/mpegts demux has an issue that passes a bad pointer to the scaler. I will experiment with different 'er=xx' options to determine if a certain level of correction causes this. Using 'er=4' and the entire video sequence had flashing black bands. I guess the only other to try is 'er=1'.

[bpringlemeir@…]

-lavdopts er=2:ec=1

Causes a SEGV with mencoder. Again in RENAME(hScale)==hScale_MMX2. This time the stack is corrupt and gdb can not print out a back trace. This seems to jive with what I was seeing previously as the dst values was close to a stack address.

Is anybody listening? Can I compile with stack-protector?

[bpringlemeir@…]

Err, sorry I don't know what the esi was.

bpringle@pvr:/proc/2326$ cat maps
08048000-087ce000 r-xp 00000000 08:01 22824879 /home/src/mplayer/mencoder
087ce000-087fb000 rw-p 00785000 08:01 22824879 /home/src/mplayer/mencoder
087fb000-08c88000 rw-p 00000000 00:00 0
0a4d9000-0a7ba000 rw-p 00000000 00:00 0 [heap]
b56ce000-b580c000 rw-p 00000000 00:00 0
b580c000-b581d000 r--p 00000000 08:01 3408180 /usr/share/fonts/truetype/ttf-bitstream-vera/Vera.ttf

bpringle@pvr:/proc/2326$ cat maps
08048000-087ce000 r-xp 00000000 08:01 22824879 /home/src/mplayer/mencoder
087ce000-087fb000 rw-p 00785000 08:01 22824879 /home/src/mplayer/mencoder
087fb000-08c88000 rw-p 00000000 00:00 0
0a4d9000-0a7ba000 rw-p 00000000 00:00 0 [heap]
b56ce000-b580c000 rw-p 00000000 00:00 0
b580c000-b581d000 r--p 00000000 08:01 3408180 /usr/share/fonts/truetype/ttf-bitstream-vera/Vera.ttf

(gdb) disass $pc $pc+1
Dump of assembler code from 0x85d2722 to 0x85d2723:
0x085d2722 <hScale_MMX2+392>: movd (%esi,%eax,1),%mm0
End of assembler dump.
(gdb) p $esi
$2 = -1251268544
(gdb) p /x $esi
$3 = 0xb56b2840
(gdb) p /x $eax
$4 = 0x0

It is just before the [heap] map entry.

[bpringlemeir@…]

Reducing the size of the test encode files does not help. The segv happens in the vpass=2. Also an lavdopts with threads=2 speeds the occurance of the SEGV, but it still happens with out threaded decoding. Attempting to isolate in a debugger.

[bpringlemeir@…]

In r30099 of swscale_template.c in RENAME(swScale) there is a 'src1' calculation as follows,

uint8_t *src1= src[0]+(lastInLumBuf + 1 - srcSliceY)*srcStride[0];

(gdb) p lastInLumBuf
$18 = 286
(gdb) p srcSliceY
$19 = 302

This results in negative value which is multiplied by srcStride and steps back out of the segment in some cases (and will get random values in others). In the /proc/xxxx/map I dumped previously, the src[0] is in the b56ce000 section, but the src1 calculation takes it back out of the section and the RENAME(hScale) ends up with a SEGV.

The previous mpeg_decode_slice() has returned an error and called ff_er_add_slice() which set a portion of the end of the error map. I think this is related.

[mpeg2video @ 0xa254ae0]00 motion_type at 20 22

[bpringlemeir@…]

Adding,

#undef NDEBUG
#include <assert.h>

to the top of swscale_template.c and recompile will trigger the RENAME(swScale) assertion,

assert(lastInChrBuf + 1 - chrSrcSliceY >= 0);

I think that assertions should be enabled if '--enable-debug=3' is set? Assertions in swscale_template.c seem to be useless due to swscale_internal.h including libavutil/avutil.h -> common.h -> internal.h

#if !defined(DEBUG) && !defined(NDEBUG)
# define NDEBUG
#endif

Or there should be a config option to define DEBUG. I think that re-running test cases with assertions active might be very useful for trouble shooting (especially in the case of a SEGV).

[bpringlemeir@…]

This fix works for luma (format=y800),

Do horizontal scaling
while(lastInLumBuf < lastLumSrcY) {

+ int tmpLumBuf = (lastInLumBuf < srcSliceY - 1) ? srcSliceY - 1 : lastInLumBuf;
+ uint8_t *src1= src[0]+(tmpLumBuf + 1 - srcSliceY)*srcStride[0];
+ uint8_t *src2= src[3]+(tmpLumBuf + 1 - srcSliceY)*srcStride[3];

• uint8_t *src1= src[0]+(lastInLumBuf + 1 - srcSliceY)*srcStride[0];
• uint8_t *src2= src[3]+(lastInLumBuf + 1 - srcSliceY)*srcStride[3];

lumBufIndex++;

I don't know why it doesn't work with chroma. Maybe because of the shifts or I have missed something. I will look at it after some sleep.

[bpringlemeir@…]

format=y800 causes a special grayscale 1-1 scaler to run which doesn't run the MMX filter code.

[bpringlemeir@…]

This is an ffmpeg bug.
​http://ffmpeg.org/developer.html

[bpringlemeir@…]

This is a problem in ffmeg. Use of '-vfm libmpeg2' solves this issue.

[compn]

doh, we lost the sample. still crash for you in svn ?

[bpringlemeir@…]

(In reply to comment #25)

doh, we lost the sample. still crash for you in svn ?

I don't have the sample. However, I am sure I can get you another one if needed. In mplayer/ffmpeg/libavcodec/mpeg12.c, the function mpeg_decode_mb() will return an error for an mpeg-ts stream on occasion. In mpeg_decode_slice(), it returns immediately without setting up any data. A filter such as SWS that downscales runs a convolution on a larger matrix to produce a single pixel. When the SWS code runs, it tries to access memory from the previous slice which punts on an mpeg_decode_mb() error.

I have deleted the sample, but if I change back to using ffmpeg I will probably find a sample in a few days. I encode about 3-5 TS streams everyday. However, as the generated AVIs may be corrupt, I will need to keep all of the TS until I watch the AVIs. Do you have enough information, or do you want a sample?

If mpeg_decode_slice(), provided some default data on mpeg_decode_mb() failure, then a convolution filter like SWS would not act on bad data.

[bpringlemeir@…]

I have place a file bug_1613_sample.ts in MPlayer/Incoming. This is trimmed from a larger 30minute ATSC stream.

bpringle 2247533232 Jan 18 06:59 /dtv/CliffordBig-0118-0630.2.ts
bpringle 5175076 Jan 18 20:55 bug_1613_sample.ts

This plays fine,

/home/src/mplayer.HEAD/mplayer -benchmark bug_1613_sample.ts

This does not,

/home/src/mplayer.HEAD/mplayer -vf scale=320 -benchmark bug_1613_sample.ts

My source might be a day or two old (r32792). I will update and retry.

[bpringlemeir@…]

My source might be a day or two old (r32792). I will update and retry.

Still occurs with 32796.

[bpringlemeir@…]

This is fixed in the latest SVN/Git. The issue was fixed by Michael Niedermayer in git commit 7f125c3e87f0ce48abfdbcf02e3d4c1c33f025ba with log message,

Move frame_pred_frame_dct check elsewhere.

See also,

​https://roundup.libav.org/issue2405

[bpringlemeir@…]

Option parsing seems to have changed. I need to use 'scale=320:240' in order for the scaling filter to resize. When this happens, the bug still occurs on the head.