Crash due to (buggy?) ffmpeg/libswscale update

[porezag@…]

Since roughly 2010/09/29, the HEAD version of gmplayer (GUI mode) crashes upon starting. Crashing is independent of the VO module used and it occurs for all AVI files that I've tested (these are "standard" DIVX movies). Backtrace of the crash is attached.

I have tried to zero in to the problem and it seems that with the new version the SwsContext structure is not correctly initialized, leading to a crash due to a NULL-pointer like reference later on. If the following part of the code in ffmpeg/libswscale/ is reverted back to the older version, the crash disappears:

Index: utils.c
===================================================================
--- utils.c (revision 32423)
+++ utils.c (working copy)
@@ -1550,19 +1550,9 @@

}

if (!context) {

• if (!(context = sws_alloc_context()))
• return NULL;
• context->srcW = srcW;
• context->srcH = srcH;
• context->srcFormat = srcFormat;
• context->dstFormat = dstFormat;
• context->flags = flags;
• context->param[0] = param[0];
• context->param[1] = param[1];
• if (sws_init_context(context, srcFilter, dstFilter) < 0) {
• sws_freeContext(context);
• return NULL;
• }

+ return sws_getContext(srcW, srcH, srcFormat,
+ dstW, dstH, dstFormat, flags,
+ srcFilter, dstFilter, param);

}
return context;

}

[porezag@…]

gdb run log

[reimar]

Argh! I just figured this all out on my own after reading a bug report on the -users list when you already had such a nice summary. Well, didn't take that long at least.
Anyway, it's fixed now. In case you're wondering: The most important part that code missed was setting dstW and dstH.

[porezag@…]

Well sorry that I didn't find it earlier to save you some work. Thanks for the details, dstW and dstH were kind of obvious but I wasn't sure if something else could be missing.