Win32 VO -wid problem with negative window IDs

[db3l.net@…]

Trying to supply a -wid command line option with a negative value fails with the vo_gl and vo_direct3d VO drivers under Windows. Instead of attaching to the desired window, mplayer simply ignores the command line option and opens up its own window instead.

The w32_common and vo_gl modules have checks for WinID >= 0 and WinID < 0 rather than WinID != -1 and WinID == -1 respectively. This breaks when a particular HWND is a large enough pointer to be perceived as negative when treated as a 32-bit signed number.

(As it turns out, you also *must* supply the wid as a negative decimal number on the command line or the parse_int routine (via strtol) will silently truncate it at LONG_MAX (0x7fffffff) but that's arguably correct behavior as long as wid is being defined as a signed integer)

[db3l.net@…]

Patch for WinID checking in vo_gl and w32_common

[db3l.net@…]

Here is a proposed patch that simply changes the WinID comparisons in vo_gl and w32_common to be explicit checks for -1. The patch is against r28311, but the affected code is the same in the current HEAD. I am running a player with this patch in production and have successfully operated against windows with negative WIDs.

vo_directx under Windows already makes these sorts of checks so seems ok. Although in glancing through the source again, vo_gl2 may need a similar tweak (I haven't had a need to use gl2 myself under Windows).

[reimar]

Should be fixed in a different way in r28795