-fno-tree-vectorize FLAG Breaks css function ioctl_ReadCopyright

[visenri@…]

Here I am again with a very strange bug.

Using gcc 4.6.1 to build mplayer > r33574, i get blocky picture and read errors with encrypted dvds (OS is windowsXP).

Dumping the stream and examining the files i discovered that the problematic files were the ones with encryption.

After two days examining the code i found that it does not seem to be an mplayer bug, but a compiler bug.

With mplayer >r33574, in configure file threre is a new compiler flag:

if test "$cc_vendor" = "gnu" ; then

cflag_check -fno-tree-vectorize && CFLAGS="$CFLAGS -fno-tree-vectorize"

Here are the logs i get changing the flag, with two discs, one encrypted and one unencrypted:

DVD WITH COPY PROTECTION:

---

Compiled without -fno-tree-vectorize

libdvdcss debug: opening target `K:'
libdvdcss debug: using Win2K API for access
libdvdcss debug: disc reports copyright information 0x1
libdvdcss debug: drive region mask 0xfd, RPC-II, region code set
libdvdcss debug: requesting AGID
libdvdcss debug: drive authenticated, using variant 0
libdvdcss debug: authentication established
libdvdcss debug: GetASF authenticated, ASF=1
libdvdcss debug: decrypting disc key f4:d3:7b:1e:0d
libdvdcss debug: trying player key 01:af:e3:12:80
libdvdcss debug: decrypted disc key is 00:00:00:00:0b

...
No errors and picture is ok.

---

Compiled with -fno-tree-vectorize

libdvdcss debug: opening target `K:'
libdvdcss debug: using Win2K API for access
libdvdcss debug: disc reports copyright information 0x0
libdvdcss debug: drive region mask 0xfd, RPC-II, region code set

...
Errors reading disc and blocky image.

DVD WITHOUT COPY PROTECTION:

---

Compiled without -fno-tree-vectorize

libdvdread: Using libdvdcss version 1.2.10 for DVD access
libdvdcss debug: opening target `K:'
libdvdcss debug: using Win2K API for access
libdvdcss debug: disc reports copyright information 0x0
libdvdcss debug: drive region mask 0xfd, RPC-II, region code set

...
Skips Title key search and plays Titles ok.

---

Compiled with -fno-tree-vectorize

libdvdread: Using libdvdcss version 1.2.10 for DVD access
libdvdcss debug: opening target `K:'
libdvdcss debug: using Win2K API for access
libdvdcss debug: disc reports copyright information 0xf5
libdvdcss debug: drive region mask 0xfd, RPC-II, region code set
libdvdcss debug: requesting AGID
libdvdcss debug: drive authenticated, using variant 0
libdvdcss debug: authentication established
libdvdcss error: ioctl ReadDiscKey failed
libdvdcss debug: could not get disc key

...
Tries to decode Title keys, but all result in non encrypted titles.
Finally plays Titles ok.

As you can see function ioctl_ReadCopyright gets extrange results with -fno-tree-vectorize.

Any Ideas?

I'v googled about the flag and found this:
​http://www.mentby.com/Group/mplayer-users/fresh-svn-mplayer-crashes-on-many-ac3-streams.html
so, i suppose this flag was introduced for this reason.

Can we put (as a temporal fix) -fno-tree-vectorize for the code that handles the ac3 and
leave vectorization activated (-ftree-vectorize) for the rest?