The output when using -v would be a lot more useful.
What about -vf format=bgr24 and -vf format=rgb24??
Are the colours wrong in the same way for -vf format=rgb32 and -vf format=bgr32?
Also try -vf format=bgr32,rgb2bgr=swap etc.