Assertion i>=0 failed at libavcodec/wmaenc.c:381

[sherpya@…]

While trying to encode wmav2 I always get this assertion after 2-3 frames
the i value goes negative

i=731
i=731
i=731
i=-14181
Assertion i>=0 failed at libavcodec/wmaenc.c:382

these values for 128k bitrate

the command line (simplified, originally the output was asf):
mencoder.exe video-in.ext -ovc copy -oac lavc -lavcopts acodec=wmav2:abitrate=128 -o out.avi

I've tried with wine and depending on filenames on command line it crashes instead, attached a backtrace

same kind of encoding works fine on ffmpeg, the original command line of mencoder was using lavf output format.

[sherpya@…]

wine crash report when mencoder crashes instead of asserting

[reimar]

As far as I can tell the assert means that the encoder wrote far too much data for the bitrate.
I think you really should report this to ffmpeg, even though it is unfortunate you cannot preproduce it.
Possibly mencoder gives the encoder invalid data that causes this.
Have you tried if you can trigger this in FFmpeg by encoding a wav file with random data or so?

[reimar]

And the crash is probably the same thing, the assert is triggered when the encoder wrote more data than expected. In this case so much more that it might have written outside the buffer.

[sherpya@…]

unfortunately ffmpeg is able the encode the same file with same params, i.e. not reproducible on ffmpeg :(

[reimar]

First, triggering the assert is a bug in FFmpeg whether it is reproducible with it or not.
Second, as I said encoding the same file with FFmpeg does not at all mean that the encoder gets the same input, that's why I suggested trying to encode a file with random data with FFmpeg.

[reimar]

While I made mencoder print the correct error message, the encoder has several issues.
Basically it does not sufficiently validate its input, which can lead to further issues.
The problematic input seems to be large numbers and NaNs.
They can lead to multiple asserts being triggered and incorrect data being returned from the encoder, at the very least.

[reimar]

Forgot to say: fixed in r36268, patch to FFmpeg list sent as well (though that one is not a good patch, it just fixes the worst issues, like triggering an assert).

[sherpya@…]

Something is not clear for me:

now mencoder refuses to encode any kind o video I've tried saying unsupported input so I suppose it should always refuses to do such encoding.

The strange things is (I've modified the warn to show the format name too)

with acodec=wmav2:

Unsupported AVSampleFormat fltp (8)
Unsupported AVSampleFormat fltp (8)
Audio encoder requires unknown or unsupported input format

with acodec=libmp3lame

Unsupported AVSampleFormat s32p (7)
Unsupported AVSampleFormat fltp (8)
Unsupported AVSampleFormat s16p (6)
Unsupported AVSampleFormat s32p (7)
Audio encoder requires unknown or unsupported input format

with the same clip.

I can encode audio without problem if I use -oac mp3lame
and without problems using ffmpeg -vcodec wmav2

-af format=xxxx where xxxx is one of samplefmt_conversion_map in fmt-conversion.c seems to make nothing different, trying to force in the code FLTP as FLT produces weird audio but completes the encoding, perhaps it's just a random test because I have no idea of what is float planar

FLT uses floatne (be?), I tried le and be, le crashes while be produces wrong output

[sherpya@…]

looks like "input" refers to "codec" input not to format in the input file, mencoder is unable to provide a suitable sample format to the codec, this may be reasonable for wmav2 but strange with mp3lame, it wouldn't be nice to have a float planar converter?

[reimar]

"planar" means instead of having in memory
channel 0 sample 0, channel 1 sample 0, channel 0 sample 1, ...
you have
channel 0 sample 0, channel 0 sample 0, ...., channel 1 sample 0, ...
Support for converting that (in the other direction) is in ad_ffmpeg.c, but not in ae_lavc.c.
Your have of treating it like non-planar might work if you only try to encode mono.

[reimar]

I think passing planar data to FFmpeg encoders should now be working, thanks to fixes done for af_lavcac3enc.
It is only lightly tested though.