Also read section 7.10.7. "Encoding options of libavcodec of the MPlayer doc", the file encoding-tips.txt and the links it points to. Plus I've seen some more tests done at http://www.ee.oulu.fi/~tuukkat/mplayer/tests/readme.html.
vqmin
Always use 2.
vqscale
I read the recommendation to use 2 in the first pass and then set bitrate in pass 2. It didn't work on any of my tests.
vqmax
Leave it alone. 12 should be ok if you encode at high quality. I read you should set it to 6 if you want to use it with qprd, but the bit rate has to be high enough...
vmax_b_frames
Should be set to 1, more isn't giving good results. Use it together with bidir_refine=2 and vb_strategy=1 (pass 1). There is one case when it shouldn't be used: anime with smoke or other translucent effect.
mbd
Always 2.
v4mv
Always use it.
vpass
Always encode in 2 passes.
vbitrate
900 is a good start.
vqblur
-
vqcomp
Leave it to 0.5, up to 0.6 with high bit rate.
vlelim
Don't use it.
vcelim
Don't use it.
lumi_mask
Don't use it.
dark_mask
Don't use it.
scplx_mask
Don't use it.
mbcmp
Only used if mbd=0.
precmp, cmp, subcmp
Experiment with values of 0 (default), 2 (Hadamard), 3 (dct), and 6 (rate distortion). Try adding 256 to take chroma into account: gives good results but there might be problems with B-frames.
- Without B-frames: precmp=259:cmp=258:subcmp=262
- With B-frames: precmp=3:cmp=2:subcmp=3
predia
Requires preme=2 to really be useful. Use predia=3, or predia=-1 to be faster.
dia
Use dia=4, or dia=3 to be faster.
trell
Always use it.
cbp
small speed cost for small quality gain.
mv0
small speed cost for small quality gain.
qprd
Only test with high bitrates.
last_pred
Use 3, up to 4 for better quality but not more.
preme
Always use 2.
qns
qns=1 can be useful at high bitrate.
vqdiff
Use the default 3.
qpel
Warning: some DIVX players (like KISS DP500) do not support Qpel decoding.