After a lot of tests I found of, that the number of parameters of A4J.AJAX.Submit() has changed.

So after updating the curresponding part at dialog.js it also works with RichFaces 3.3.3.CR1.
Also...