Did you run any experiments in XNLI? Also curious how it compares to XLM. Also, shameless plug for the cross-lingual QA dataset we just released, MLQA
https://github.com/facebookresearch/MLQA - could be a great testbed for models like this

❇ @AI_Python_EN