Bonnie Li published a paper on Contrastive World Models. The method replaces pixel reconstruction with a contrastive objective. It maximizes mutual information between state-action sequences and future observation patches. The approach builds on Dreamer. It matches Dreamer in simple settings. It outperforms Dreamer when distractors or natural backgrounds are added. It removes the pixel decoder entirely. Training is more efficient. The method assumes access to state-action sequences and future observations. Results suggest robustness to visual nuisance factors.
What it unlocks
Building world models that ignore visually irrelevant details in complex environments.
Sources