A MetaTransfer Objective for Learning to Disentangle Causal Mechanisms
Abstract
We propose to metalearn causal structures based on how fast a learner adapts to new distributions arising from sparse distributional changes, e.g. due to interventions, actions of agents and other sources of nonstationarities. We show that under this assumption, the correct causal structural choices lead to faster adaptation to modified distributions because the changes are concentrated in one or just a few mechanisms when the learned knowledge is modularized appropriately. This leads to sparse expected gradients and a lower effective number of degrees of freedom needing to be relearned while adapting to the change. It motivates using the speed of adaptation to a modified distribution as a metalearning objective. We demonstrate how this can be used to determine the causeeffect relationship between two observed variables. The distributional changes do not need to correspond to standard interventions (clamping a variable), and the learner has no direct knowledge of these interventions. We show that causal structures can be parameterized via continuous variables and learned endtoend. We then explore how these ideas could be used to also learn an encoder that would map lowlevel observed variables to unobserved causal variables leading to faster adaptation outofdistribution, learning a representation space where one can satisfy the assumptions of independent mechanisms and of small and sparse changes in these mechanisms due to actions and nonstationarities.
 Publication:

arXiv eprints
 Pub Date:
 January 2019
 arXiv:
 arXiv:1901.10912
 Bibcode:
 2019arXiv190110912B
 Keywords:

 Computer Science  Machine Learning;
 Statistics  Machine Learning