Multimodal Modular Meta-Learning

Many real world prediction problems involve structured tasks across multiplemodalities. We propose to extend previous work in modular meta learning to themultimodal setting. Specifically, we present an algorithmic approach to apply taskaware modulation to a modular meta learning system that decomposes structuredmultimodal problems into a set of modules that can be reassembled to learn newtasks. We also propose a series of experiments to compare this approach withstate of the art modular and multimodal meta learning approaches on multimodalfunction prediction and image classification tasks.


Full Preprint found here.