Putting an End to End-to-End: Gradient-Isolated Learning of Representations

Sindy Lowe, Peter O’ Connor, Bastiaan S. Veeling, NIPS-2019


The paper proposes a novel self-supervised learning technique which rather than employing end-to-end training, focusses on employing isolated training of various modules of the network using greedy InfoNCE objective. This paper heavily inspires from biological phenomenons. This paper won Honorable Mention Outstanding New Directions Paper Award at NeurIPS 2019.

Main Contributions

Implementation Details

Our two cents