20-07-13 Deep Ideas in Class Discovery

Category: Idea Lists (Upon Request)

Read the original document

<!-- gdoc-inlined -->

1. Transfer objective 2. David’s layer type (anisotropic) 3. Training in sim 4. Recursive self-improvement

1. Training leads to improved class creation
2. Modifying the ontology dynamically
   1. Class merging, splitting, creation
3. Learnability as the prior
4. Start with contrastive learning (to get a good clusterable representation)
5. One-vs-all / Object Detection / Escape the one class per image assumption
6. Internet scale training
   1. Discover everything for incredible transfer
7. Transfer because you’ve already seen and classified similar data
8. Curriculum as you transfer to similar classes

Mathematical possibilities:

• David’s Anisotropic Layer
• Semi-supervised clustering
• Geometry of metric space

Obstacles and Limitations:

• Deep metric clustering outperformed by raw image space clustering
• Computational expense of transfer evaluations
• Noisy label training

Source: Original Google Doc