Jigsaw Regularization in Whole-Slide Image Classification

So Won Jeong, Veronika Ročková

Computational pathology involves the digitization of stained tissues into whole-slide images (WSIs) that contain billions of pixels arranged as contiguous patches. Statistical analysis of WSIs largely focuses on classification via multiple instance learning (MIL), in which slide-level labels are inferred from unlabeled patches. Most MIL methods treat patches as exchangeable, overlooking the rich spatial and topological structure that underlies tissue images. This work builds on recent graph-based methods that aim to incorporate spatial awareness into MIL. Our approach is new in two regards: (1) we deploy vision \emph{foundation-model embeddings} to incorporate local spatial structure within each patch, and (2) achieve across-patch spatial awareness using graph neural networks together with a novel {\em jigsaw regularization}. We find that a combination of these two features markedly improves classification over state-of-the-art attention-based MIL approaches on benchmark datasets in breast, head-and-neck, and colon cancer.

• Google
• Berggruen Institute
• Santa Fe Institute
• University of California San Diego
• University of Chicago
• Linköping University
• Mohamed bin Zayed University of AI
• University of Chicago
• University of Wisconsin-Madison
• Carnegie Mellon University
• Harvard Belfer Center
• Stanford University
• The University of Texas at Austin
• University of Chicago
• University of Minnesota
• Northwestern Universit
• Stanford University
• The University of Sydney
• University of Chicago
• Shanghai AI Laboratory
• Shanghai Jiao Tong University
• Sichuan University
• The Hong Kong University of Science and Technology (Guangzhou)
• Tsinghua University
• University of Chicago
• Google DeepMind
• Max Planck Institute for Human Development
• Mila–Quebec AI Institute
• Santa Fe Institute
• University of Chicago
• University of Toronto