Abstract 15- 1300-1315
Category: Basic Science

At the end of the session,
participants will be able to:

  • Learn about an automated workflow that can integrate morphologic and molecular data for tumour classification.
  • Learn about computer vision tools that can efficiently analyze whole slide images of immunohistochemistry stains.

Dr. Michael Lee


Michael K. Lee1, Kevin Faust2,3, Anglin Dent1, Clare Fiala3, Alessia Portante1, Madhu Rabindranath1, Noor Alsafwani3,4, Andrew Gao1,3, Ugljesa Djuric5, Phedias Diamandis1,3,5,6

1Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada

2Department of Computer Science, University of Toronto, Toronto, ON, Canada

3Laboratory Medicine Program, Department of Pathology, University Health Network, Toronto, ON, Canada

4Department of Pathology, College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia

5Princess Margaret Cancer Centre, Toronto, ON, Canada

6Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada

Target Audience:


Integrating morphologic and molecular histopathological features through whole slide image registration and deep learning


Background and Objective: Despite recent innovations in deep learning1–3, integrating histomorphologic and molecular information found on respective H&E and IHC-stained tissue sections still remains a challenge. While human observers can easily align these different tissue sections, routine computational approaches for image registration of giga-bite sized WSIs are still needed. Here, we aim to address this issue by incorporating another computer vision tool, scale-invariant feature transform (SIFT), to align H&E-stained sections with accompanying IHC studies for automated subclassification of gliomas.

Method: To test the workflow, we first trained a VGG19 convolutional neural network (CNN) using pathologist-annotated H&E WSIs to recognize histological patterns of 16 common tissue and lesion classes. Afterwards, we optimized a different set of CNNs to recognize various IHC markers, such as IDH1-R132H and ATRX, which are relevant for molecular subclassification of gliomas4–5. For the integrated analysis, we employed SIFT to find features for image matching which were used to align lesional regions of H&E and IHC slides.

Results: The histomorphologic classifier (CNNH&E) excelled at classification with accuracies of 100% for glioma, meningioma and metastatic carcinoma, and 93% for schwannoma (n = 125). Using the newly developed CNNH&E and SIFT-based alignment, the quantitative analysis by the IHC classifiers significantly improved for ATRX retained and IDH1-R132H positive WSIs when compared to that of unaligned WSIs.

Conclusion: SIFT can work in concert with deep learning tools to provide a pathologist-inspired workflow to help automate advanced immunohistochemically diagnostic tasks, such as subclassification of glioma.