Project Summary Adoption of digital histopathology has increased demands for gigapixel image analysis tools and methods for clinical applications. Integration of deep learning algorithms for automated tumor segmentation and cancer diagnosis from Hematoxylin and Eosin (H&E) dye or virtual (stainless) staining of tissue biopsy images have been reported. There is a significant lack of explanation and performance evaluation (black box phenomenon) of tumor segmentation and stainless staining deep learning models limiting clinician adoption in oncology practice. Results from these deep learning systems are known after the biopsy is irreversibly processed by H&E staining, and do not include confirmatory immunohistochemistry (IHC) biomarker staining diagnosis performed on adjacent tissue sections. We have previously described deep neural network systems to convert native nonstained whole slide prostate tissue biopsy images (WSI) to virtual computationally H&E stained versions validated for tumor segmentation with high precision. The goal of this proposal is developing novel methods and algorithms for pixel-by-pixel explanations to clinicians and cancer researchers for explainability of virtual H&E staining augmented with prostate tumor grade segmentation and IHC expression patterns by deep learning models for digital biopsies. Our previously published and physician authenticated Generative Adversarial Neural Network (GAN-CS) models trained with 93,199 image pairs of prostate biopsy images for virtual H&E staining and prostate tumor segmentations, and a publicly available database of clinically validated IHC analysis of prostate biopsy images will be used to generate the explainability software. A histological map that visualizes and interprets correspondence between neural representations in GAN-CS model with virtually stained prostate Gleason grade tumors in WSI at a pixel level will be generated. A separate explainable GAN- CSS model for segmentation and Gleason grading of tumors using clinician annotations from H&E and conjunctive IHC image biomarker labels will also be generated. Researchers can upload WSI into GAN-CS/S software to perform computational H&E staining multiplexed with morphological tumor segmentations and IHC expression with pixel-by-pixel visualization and explanation to characterize cellular phenotypes for cancer research and accelerate histopathology diagnoses. The open source software toolkit developed from this research can be generalized to majority of deep learning model architectures using disease labels from widely available non-stained, chemical, or virtual H&E and IHC stained images from different cancer types.

Project Narrative Stainless staining of tissue biopsy images and subsequent tumor segmentation by deep learning models is a rapidly emerging technology for cancer research and histopathology diagnoses. We will develop new methods and algorithms to generate knowledge-driven visualization and pixel-by-pixel explanations from clinically validated deep learning models for virtual and chemical H&E and IHC staining to characterize cellular phenotypes for cancer research and accelerate histopathology diagnoses.