Similar Image Search for Histopathology: SMILY

by   Narayan Hegde, et al.

The increasing availability of large institutional and public histopathology image datasets is enabling the searching of these datasets for diagnosis, research, and education. Though these datasets typically have associated metadata such as diagnosis or clinical notes, even carefully curated datasets rarely contain annotations of the location of regions of interest on each image. Because pathology images are extremely large (up to 100,000 pixels in each dimension), further laborious visual search of each image may be needed to find the feature of interest. In this paper, we introduce a deep learning based reverse image search tool for histopathology images: Similar Medical Images Like Yours (SMILY). We assessed SMILY's ability to retrieve search results in two ways: using pathologist-provided annotations, and via prospective studies where pathologists evaluated the quality of SMILY search results. As a negative control in the second evaluation, pathologists were blinded to whether search results were retrieved by SMILY or randomly. In both types of assessments, SMILY was able to retrieve search results with similar histologic features, organ site, and prostate cancer Gleason grade compared with the original query. SMILY may be a useful general-purpose tool in the pathologist's arsenal, to improve the efficiency of searching large archives of histopathology images, without the need to develop and implement specific tools for each application.



There are no comments yet.


page 11

page 12

page 24

page 26


Combining Real-Valued and Binary Gabor-Radon Features for Classification and Search in Medical Imaging Archives

Content-based image retrieval (CBIR) of medical images in large datasets...

Fast and Scalable Image Search For Histology

The expanding adoption of digital pathology has enabled the curation of ...

Yottixel – An Image Search Engine for Large Archives of Histopathology Whole Slide Images

With the emergence of digital pathology, searching for similar images in...

Stacked Autoencoders for Medical Image Search

Medical images can be a valuable resource for reliable information to su...

Pan-Cancer Diagnostic Consensus Through Searching Archival Histopathology Images Using Artificial Intelligence

The emergence of digital pathology has opened new horizons for histopath...

One-Shot Item Search with Multimodal Data

In the task of near similar image search, features from Deep Neural Netw...

MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network

The inability to interpret the model prediction in semantically and visu...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.