AFINITI: attention-aware feature integration for nuclei instance segmentation and type identification

Accurately identifying and analyzing nuclei is pivotal for both the diagnosis and examination of cancer. However, the complexity of this task arises due to the presence of overlapping and cluttered nuclei with blurred boundaries, variations in nuclei sizes and shapes, and an imbalance in the available datasets. Although current methods utilize region proposal techniques and feature encoding frameworks, but they often fail to precisely identify occluded nuclei instances. We propose a model named AFINITI, which is both simple, efficient, achieves high accuracy, recognizes instance boundaries cluttered and overlapping nuclei, and addresses class imbalance issues. Our approach utilizes nuclei pixel positional information and a novel loss function to yield accurate class information for each nuclei. Our network features a lightweight, attention-aware feature fusion architecture with separate instance probability, shape radial estimator, and classification heads. We use a compound classification loss function to assign a weighted loss to each class according to its occurrence frequency, thereby addressing the class imbalance issues. The AFINITI model outperforms current leading networks across eight major publicly available nuclei segmentation datasets achieving up to an 8% increase in Dice Similarity Coefficient (DSc) and a 17% increase in Panoptic Quality (PQ) compared to existing techniques demonstrating its effectiveness and potential for clinical applications. The source code and the weights of the trained model have been released to the public and can be accessed at: https://github.com/Vision-At-SEECS/AF-Net. © The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2024.