Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays

MICCAI 2026
Jeongin Kim1, Sohyun Ahn2, Seo Young Kang3, Jaeyi Sung1, Soomin Kim1, Sungho Cho4, Rena Lee4, Kwanchang Kim5†, Junhyug Noh1†
1Division of AI & Software, Ewha Womans University   2Ewha Medical AI Research Institute   3Dept. of Nuclear Medicine, Ewha Womans University   4REMEDI Inc. R&D Center   5Ewha Womans University Seoul Hospital
Paper Code
Class-wise predictions of ASH-MIL with evidence maps on NIH ChestX-ray8

We present ASH-MIL,
Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays.

Abstract

Weakly-supervised thoracic disease detection in chest X-rays (CXR) is challenging due to subtle appearances and complex anatomical overlap, motivating anatomy-aware modeling for improved localization. However, prior anatomy-aware methods typically rely on coarse region proxies or static spatial priors, which may restrict dynamic instance discovery and limit precise localization of small abnormalities.

We propose Anatomy-Structured Hierarchical Multiple Instance Learning (ASH-MIL), a framework that introduces parallel anatomy-structured observation branches (cardiac, pulmonary, and agnostic) combined with hierarchical MIL aggregation. Anatomical priors are injected as soft spatial biases into decoder cross-attention, enabling anatomically grounded evidence maps without disease bounding-box supervision. Instance localization is derived directly from MIL-weighted cross-attention maps without bounding box supervision. Experiments on CXR8 and cross-domain MIMIC-CXR held-out sets demonstrate consistent improvements over prior weakly-supervised and anatomy-aware approaches, particularly under stricter localization criteria.

Method

Model Architecture

Overall architecture of ASH-MIL

Overall architecture of ASH-MIL. Three parallel branches (cardiac, pulmonary, and anatomy-agnostic) extract complementary evidence from shared image features. Hierarchical MIL combines this evidence across queries and branches for each disease. The pulmonary branch is shown in detail.

ASH-MIL learns where to look for disease evidence and how to combine it, bringing anatomical structure to lesion localization with image-level disease labels.

Anatomy-guided feature encoding. Overlapping structures in chest X-rays can make disease evidence difficult to distinguish. ASH-MIL uses patient-specific organ priors as soft cross-attention biases, guiding learnable queries toward relevant anatomical regions while allowing them to attend beyond organ boundaries. Cardiac and pulmonary branches capture evidence in distinct anatomical contexts, while an anatomy-agnostic branch explores the image without an organ-specific bias.

Hierarchical evidence aggregation. Our hierarchical MIL framework learns which queries and anatomical branches provide useful evidence for each disease.

  • Query-Level Aggregation (QLA): Learns each query’s relevance to a disease by projecting its features onto a learnable class-specific vector. The resulting attention weights combine query features into a class-specific representation within each branch.
  • Branch-Level Aggregation (BLA): Uses these representations to predict disease presence and learn the relative importance of each branch. The learned attention weights determine how much each branch’s prediction contributes to the final disease probability.

By combining attention maps with the learned query and branch weights, ASH-MIL localizes suspected lesions and reveals the evidence supporting its predictions, without lesion bounding-box annotations during training.

Experiments

Quantitative Comparison

Detection performance on CXR8 and MIMIC-CXR

Detection performance and cross-domain generalization. ASH-MIL achieves the highest overall mAP on both CXR8 (in-domain) and MIMIC-CXR (cross-domain). Ablations highlight the complementary benefits of anatomical priors and multi-branch aggregation.

Anatomy-Aware Evidence Routing: Nodule

Qualitative results on CXR8

Learning which anatomical evidence matters. For this small nodule within the lung parenchyma, ASH-MIL assigns the greatest weight to the pulmonary branch (0.84), whose evidence map closely aligns with the lesion. Combining the branches with these learned weights produces a compact localization that closely matches the ground truth. Red box: ground truth. Green box: prediction.

BibTeX

@inproceedings{kim2026ashmil,
  title     = {Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays},
  author    = {Kim, Jeongin and Ahn, Sohyun and Kang, Seo Young and Sung, Jaeyi and Kim, Soomin and Cho, Sungho and Lee, Rena and Kim, Kwanchang and Noh, Junhyug},
  booktitle = {International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)},
  year      = {2026},
}