Paper Project Page

HVLFormer

Segmenting Visuals With Querying Words: Language Anchors For Semi-Supervised Image Segmentation

Numair Nadeem, Saeed Anwar, Muhammad Hamza Asad, Abdul Bais

Paper Code Checkpoints BibTeX

HVLFormer

HVLFormer is a vision-language segmentation framework that anchors mask prediction with dataset-aware textual queries. It extends TQDM with hierarchical textual query generation, pixel-text refinement, and cross-view modal consistency for semi-supervised learning.

3 textual queries per class 6 pixel-decoder layers 9 masked-attention layers 8 + 8 labeled and unlabeled images

Method

HVLFormer architecture with hierarchical textual query generation, semantic relevance estimation, pixel-text refinement, transformer decoding, and consistency regularization.
HVLFormer generates dataset-aware hierarchical textual queries, refines them with pixel features, and regularizes predictions across augmented views.

Hierarchical Textual Query Generation

Dataset-specific prompts produce attribute-rich class descriptions. Three MLP heads generate scale-aligned queries for coarse structure, mid-level parts, and fine texture or boundary cues. Semantic relevance estimation suppresses absent-class queries.

Pixel-Text Refinement

PTRM injects image context into textual queries and transfers class-level semantics back into pixel features using spatial gates over text-guided, pixel-guided, and fused attention maps.

Cross-View Modal Consistency

CMCR regularizes unlabeled examples across original, weakly augmented, and strongly augmented views, aligning masks, class predictions, and pixel-text correspondences at decoder stages.

Qualitative Results

Qualitative segmentation comparison showing input image, ground truth, previous SOTA, and HVLFormer prediction.
HVLFormer improves class-consistent masks compared with the previous SOTA, especially when language anchors separate visually similar object regions.

Experiments

Dataset Crop Epochs Metric
Pascal VOC512 x 51280mIoU
COCO512 x 51210mIoU
ADE20K512 x 51240mIoU
Cityscapes801 x 801240mIoU

BibTeX

@inproceedings{nadeem2026segmenting,
  title={Segmenting Visuals With Querying Words: Language Anchors For Semi-Supervised Image Segmentation},
  author={Nadeem, Numair and Anwar, Saeed and Asad, Muhammad Hamza and Bais, Abdul},
  booktitle={European Conference on Computer Vision},
  year={2026}
}