Enhanced ENCODER for Composed Thangka Image Retrieval via Domain Adaptation and Hierarchical Text Alignment

Abstrak

Composed image retrieval (CIR) for Thangka imagery presents a challenging domain gap: queries combine a sketch with hierarchical natural language descriptions, yet general-purpose vision–language models struggle with the dense iconographic structures of this culturally specific domain. This paper proposes four parameter-efficient modifications to ENCODER, the strongest supervised baseline on the CIRThan benchmark: Cross-Domain Projection (CDP) residual adapters, Level-Aware Tags (LAT), Hard Negative Mining (HNM), and UCDR Domain Prompts. All modifications are integrated via component-level code transplantation and fine-tuned for 20 epochs on a Kaggle Tesla T4 GPU. Our re-run of ENCODER achieves 52.91% average R@1. UCDR domain prompts alone contribute the largest single gain (+1.43 pp), more than the other three modifications combined (+0.31 pp), confirming that visual domain adaptation is the primary bottleneck. The full model (UCDR+CDP+LAT+HNM) achieves 55.61% average R@1 (+2.70 pp over baseline), adding only 2.23M trainable parameters to the 151M frozen CLIP backbone. A full component-wise ablation across 16 variants shows that all top 5 combinations include UCDR domain prompts. Multi-seed evaluation across three random seeds further shows that ENCODER+CDP+LAT+HNM achieves the best single-seed result of 58.95% average R@1 (seed 123), while the full model maintains more consistent performance across seeds (53.14%–56.09%). The L3–L1 spread narrows from 4.06 pp (ENCODER) to 1.67 pp (full model), indicating improved handling of coarse-level queries.

Identifier

• DOI: https://doi.org/10.67561/jaids.v2i1.4
• OpenAlex: https://openalex.org/W7169887530