City Research Online

Abstract

Recent adapters for vision-language models (VLMs) still suffer from dense cross-modal interactions and limited control over alignment capacity. To address this issue, we propose a two-stage Shallow-Deep Squeezing Adapter (SDSA) that explicitly regulates cross-modal interaction density and alignment capacity. In the shallow squeezing stage, SDSA leverages token-level masking to impose structured sparsity within a sparse bottleneck space. Then, in the deep squeezing stage, SDSA applies a shared low-rank transformation and a cross-modal attention module: the low-rank module consolidates alignment into a compact set of dominant shared directions, while cross-modal attention refines representations through selective interaction. Extensive experiments on 11 datasets show that SDSA delivers superior base-to-novel generalization and cross-dataset evaluation, and significantly improves the generalizability of VLMs under few-shot conditions. The code can be found at https://github.com/haowang-ac/SDSA.

Publication Type:	Conference or Workshop Item (Paper)
Additional Information:	© 2026 Springer.
Publisher Keywords:	Artificial Intelligence & Image Processing, 46 Information and computing sciences
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments:	Bayes Business School Bayes Business School > Faculty of Actuarial Science & Insurance
SWORD Depositor:	Symplectic Administrator

Export

Downloads