City Research Online

Abstract

This article introduces convolutional shared vision transformers (ConvShareViT), a novel deep learning architecture that adapts the vision transformer (ViT) architecture to the 4f free-space optical system. ConvShareViT replaces linear layers in multi-head self-attention (MHSA) and multilayer perceptrons (MLPs) with a depthwise convolutional layer with shared weights across input channels. The effectiveness of the attention mechanism was analyzed systematically in 12 experiments with different Models. Experimental results demonstrated that configurations with valid-padded shared convolutions successfully learned attention, achieving comparable quantitative attention scores to those obtained with standard ViTs. However, same-padded convolutions showed limitations in attention learning and operated like regular convolutional neural networks (CNNs) rather than transformer models. In terms of speed, ConvShareViT can theoretically achieve up to 3.04 times faster inference than GPU-based systems. This potential acceleration makes ConvShareViT an attractive candidate for future optical deep learning applications.

Publication Type:	Article
Additional Information:	© 2026 IEEE. This accepted manuscript is made available under the terms of the Creative Commons Attribution License (CC-BY), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited.
Publisher Keywords:	Modeling, Convolution, Vision transformers, Kernel, Transformers, Training, Tiles, Architecture, Computer architecture, Matrices
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments:	School of Science & Technology School of Science & Technology > Department of Computer Science
SWORD Depositor:	Symplectic Administrator

Export

Downloads