ConvNN-Attention
Convolutional Nearest Neighbor Attention (ConvNN-Attention) for Transformers
TL;DR
Convolutional Nearest Neighbor Attention (ConvNN-Attention) is a novel attention mechanism that combines convolutional operations with nearest neighbor search to enhance the performance of Transformer models in both vision and language tasks.
Paper
The paper on Attention Via Convolutional Nearest Neighbors is available on arXiv: here.