The article “Geometry Meets Attention: Interpretable Transformers via SVD Inspiration” was published in IEEE Access in 2025.
The paper introduces an interpretable transformer attention mechanism inspired by singular value decomposition, opening a line of work where attention directions and learned spectral structure can be inspected and reasoned about rather than treated as opaque internal weights.
- Authors: Vasileios Arampatzakis, George Pavlidis, Nikolaos Mitianoudis, Nikos Papamarkos
- Journal: IEEE Access, 13, 119077-119089
- Publication year: 2025
- DOI: 10.1109/ACCESS.2025.3586739
You may also like
-
TMLR 2026: prescriptive SVD-inspired attention via spectral energy retention
-
Towards Heritage World Models: from digital twins to predictive heritage systems
-
ICASSP 2026: representation-diverse self-supervision for bioacoustic learning
-
AISTATS 2026: process-tensor tomography of SGD and training memory
-
ICPRAM 2026: two-stage angular alignment for positive-unlabeled learning