One-sentence definition
Neural architecture based primarily on attention that processes relationships between elements of a sequence or multiple modalities.
How to interpret this concept
A transformer is a neural network architecture that uses attention to relate elements of a sequence. Attention allows combining information from different positions; position representations help preserve order. The original paper introduced it for machine translation, and subsequently multiple variants and uses appeared.
Architecture and product are not the same. Two transformer-based models may differ in data, training, size, modalities, limits, and behavior. Nor do all artificial intelligence systems use this architecture. When choosing a model, it matters to measure the specific task, not assume capabilities based on the transformer label.
Practical example: in a sentence with several people, attention can help relate a pronoun to previous information. This does not demonstrate human understanding nor guarantee that a generated explanation faithfully describes the internal calculation. In document-based applications, references, negations, and long dependencies must be evaluated with examples that have verifiable answers.
