A very popular AI neural network that has been used for analyzing data presented as a series. Prior to the transformer method, recurrent neural networks (RNNs) were the primary architecture for speech, handwriting and language recognition.
Every word in speech and natural language recognition is highly dependent on the words before and after it. However, recurrent neural networks have difficulty remembering the relationships between words in long sentences and large documents, which is what the transformer architecture solved (see
AI transformer).
Contrast with
convolutional neural network, which is a widely used AI architecture for recognizing grid patterns such as images. See
neural network.