Example Concept: Transformers use self-attention to compute a weighted representation of the entire input sequence, allowing each token to attend to every other token. This mechanism helps maintain context over long distances by assigning higher weights to relevant tokens, thus preserving important information. Additionally, positional encoding is added to input embeddings to provide the model with information about the order of tokens, which is crucial for understanding sequences.