How can a sequence model connect distant tokens?

Our workspace follows the shift from recurrence to attention.

• Read the literature review.
• Open the paper.

Attention weights the available context

\operatorname{Attention}(Q,K,V) = \operatorname{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V

The weights combine values for each query, so every token can draw on distant context in one step.

Keep the talk beside its sources

Write the talk, review and paper in the same repository.

Review changes together, then share the rendered talk as a single HTML file.

Review local changes with asta workspace what-changed; PR previews use the same comparison.

Read the original paper

The transformer was introduced in Attention Is All You Need.

A figure carried inside the slide

Queries, keys and values

The figure is an inline data: image, so the deck needs no separate file for it.