A question from AI702 class. Writing in progress, and includes "tranformer" generated content. I think I'll think about this question more deeply, and add more content.

Let’s look at the intuitive visualization) (LLM Visualization — Bbycroft.net, n.d.) .

An interactive visualization of an LLM.

The most important unit of the transformer

What is the most important unit of the transformer?

From a KAIST class

The self-attention mechanism is the most important unit of the transformer (Vaswani et al., 2017) AI model because it allows the model to weigh the importance of different parts of the input sequence in relation to each other, enabling it to understand long-range dependencies and context globaly. This mechanism, especially in its multi-head attention form, is what enables transformers to outperform previous models like RNNs and CNNs

How the Self-Attention Mechanism Works

1. Input Representation: Each word (or token) in the input sequence is converted into three vectors: a Query (Q), a Key (K), and a Value (V) vector.
2. Attention Scores: The model calculates an attention score by taking the dot product of the Query vector for a given word and the Key vector for every other word in the sequence. This score determines how relevant each word is to the word being focused on.
3. Weighted Values: These scores are scaled and then passed through a softmax function to get attention weights, which are used to create a weighted sum of the Value vectors.
4. Contextual Output: The final output for each word is a vector that represents its meaning in the context of the entire sequence, effectively capturing the relationships between all words. 

Why It’s Crucial

• Global Context: Unlike RNNs that process data sequentially, self-attention allows the transformer to look at all words at once, providing global context. 
• Long-Range Dependencies: It effectively captures relationships between words that are far apart in a sentence, which is crucial for understanding complex language. 
• Parallelization: The attention mechanism can be computed in parallel, allowing for greater computational efficiency and scalability.

What are other important components of the Transformer besides self-attention?

Besides the self-attention mechanism, the following are other crucial components of the Transformer architecture

: 

• Positional encoding: Since the self-attention mechanism processes all words simultaneously, it loses the sequential order of the input. Positional encoding adds information about the relative or absolute position of each token in the sequence. Without this, the model could not distinguish between “The dog bites the man” and “The man bites the dog”.
• Feed-forward networks: Each self-attention layer is followed by a position-wise, fully connected feed-forward network (FFN). This FFN applies a non-linear transformation to the output of the attention layer, allowing the model to learn more complex patterns and relationships in the data.
• Encoder-decoder architecture: The original Transformer model used a two-part structure. The encoder processes the input sequence and creates a contextual representation. The decoder then uses that representation, along with previously generated output, to produce the final output sequence.
• Residual connections and layer normalization: These components work together to stabilize and accelerate the training process.
  • Residual connections, also known as skip connections, add the input of a layer to its output. This helps combat the vanishing gradient problem, especially in very deep networks.
  • Layer Normalization normalizes the output of each sub-layer. It helps keep the activation values in a stable range, which prevents training from becoming unstable.
• Word embeddings: These high-dimensional vector representations convert the input text tokens (words or sub-words) into a numerical format that the model can understand and process. The model learns to adjust these embeddings during training.
• Cross-attention (in the decoder): In the original encoder-decoder model, the decoder uses a cross-attention layer to “look” at the encoder’s output. This allows the decoder to focus on relevant parts of the input sequence while generating the output. 

While the self-attention mechanism is the most innovative and defining feature, the combination of all these elements is what makes the Transformer architecture so powerful

Visualization

After watching the video, please propose a way to understand or visualize the decision/generation boundary of a transformer-based LLM model.

Prof. Choi

IPAM 2023 Towards Novel Insight Workshop: “Explainable AI to Analyze Internal Decision Mechanism of Deep Neural Networks”

I really like the EG-BAS paper (Jeon et al., 2020) .

Rapidly-exploring random trees, from 1998.

Of course, RRT (LaValle, 1998) is familiar for all robotics people. I think I can come up with a similar style for LLM models.

I really like this visualization from Anthropic (Mapping the Mind of a Large Language Model — Anthropic.com, n.d.)

Anthropic maps the mind

To be continued

Thinking deeply about each components should give more insights and research ideas. To be continued…

References

1. LLM Visualization — bbycroft.net.
2. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need. CoRR, abs/1706.03762. http://arxiv.org/abs/1706.03762
3. Jeon, G., Jeong, H., & Choi, J. (2020). An efficient explorative sampling considering the generative boundaries of deep generative neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 4288–4295.
4. LaValle, S. M. (1998). Rapidly-exploring random trees : a new tool for path planning. The Annual Research Report. https://api.semanticscholar.org/CorpusID:14744621
5. Mapping the Mind of a Large Language Model — anthropic.com.