In Module 1, we learned that multimodal models can represent text and images in a shared embedding space. We also learned how contrastive learning encourages matching text-image pairs to have compatib
In our original Understanding LLM Embeddings from Scratch series, we learned how text becomes vectors, how those vectors learn semantic relationships, and how sentence embedding models represent entir
When I first learned self-attention, the formula looked straightforward: $$\operatorname{Attention}(Q,K,V)=\operatorname{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}+M\right)V$$ But one question bothered
Hey folks, this is the final module of our Understanding LLM Embeddings from Scratch series. Across the previous seven modules, we've learned how text becomes token IDs, how embeddings are learned, ho
From contextual token representations to a single vector for an entire sentence and why that vector is useful for retrieval. In Part 6, we learned how self-attention changes a token's representation d
How does the same token represent different meanings in different sentences? Let's work through self-attention from first principles, with actual numbers and PyTorch. In Part 5: Word2Vec, we saw how w
In Module 4, we treated embeddings as raw parameters—numbers in a matrix tuned by backpropagation to minimize a loss value. Now, we step back from the raw mechanics to look at the emergence of meaning
So far, we've learned why embeddings exist, how vectors, dimensions, and similarity work, how nn.Embedding stores vectors and retrieves them using token IDs, how padding and attention masks handle var
In Module 1, we learned why embeddings exist. In Module 2, we understood what vectors are, including their dimensions, magnitude, dot products, and cosine similarity. Now, let's connect everything to
