The Complete AI Architecture Deep Dive — From LLM to Autonomous Agent (48 min)

The longest cut of the AI Stack material: 48 minutes, layer by layer, from how text becomes tokens to how a coding agent decides its next step. The 15-minute video gives you the framework and the 22-minute podcast gives you the walkthrough. This one goes all the way through every layer in one sitting.

The one idea

An LLM can only output text. It cannot browse, call an API, or remember you. Everything that looks like action or memory is the program around it: the program reads the model’s text, does the work, and feeds the result back. LLM talks, program walks. Every layer below is that loop with more structure around it.

What each layer covers

• LLM (5:05). A next-token predictor. It does not know what France is. It predicts that “Paris” usually follows “the capital of France is”.
• Tokens (8:04). The tokenizer splits text by statistical patterns learned in training, not by words or grammar. That is why models miscount the R’s in “strawberry” (11:05): they never see letters, only numbered chunks.
• Context (13:11). The model keeps nothing between calls. The program resends the whole conversation every turn, so the context window works like a desk with a fixed size. A 1-million-token window holds about 750,000 English words, and filling it costs time and money on every reply (16:03).
• RAG (18:02). Instead of piling the whole manual on the desk, retrieve the three relevant pages. Most “hallucination fixes” are retrieval fixes: the model answers from what is on the desk.
• Prompts (21:05). Two kinds: the user prompt says what to do now, the system prompt (placed first, invisible to the user) says who the model is and what rules it follows.
• Function calling (24:07). The model outputs a structured JSON request (“call get_weather with city=Tokyo”). The program runs the tool and appends the result. The model then writes the answer from that result.
• MCP (30:03). A standard catalog: the program asks an MCP server what tools it has, and calls them in one shared format instead of per-vendor glue code.
• Agents (34:10). The function-calling loop repeated until the task is done. This is what Claude Code, Cursor and Copilot run under the hood (37:00).
• Skills (39:07). Markdown rulebooks. With hundreds of them, the program loads only the matching one when a task needs it (progressive disclosure, 41:05), so the desk stays small.

Two questions for any AI product

1. What text did the model output?
2. Which program read that text and turned it into an action?

If you can answer both, the product is no longer a black box (46:00).

Related

• 15-min original: The AI Stack Explained
• 22-min podcast: The AI Stack Explained, Extended
• Blog post with diagrams and code: LLM talks, program walks
• Runnable demo: function calling from scratch