How Large Language Models Work

The instructor is ready

How Large Language Models Work

Interactive digital-human course

How Large Language Models Work

A beginner-friendly overview of how large language models learn, reason, and generate text.

My workspace32 minFree to watch

What you’ll learn

  1. 01How Large Language Models WorkWelcome. I'm really glad you're here. Today, we're going to demystify something that's probably already part of your daily life: large language models. You know, the technology behind the chatbots and assistants that help us write, brainstorm, and find information. Our goal is to move past the magic and mystery, and give you a clear, practical understanding of how these models actually work. This knowledge is becoming essential for making smart decisions about the tools we use every day, from fact-checking a summary to understanding why an AI might get something wrong. Over the next 16 short modules, we'll build this picture step by step. We'll start with the very basics and gradually connect them to the fluent conversations you see on the surface. Think of it as a guided tour, where I'm here to walk you through it in plain, simple language, without any assumed expertise. Let's get started, beginning with the foundational idea behind it all: from text prediction to fluent conversation.How Large Language Models Work2 min
  2. 02From Text Prediction to Fluent ConversationThink about the autocomplete on your phone. You type a few words, and it suggests the next one. That’s the core principle behind a large language model, but scaled up to an almost unimaginable degree. It’s a next-token predictor, choosing the most likely word, or token, to follow what came before. This simple mechanism, when trained on trillions of tokens from across the internet, transforms into something that feels like fluent conversation. It’s important to understand that this is different from a search engine, which retrieves existing documents. And it’s not a database that returns exact matches. Instead, an LLM generates a new, probabilistic response each time. It’s choosing the most probable next step in a conversation, not just pulling out a stored fact. This ability to generate rather than retrieve is what makes the interaction feel so dynamic. Let’s take this idea further and look at how words actually become these numbers the model can work with, in 'Tokens and Embeddings: How Words Become Numbers.'From Text Prediction to Fluent Conversation1 min
  3. 03Tokens and Embeddings: How Words Become NumbersNow we get to the fascinating part where words become numbers. This is the secret sauce that lets a language model actually work with language. The first step is tokenization. Think of a token as a chunk of text the model sees. Interestingly, these chunks are often smaller than full words, maybe parts of words or common letter combinations. This is why the model can sometimes stumble over a rare or misspelled word. It has to break it into unfamiliar, smaller pieces, which makes it harder to understand the meaning. Some of the odd glitches you might see, like a model struggling to count letters in a word or being unexpectedly sensitive to capital letters, often trace right back to this tokenization step. Once we have our tokens, the next magic trick is turning them into embeddings. Imagine a giant, multi-dimensional map. An embedding is the coordinate where a token lives in this map. The key idea is simple and powerful: tokens with similar meanings are placed close together. So the word 'happy' isn't just a definition; it's a point in space right next to 'joyful' and farther away from 'sad'. This geometric representation of meaning is the foundation for everything the model does. Next, we'll step inside the engine that processes these embeddings, the Transformer architecture.Tokens and Embeddings: How Words Become Numbers2 min
  4. 04Inside the Transformer ArchitectureNow, let's step inside the engine itself—the Transformer architecture. When we talk about today's large language models, we're almost always talking about what's called a decoder-only design. Think of it as a supercharged text predictor, built to focus purely on generating the next word. The real magic inside is something called self-attention. It's a mechanism that allows every single word, or token, in your input to check in with every other token, all at once, to understand the full context. This is what helps the model grasp that 'bank' means a river, not a financial institution, based on the surrounding words. To make this work on a massive scale, we use parallel training across thousands of specialized computers, enabling models with hundreds of billions of parameters. You might also hear about the original encoder-decoder design, which is a bit different. It separates the task of understanding what you said from the task of generating a response, but for pure text generation, the decoder-only approach has become the dominant force. Next, we'll see how all this architecture gets shaped into a useful assistant when we explore the training pipeline, covering data, pretraining, and alignment.Inside the Transformer Architecture2 min
  5. 05The Training Pipeline: Data, Pretraining, and AlignmentNow let's pull back the curtain and look at how these models are actually built. We call it the training pipeline, and it moves through three major stages. First, the model is exposed to a massive, curated collection of text from the open web, books, code, scientific papers, and dialogue. The objective is simple but powerful: predict the next word, trillions of times, building a deep model of the world. But that raw model is a completion engine, not an assistant. So the second stage is instruction tuning, which teaches it to follow directions. Finally, we align the model with human preferences using techniques like reinforcement learning from human feedback, or R L H F. This alignment pipeline adds safety guardrails, reduces harmful outputs, and generally makes the model more helpful and reliable. Let's go deeper into the first step of that pipeline: data quality, including filtering, deduplication, and decontamination.The Training Pipeline: Data, Pretraining, and Alignmentalicelabs.aihuggingface.codev.to+21 min
  6. 06Data Quality: Filtering, Deduplication, and DecontaminationNow let’s talk about a part of the pipeline that doesn’t sound glamorous, but makes everything else possible: data quality. Raw text from the web is mostly boilerplate, spam, and noise. It is not training data. So the first job is filtering. We use multi-stage filters. Fast heuristics catch the obvious junk, like pages with very little text or too many repeated words. Then lightweight classifiers catch the low-quality content that slips through. Next comes deduplication. We remove exact copies by hashing, then find near-duplicates across the whole corpus. If you only deduplicate per source, you miss the same article syndicated across dozens of domains. After that, decontamination is mandatory. Benchmarks are published on the web, so crawls will ingest them. If your training data contains those benchmarks, your evaluation scores become meaningless. Finally, we redact personally identifiable information and track provenance. This is now a compliance requirement, not a nice-to-have. Provenance documentation travels with the data so downstream users know exactly what they’re training on. Up next, we’ll explore what these models can actually do once the data is ready. We’ll look at emergent abilities and what LLMs can do.Data Quality: Filtering, Deduplication, and Decontaminationdotdatalabs.aiaethyn.iotoloka.ai+22 min
  7. 07Emergent Abilities: What LLMs Can DoNow, let's talk about one of the most fascinating aspects of these models: emergent abilities. These are skills that appear without being explicitly trained. For example, a model can learn a new task from just a few examples you give it in the prompt, something we call few-shot reasoning. With chain-of-thought prompting, we can unlock step-by-step logical problem-solving, allowing the model to handle complex math or code generation. These skills, from creative writing to reasoning, emerge once models cross certain thresholds of scale and data. For the enterprise, this is game-changing. It means a single, flexible model can be deployed for many different tasks. But this flexibility comes with a flip side. As we scale these models, we also see the emergence of a problem called hallucination. Let's explore what that means and why it happens.Emergent Abilities: What LLMs Can Dosciencedirect.comarxiv.orgaclanthology.org+21 min
  8. 08Hallucination: Why LLMs Make Things UpNow let's talk about one of the most puzzling aspects of large language models: hallucination. Why do they sometimes just make things up? The surprising finding from recent research is that hallucination isn't really a knowledge gap. The model often does detect uncertainty internally. It knows the input is questionable. The problem is that this internal alarm signal doesn't get properly connected to the output layer. Think of it like a fire alarm ringing in a sealed room, the warning is there, but the rest of the building never hears it. Instead of forming a single, clear 'I don't know' state, that uncertainty fragments into disconnected pieces. These fragments then drift around and can accidentally get amplified into a confident, but wrong, response. We can group these errors into three main types: factual errors where the model states incorrect information, reasoning errors where the logic falls apart, and instruction-following errors where the model ignores the task you gave it. Understanding this breakdown is the first step toward designing more reliable systems. Next, we'll explore how prompting, context, and grounding strategies can help anchor models and reduce these failures.Hallucination: Why LLMs Make Things Upsciencedirect.comarxiv.orgaclanthology.org+22 min
  9. 09Prompting, Context, and Grounding StrategiesNow, to get the most out of these models, we need to talk about how we communicate with them. This is where prompting, context, and grounding come in. Think of the system prompt as the starting instruction set or the guardrails for the model's behavior. It’s like telling a new assistant, ‘You are a helpful, patient tutor who explains things simply,’ before you even ask a question. That sets the entire tone. Next, we have few-shot examples. By showing the model just a couple of examples of the format and tone you want right inside your prompt, it learns the desired pattern immediately. For complex reasoning, there’s a powerful technique called chain of thought prompting. Instead of just asking for an answer, you instruct the model to think through the problem step by step. This dramatically improves its accuracy on things like math problems or logic puzzles. Finally, we have Retrieval-Augmented Generation, or RAG. This is a game-changer for connecting a model to your own documents. It lets the model search your private database in real time and ground its answer in your specific information, not just its general training data. It’s like giving the model an open-book test with your company’s manuals. Up next, we’ll explore the different ways to access this technology, looking at the LLM ecosystem, from open models to APIs and deployment options.Prompting, Context, and Grounding Strategies2 min
  10. 10The LLM Ecosystem: Open Models, APIs, and Deployment OptionsNow let's look at the open LLM ecosystem. By mid-2026, open-weight models have reached genuine frontier parity, and permissive licenses are becoming the norm. Families like Llama 4, DeepSeek V4, Qwen 3.6, Gemma 4, and Phi-4 are all competitive options. You have real choice in how you deploy them. You can use hosted APIs through providers like Together AI or Groq and pay only per token. You can go with managed cloud inference on platforms like Azure Foundry, where the provider handles patching and scaling. Or you can self-host on-premise, which is often the right answer for regulated industries that need full data residency. Licensing does vary, so pay attention here. You will see MIT, Apache 2.0, and custom community licenses, some with monthly active user thresholds that trigger additional agreements. The key takeaway is that the landscape is more open and flexible than ever, and the right deployment model depends on your compliance needs, your scale, and your budget. Let's carry that into the next topic, where we will explore the trade-offs between cost, latency, and infrastructure.The LLM Ecosystem: Open Models, APIs, and Deployment Optionsalicelabs.aihuggingface.codev.to+22 min
  11. 11Cost, Latency, and Infrastructure Trade-offsNow let's talk about the real-world trade-offs when you're actually putting these models to work: cost, latency, and infrastructure. Picture three distinct price tiers. At the top, you have closed frontier APIs from the major labs, roughly two to fifteen dollars per million tokens. Then there's the middle tier, where you use an open-weight model but let a hosting provider run the hardware for you, typically five to fifty times cheaper. Finally, you can self-host, running the model entirely on your own machines. A key detail here: the exact same open-weight model can cost five to ten times more, or less, depending on which hosting provider you pick, because you're really paying for their specific hardware, speed, and margin. The break-even point for self-hosting depends on your volume, your latency requirements, and any compliance needs, like data residency. Architecturally, Mixture of Experts, or MoE, designs cut compute cost dramatically by activating only a fraction of the total parameters per token. The general rule of thumb is that high-volume workloads, above roughly fifty million tokens per month, tend to favor self-hosting economics. Up next, we'll look at the critical side of deploying these models: responsible use, bias, safety, and governance.Cost, Latency, and Infrastructure Trade-offsalicelabs.aihuggingface.codev.to+22 min
  12. 12Responsible Use: Bias, Safety, and GovernanceNow let’s talk about responsible use, because using these models well means managing bias, safety, and governance all at once. Bias isn’t one single thing. It can show up across gender, race, and very specific domains like healthcare or finance. The practical approach is to define the categories that matter most for your use case, and run targeted evaluations for each one. On the safety side, a complete program in 2026 covers six critical surfaces: jailbreaks and prompt injection, hallucination and factual drift, personally identifiable information leakage, bias and toxicity, role and policy violations, and supply chain risk. These aren’t separate checklists. They’re connected layers that need offline evaluations, runtime guardrails, and regular red teaming. Governance is where the legal timelines come in. The European Union AI Act phases general purpose AI obligations from August 2025, and broad high risk rules from August 2026. Meanwhile, the NIST AI Risk Management Framework guides your practice, and ISO slash IEC 42001 certifies your AI management system. Together, they move safety from an aspiration to something measurable and auditable. Next, we’ll dive into one of the most powerful tools for uncovering these risks: red teaming, or stress testing your AI systems.Responsible Use: Bias, Safety, and Governancearxiv.orgfutureagi.comdl.acm.org+22 min
  13. 13Red-Teaming: Stress-Testing Your AI SystemsNow we turn to one of the most powerful ways to build confidence in your AI systems before they reach real users: red-teaming. Think of it as a structured, adversarial stress test. Instead of waiting for something to go wrong in production, you actively try to break the system under controlled conditions. The process follows a six-phase model. You start by scoping the engagement, then build a threat model that maps likely attacker profiles. Next, you run automated scans for broad coverage, followed by deep manual exploration where skilled testers launch creative, multi-step attacks. After that, you score and document every finding, and finally, you verify that any fixes actually close the gaps. This isn't just a best practice anymore. The EU AI Act mandates adversarial testing for general-purpose AI models with systemic risk. At the board level, strategic red-teaming is also emerging as a governance tool. It stress-tests the assumptions behind a major AI commitment before the organization locks itself into a path it cannot easily reverse. The key output is documentation: test sets, failure rates, and the mitigations applied, all mapped to the relevant regulatory framework. Let's carry that discipline into the next slide, where we'll look at practical governance steps for teams.Red-Teaming: Stress-Testing Your AI Systemsarxiv.orgfutureagi.comdl.acm.org+22 min
  14. 14Practical Governance Steps for TeamsNow that we've explored the big picture, let's bring this down to the team level. What practical steps can you take today to build a real governance program? I want to share four concrete actions. First, build what we call a six-surface safety program. This means covering jailbreaks, hallucinations, data leakage, bias, toxicity, role violations, and supply-chain risks. For each surface, you need offline evaluations in your build pipeline, runtime guardrails in your gateway, production monitoring, and structured red-teaming. Don't skip any layer. A clean eval suite does not mean you can skip red-teaming. They find different failure modes. Second, treat documentation as the deliverable. Auditors don't just want to know you're safe. They want to see the evidence. That means maintaining model cards, risk registers, eval results, red-teaming reports, and incident logs. Traceability is what matters. Third, map each control to a framework. Tie your work to the EU AI Act, NIST AI RMF, or ISO 42001. This makes audit readiness a natural byproduct of your program, not a last-minute scramble. And fourth, red-team continuously. Run it before launch, monthly in steady state, and after every model or prompt change. Feed failing prompts back into your eval dataset. This closes the loop and makes your system stronger over time. Let's shift gears and talk about when to actually use an LLM, and when it might be the wrong tool for the job.Practical Governance Steps for Teamsarxiv.orgfutureagi.comdl.acm.org+22 min
  15. 15When to Use an LLM — and When Not ToSo, how do you decide when to reach for a large language model, and when to step back? It really comes down to three main levers: how accurate the result needs to be, how fast you need it, and what it costs to run. LLMs shine in areas like summarization, drafting, coding, analysis, and pulling information from a messy pile of documents. But they’re not the right tool for everything. Avoid using them for precise math, for questions that need real-time facts, or for high-stakes decisions where an error could cause real harm. In those cases, a calculator, a live database, or a deterministic system is a much safer bet. And here’s a key rule: for any regulated domain, like healthcare, law, or finance, you must keep a human in the loop. The model can suggest, but a person should always review and approve. Now let’s wrap up with a clear roadmap for your next steps.When to Use an LLM — and When Not To1 min
  16. 16Your Roadmap: Next Steps for Applying LLMsAnd here we are at the final stop on our roadmap. Let's turn everything we've covered into a practical, low-risk starting plan. First, start small. Pick a focused task, define what success looks like in simple terms, and measure the outcome honestly. Then iterate. Small wins build confidence and momentum far better than big, vague projects. Next, treat your prompts and evaluations like valuable assets. Build a shared prompt library, keep lightweight evaluation datasets, and capture best practices as you go. This turns individual learning into team capability. For any decision that carries real risk or touches regulations, always keep a human in the loop. Trust the model as a helpful advisor, not as the final decision-maker. Finally, make continuous learning part of your routine. Follow a few trusted communities, skim research summaries, and stay curious about how the technology is evolving. You don't need to read everything, just enough to stay informed and avoid surprises. Thank you for spending this time with me. You now have a solid mental model of how large language models work and a practical path to apply them responsibly. Start small, stay curious, and enjoy the journey ahead.Your Roadmap: Next Steps for Applying LLMs2 min

Sources consulted

Web sources consulted while building this course.