RL — LECTURE 01 · BEGINNER TO INTERMEDIATE · NEXT LECTURE: MARKOV DECISION PROCESS (MDP)
Introduction to Reinforcement Learning
Instead of memorizing a giant answer book, an intelligent agent learns through interaction: try an action, observe what changes, and receive rewards or penalties. Explore every core RL idea visually before heavy mathematics.
One-Look Intuition Map of Reinforcement Learning
01 LEARNING THROUGH TRIAL & ERROR
No answer key — just action, consequence, and adjustment
Reinforcement Learning is a machine learning approach in which an agent learns to make decisions by interacting with an environment and receiving rewards or penalties.
ONE-LOOK REMINDER · THE TRIAL-AND-ERROR HABIT
Pick an example card below to see how one universal 5-step habit powers bicycles, maze robots, and game AIs.
INTERACTIVE TRIAL LAB · THREE MYSTERY DOORS
Guide the Agent: Try Any Door and Watch Its Confidence Adapt
Click Door A, Door B, or Door C below. Watch our Agent step toward that door, react to the consequence, and adjust its future confidence bars.
02 THREE MACHINE LEARNING PARADIGMS
Teachers, pattern clusters, and learning by doing
Compare how Supervised Learning, Unsupervised Learning, and Reinforcement Learning receive information and improve over time.
ONE-LOOK DISTINCTION
Supervised copies a labeled answer key · Unsupervised discovers hidden clusters · Reinforcement learns from action consequences.
“Which learning approach would you choose for training a robot to walk?”
03 FIVE FUNDAMENTAL COMPONENTS OF RL
One Robot, One Maze — five building blocks in action
Meet every Reinforcement Learning cast member inside a single interactive maze: move the Robot yourself and click any component on the right to spotlight it inside the scene.
ONE-LOOK CAST SHEET · ROBOT NAVIGATING A MAZE
Agent = Robot · Environment = Maze World · State = Current Position · Action = Direction Picked · Reward = +10 Goal / -1 Move / -2 Wall.
Tip: Keyboard arrow keys also steer the Robot!
04 AGENT–ENVIRONMENT INTERACTION LOOP
The infinite conversation: observe, act, react, learn
Advance step by step across the loop boundary to see how the Agent and Environment trade signals at every discrete tick t.
ONE-LOOK NOTATION DECODER · S(t) → A(t) → R(t+1), S(t+1)
Read left to right: See situation S(t) → Pick move A(t) → World pays R(t+1) & updates situation to S(t+1).
05 EPISODES, TERMINAL STATES & TRAJECTORIES
One complete attempt — and its recorded play-by-play
In tasks with clear endings (mazes, board games, arcade rounds), every full attempt runs from Start until hitting a Terminal State.
ONE-LOOK VOCABULARY
Episode = One complete run (Kickoff to Finish) · Terminal State = Goal/Game-Over boundary · Trajectory = Recorded step sequence.
06 REWARD VS RETURN & DISCOUNT FACTOR γ
Immediate coin today vs accumulated return over time
Reward is instant single-step feedback. Return G(t) sums rewards across steps — weighted by Discount Factor γ (between 0 and 1) to reflect how strongly future rewards matter today.
ONE-LOOK INTUITION · SHORT-TERM VS LONG-TERM THINKER
“A short-term thinker (γ near 0) cares mostly about immediate rewards. A long-term thinker (γ near 1) gives big importance to future goals.”
| Action Step | Immediate Reward |
|---|---|
| Move 1 | -1 |
| Move 2 | -1 |
| Move 3 | -1 |
| Reach Goal | +10 |
Undiscounted: G(t) = R(t+1) + R(t+2) + ...
Discounted: G(t) = Σ γ^k · R(t+k+1)
Notice how k = 0 leaves today unshrunk (γ^0 = 1), while k = 1, 2, 3, 4 shrinks further futures when γ < 1.
07 EXPLORATION VS EXPLOITATION
Play it safe with what works — or test untried treasures?
Every learner faces a dilemma: Exploitation repeats the best choice already seen, while Exploration tests unfamiliar actions to uncover hidden better payouts.
Pull Levers, Track Averages & Uncover the Hidden Best Machine
08 WHAT IS A POLICY?
The Agent's Playbook — mapping situation to action
A Policy is the strategy an agent follows to decide which action to take in a given situation. It can lock onto a single fixed rule (Deterministic) or roll across action probabilities (Stochastic).
ONE-LOOK DISTINCTION · RULEBOOK VS PROBABILITY DICE
Deterministic Policy: same state always picks the exact same action · Stochastic Policy: assigns probabilities (e.g. RIGHT 60%, LEFT 20%, UP 10%, DOWN 10%).
Note: Modern AI agents and language models also rely on probability distributions over choices (words or actions) when deciding what comes next.
09 TWO TYPES OF RL METHODS · VALUE-BASED VS POLICY-BASED
Two ways an RL brain decides: Score Maps vs Direct Reflexes
Both methods seek the highest cumulative return, yet they store completely different knowledge inside the Agent: 1 · Value-Based learns desirability scores first, whereas 2 · Policy-Based tunes action habits directly.
ONE-LOOK INTUITION · SCOREKEEPER VS COACH'S PLAYBOOK
Value-Based scores every move or square first (+9 beats +4) and picks the winner · Policy-Based adjusts action probabilities directly (RIGHT 82%) with zero score table!
Watch How Both Methods Improve Across Trials
| Dimension | 1 · Value-Based RL | 2 · Policy-Based RL |
|---|
Looking ahead: later advanced RL lectures also show how intelligent agents can combine a value scorekeeper and a direct policy together!
10 PRACTICAL APPLICATIONS OF RL
Where sequential trial-and-error shapes the real world
Whenever decisions ripple across time — and feedback arrives only after acting — Reinforcement Learning shines.
How Playing Millions of Episodes Against Itself Unlocked Original Strategy
Go has more board arrangements than atoms in the observable universe. By playing full Episodes against past versions of itself (+1 Win / -1 Loss), AlphaGo discovered original tactics like Game 2 “Move 37” that even world masters had never seen.
11 REVISION & MASTERY CHECK
Check your RL intuition across every core idea
Test all core fundamentals — components, feedback loops, episodes, discounted return, bandits, policies, and Value vs Policy methods.
LECTURE 01 CLOSING SUMMARY “Reinforcement Learning is about learning through interaction. An agent takes actions, receives feedback, explores possibilities, and gradually improves its decision-making strategy.”