CAS logo CAS RL LABLecture 01 · RL Basics ← Series

RL — LECTURE 01 · BEGINNER TO INTERMEDIATE · NEXT LECTURE: MARKOV DECISION PROCESS (MDP)

Introduction to Reinforcement Learning

“How can a machine learn to play a game without being given the correct answer for every move?”

Instead of memorizing a giant answer book, an intelligent agent learns through interaction: try an action, observe what changes, and receive rewards or penalties. Explore every core RL idea visually before heavy mathematics.

SIX VISUAL CHAPTERS

One-Look Intuition Map of Reinforcement Learning

01 LEARNING THROUGH TRIAL & ERROR

No answer key — just action, consequence, and adjustment

Reinforcement Learning is a machine learning approach in which an agent learns to make decisions by interacting with an environment and receiving rewards or penalties.

↺

ONE-LOOK REMINDER · THE TRIAL-AND-ERROR HABIT

Pick an example card below to see how one universal 5-step habit powers bicycles, maze robots, and game AIs.

1. Try Something 2. Observe What Happens 3. Receive Feedback 4. Learn From Mistakes 5. Improve Next Attempt

INTERACTIVE TRIAL LAB · THREE MYSTERY DOORS

Guide the Agent: Try Any Door and Watch Its Confidence Adapt

Click Door A, Door B, or Door C below. Watch our Agent step toward that door, react to the consequence, and adjust its future confidence bars.

Click a door above to begin testing!
AGENT'S LEARNED CONFIDENCE PER DOOR

02 THREE MACHINE LEARNING PARADIGMS

Teachers, pattern clusters, and learning by doing

Compare how Supervised Learning, Unsupervised Learning, and Reinforcement Learning receive information and improve over time.

3×

ONE-LOOK DISTINCTION

Supervised copies a labeled answer key · Unsupervised discovers hidden clusters · Reinforcement learns from action consequences.

QUICK INTUITION CHECK · WHICH PARADIGM FITS BEST?

“Which learning approach would you choose for training a robot to walk?”

Pick one of the three choices above to check your reasoning.

03 FIVE FUNDAMENTAL COMPONENTS OF RL

One Robot, One Maze — five building blocks in action

Meet every Reinforcement Learning cast member inside a single interactive maze: move the Robot yourself and click any component on the right to spotlight it inside the scene.

5★

ONE-LOOK CAST SHEET · ROBOT NAVIGATING A MAZE

Agent = Robot · Environment = Maze World · State = Current Position · Action = Direction Picked · Reward = +10 Goal / -1 Move / -2 Wall.

Tip: Keyboard arrow keys also steer the Robot!

CLICK A COMPONENT TO SPOTLIGHT IT ON THE MAZE

04 AGENT–ENVIRONMENT INTERACTION LOOP

The infinite conversation: observe, act, react, learn

Advance step by step across the loop boundary to see how the Agent and Environment trade signals at every discrete tick t.

⟳

ONE-LOOK NOTATION DECODER · S(t) → A(t) → R(t+1), S(t+1)

Read left to right: See situation S(t) → Pick move A(t) → World pays R(t+1) & updates situation to S(t+1).

S(t) = Situation right now A(t) = Decision made R(t+1) = Reward received S(t+1) = Situation afterward

05 EPISODES, TERMINAL STATES & TRAJECTORIES

One complete attempt — and its recorded play-by-play

In tasks with clear endings (mazes, board games, arcade rounds), every full attempt runs from Start until hitting a Terminal State.

★

ONE-LOOK VOCABULARY

Episode = One complete run (Kickoff to Finish) · Terminal State = Goal/Game-Over boundary · Trajectory = Recorded step sequence.

06 REWARD VS RETURN & DISCOUNT FACTOR γ

Immediate coin today vs accumulated return over time

Reward is instant single-step feedback. Return G(t) sums rewards across steps — weighted by Discount Factor γ (between 0 and 1) to reflect how strongly future rewards matter today.

γ

ONE-LOOK INTUITION · SHORT-TERM VS LONG-TERM THINKER

“A short-term thinker (γ near 0) cares mostly about immediate rewards. A long-term thinker (γ near 1) gives big importance to future goals.”

Action StepImmediate Reward
Move 1-1
Move 2-1
Move 3-1
Reach Goal+10
CUMULATIVE RETURN EQUATIONS

Undiscounted: G(t) = R(t+1) + R(t+2) + ...

Discounted: G(t) = Σ γ^k · R(t+k+1)

Notice how k = 0 leaves today unshrunk (γ^0 = 1), while k = 1, 2, 3, 4 shrinks further futures when γ < 1.

MANDATORY PLAYGROUND · DISCOUNT FACTOR γ SLIDER (0 → 1)
DRAG DISCOUNT FACTOR γ: γ = 0.90

07 EXPLORATION VS EXPLOITATION

Play it safe with what works — or test untried treasures?

Every learner faces a dilemma: Exploitation repeats the best choice already seen, while Exploration tests unfamiliar actions to uncover hidden better payouts.

INTERACTIVE MULTI-ARMED BANDIT CASINO

Pull Levers, Track Averages & Uncover the Hidden Best Machine

REPRODUCIBLE STRATEGY COMPARISON (EXPLORE VS EXPLOIT VS BALANCED)

08 WHAT IS A POLICY?

The Agent's Playbook — mapping situation to action

A Policy is the strategy an agent follows to decide which action to take in a given situation. It can lock onto a single fixed rule (Deterministic) or roll across action probabilities (Stochastic).

π

ONE-LOOK DISTINCTION · RULEBOOK VS PROBABILITY DICE

Deterministic Policy: same state always picks the exact same action · Stochastic Policy: assigns probabilities (e.g. RIGHT 60%, LEFT 20%, UP 10%, DOWN 10%).

UP ↑10%
DOWN ↓10%
LEFT ←20%
RIGHT →60%
Adjust any slider above or roll the dice to watch our Robot sample from its Policy!

Note: Modern AI agents and language models also rely on probability distributions over choices (words or actions) when deciding what comes next.

09 TWO TYPES OF RL METHODS · VALUE-BASED VS POLICY-BASED

Two ways an RL brain decides: Score Maps vs Direct Reflexes

Both methods seek the highest cumulative return, yet they store completely different knowledge inside the Agent: 1 · Value-Based learns desirability scores first, whereas 2 · Policy-Based tunes action habits directly.

⚖

ONE-LOOK INTUITION · SCOREKEEPER VS COACH'S PLAYBOOK

Value-Based scores every move or square first (+9 beats +4) and picks the winner · Policy-Based adjusts action probabilities directly (RIGHT 82%) with zero score table!

1 · Value-Based = Score Map First 2 · Policy-Based = Direct Playbook Goal of Both = Maximize Long-Term Return

INTERACTIVE COMPARISON ARENA · SAME PROBLEM, TWO RL BRAINS

Watch How Both Methods Improve Across Trials

SIDE-BY-SIDE COMPARISON AT A GLANCE
Dimension1 · Value-Based RL2 · Policy-Based RL

Looking ahead: later advanced RL lectures also show how intelligent agents can combine a value scorekeeper and a direct policy together!

10 PRACTICAL APPLICATIONS OF RL

Where sequential trial-and-error shapes the real world

Whenever decisions ripple across time — and feedback arrives only after acting — Reinforcement Learning shines.

SPOTLIGHT CASE · ALPHAGO SELF-PLAY

How Playing Millions of Episodes Against Itself Unlocked Original Strategy

Go has more board arrangements than atoms in the observable universe. By playing full Episodes against past versions of itself (+1 Win / -1 Loss), AlphaGo discovered original tactics like Game 2 “Move 37” that even world masters had never seen.

11 REVISION & MASTERY CHECK

Check your RL intuition across every core idea

Test all core fundamentals — components, feedback loops, episodes, discounted return, bandits, policies, and Value vs Policy methods.

YOUR MASTERY SCORE
0 / 10
LECTURE 01 CLOSING SUMMARY “Reinforcement Learning is about learning through interaction. An agent takes actions, receives feedback, explores possibilities, and gradually improves its decision-making strategy.”