Welcome
Welcome to the Winter 2024 edition of CME 241: Foundations of Reinforcement Learning with Applications in Finance.
- Instructor: Ashwin Rao
- Lectures: Wed & Fri 4:30pm-5:50pm in Littlefield Center 103
- Ashwin’s Office Hours: Fri 2:30pm-4:00pm (or by appointment) in ICME Mezzanine level, Room M05
- Course Assistant (CA): Greg Zanotti
- Greg’s Office Hours: Wed & Thur 12:00-1:00pm on Zoom
Table of contents
Overview of the Course
- Theory of Markov Decision Processes (MDPs)
- Dynamic Programming (DP) Algorithms
- Backward Induction (BI) and Approximate DP (ADP) Algorithms
- Reinforcement Learning (RL) Algorithms
- Plenty of Python implementations of models and algorithms
- We apply these algorithms to 5 Financial/Trading problems:
- (Dynamic) Asset-Allocation to maximize Utility of Consumption
- Pricing and Hedging of Derivatives in an Incomplete Market
- Optimal Exercise/Stopping of Path-dependent American Options
- Optimal Trade Order Execution (managing Price Impact)
- Optimal Market-Making (Bid/Ask managing Inventory Risk)
- By treating each of the problems as MDPs (i.e., Stochastic Control)
- We will go over classical/analytical solutions to these problems
- Then we will introduce real-world considerations, and tackle with RL (or DP)
- The course blends Theory/Mathematics, Programming/Algorithms and Real-World Financial Nuances
Learning Material
- Course Textbook (henceforth known as “RLForFinanceBook”). PDF version available as free download
- Python codebase Tikhon Jelvis and I have developed to help you “learn through coding”
- Technical Documents/Lecture Slides/Assignments Greg and I have prepared for this course
- Ed Discussion (Online discussion forum)
Grading
- 40% Exam (in week 8)
- 30% Group Assignments (to be done until Week 7)
- 30% Course Project
Access and Accommodations
Stanford is committed to providing equal educational opportunities for disabled students. Disabled students are a valued and essential part of the Stanford community. We welcome you to our class.
If you experience disability, please register with the Office of Accessible Education (OAE). Professional staff will evaluate your needs, support appropriate and reasonable accommodations, and prepare an Academic Accommodation Letter for faculty. To get started, or to re-initiate services, please visit oae.stanford.edu
If you already have an Academic Accommodation Letter, we invite you to share your letter with us. Academic Accommodation Letters should be shared at the earliest possible opportunity so we may partner with you and OAE to identify any barriers to access and inclusion that might be encountered in your experience of this course.