Calendar

Week 1

Jan 10

Lecture Slides: Course Overview

Readings: Chapter 1 of RLForFinanceBook

Assignments: Assignment 1: Instructions to get set up for the course

Jan 12

Lecture Slides: Guided Tour of Chapter 3: Markov Process and Markov Reward Process

Readings: Chapter 3 of RLForFinanceBook

Assignments: Assignment 2 due Jan 22

Optional: Review programming principles for the course in this IPython notebook

Week 2

Jan 17

Lecture Slides: Markov Decision Processes (MDP), Value Functions, and Bellman Equations

Readings: Chapter 4 of RLForFinanceBook (Section on POMDP not exam-able)

Jan 19

Lecture Slides: Dynamic Programming Algorithms

Readings: Chapter 5 of RLForFinanceBook

Assignments: Assignment 3: Due Jan 29 11:59pm

Week 3

Jan 24

Lecture Slides: Function Approximation and Approximate Dynamic Programming Algorithms

Readings: Chapter 6 of RLForFinanceBook

Optional: Appendix F of RLForFinanceBook

Jan 26

Lecture Slides: Understanding Risk-Aversion through Utility Theory (as a pre-req for Finance Applications)

Readings: Chapter 7 of RLForFinanceBook

Optional: Appendix A and C of RLForFinanceBook

Assignments: Assignment 4: Due Feb 5 11:59pm

Week 4

Jan 31

Lecture Slides: Application Problem 1 - Dynamic Asset-Allocation and Consumption

Readings: Chapter 8 of RLForFinanceBook

Optional: Optional: Appendix B, C, and D of RLForFinanceBook; Some (rough) pointers on Discrete versus Continuous MDPs, and solution techniques

Feb 2

Lecture Slides: Application Problems 2 and 3 - Optimal Exercise of American Options and Optimal Hedging of Derivatives in Incomplete Markets

Readings: Intro to Derivatives section in Chapter 9 of RLForFinanceBook

Optional: Appendix C and E of RLForFinanceBook; Derivatives Pricing Theory in Chapter 9 of RLForFinanceBook (Derivatives Pricing Theory is not exam-able); Relevant sections in Chapter 9 of RLForFinanceBook for Optimal Exercise and Optimal Hedging in Incomplete Markets; Foundations of Arbitrage-Free and Complete Markets

Assignments: Assignment 5: Due Feb 12 11:59pm

Week 5

Feb 7

Lecture Slides: Application Problem 4 - Optimal Trade Order Execution

Readings: Optimal Trade Order Execution section in Chapter 10 of RLForFinanceBook

Feb 9

Lecture Slides: Application Problem 5 - Optimal Market-Making

Readings: Optimal Market-Making section in Chapter 10 of RLForFinanceBook

Assignments: Assignment 6: Due Feb 19 11:59pm

Week 6

Feb 14

Lecture Slides: RL for Prediction (Monte-Carlo and Temporal-Difference)

Readings: MC and TD sections in Chapter 11 of RLForFinanceBook

Feb 16

Lecture Slides: RL for Prediction (Eligibility Traces and TD(Lambda))

Readings: Eligibility Traces and TD(Lambda) sections in Chapter 11 of RLForFinanceBook

Assignments: Assignment 7: Due Mar 04 11:59pm

Week 7

Feb 21

Lecture Slides: RL for Control (Optimal Value Function/Optimal Policy)

Readings: Chapter 12 of RLForFinanceBook

Feb 23

Lecture Slides: Batch RL, Experience-Replay, DQN, LSPI

Readings: Sections 13.1 to 13.6 in Chapter 13 of RLForFinanceBook

Week 8

Feb 28

Lecture Slides: Value Function Geometry and Gradient TD

Readings: Sections 13.7 and 13.8 in Chapter 13 of RLForFinanceBook

Mar 1

Lecture Slides: Policy Gradient Algorithms

Readings: Chapter 14 of RLForFinanceBook

Week 9

Mar 6

Lecture Slides: Exploration versus Exploitation (Multi-Armed Bandits)

Readings: Chapter 15 of RLForFinanceBook

Mar 8

Lecture Slides: Blending Learning and Planning, Planning & Control for Inventory & Pricing

Readings: Chapter 16 of RLForFinanceBook + Bonus Topic

Week 10

Mar 13: Guest Lecture (TBD)
Mar 15: Final Project Presentations (Class extended from 4:30pm to 7:00pm)