Notation “state variables” represent the contents of the state; “state” is a complete assignment of state variables. New Concepts Markov Decision Process stationary Markov Decision Process finite-horizon models + infinite-horizon models policy stationary policy optimal policy and optimal value function policy evaluation and policy iteration lookahead equation Bellman Expectation Equation action-value function and value-function policy advantage function Important Results / Claims policy evaluation methods solving for the utility of a policy finding the best policy policy iteration Questions why is it d seperated Interesting Factoids