Despite the reliability of modern power systems, large blackouts due to cascading failures (CFs) do occur in power grids with enormous economic and societal costs. In this article, CFs in power grids are theoretically modeled proposing a Markov decision process (MDP) framework with the aim of developing optimal load-shedding (LS) policies to mitigate CFs. The embedded Markov chain of the MDP, established earlier to capture the dynamics of CFs, features a reduced state-space and state-dependent transition probabilities. We introduce appropriate actions affecting the dynamics of CFs and associated costs. Optimal LS policies are computed that minimize the expected cumulative cost associated with CFs. Numerical simulations on the IEEE 118 and IEEE 300 bus systems show that the actions derived by the MDP result in minimum total cost of CFs, compared to fixed and random policies. Moreover, the optimality of derived policies is validated by a CF simulation based on dc power flow for the IEEE 118 bus system. Therefore, such actions developed by the proposed theoretical MDP framework can serve as a baseline for devising optimal LS strategies to mitigate CFs in power grids.