We consider a general class of dynamic programming (DP) problems with nonseparable objective functions. We show that for any problem in this class, there exists an augmented-state DP problem that satisfies the principle of optimality and the solutions to which yield solutions to the original problem. Furthermore, we identify a subclass of DP problems with naturally forward separable objective functions for which this state-augmentation scheme is tractable. We extend this framework to stochastic DP problems, proposing a suitable definition of the principle of optimality. We then apply the resulting algorithms to the problem of optimal battery scheduling with demand charges using a data-based stochastic model for electricity usage and solar generation by the consumer.