Towards Training AI Agents with All Types of Experiences: A Standardized ML Formalism

Machine Learning (ML) is about computational methods that enable machines to learn concepts from experiences. In handling a wide range of experiences ranging from data instances, knowledge, constraints, to rewards, adversaries, and lifelong interplay in an ever-growing spectrum of tasks, contemporary ML/AI research has  resulted in a large multitude of learning paradigms (e.g., supervised, unsupervised, active, reinforcement, adversarial learning), models, optimization techniques, not mentioning countless approximation heuristics and tuning tricks, plus  ombinations

of all above. While pushing the field forward rapidly, these results also make mastering existing ML techniques costly, and make it difficult, if possible at all, to build AI agents that are capable of learning from all types of experiences and thus are reusable, repeatable, reliable, and explainable in ML/AI applications and productions. In this dissertation, I present a standardized mathematical formalism of machine

learning that offers a principled framework for  understanding, unifying, and generalizing current major paradigms of learning algorithms, and for designing new

families of algorithmic solutions and applications for learning with all experiences, in a composable and mechanic manner. The dissertation consists of four parts, where we study and apply the standardized ML formalism from theoretical, methodological, applicational, and operational

aspects. In Part I, we establish the simple yet general formalism, materialized as a standard equation of the objective function which characterizes experience, divergence, and uncertainty in a learning system. The standard equation provides a succinct, structured formulation of a vast design space of learning algorithms, and is justified as we show that a wide range of well-known algorithms with varying losses, constraints, and forms of experiences, all fall under its umbrella. In Part II, we show

the formalism is a natural framework for making use of arbitrary available experiences to learn models of interest. On this basis, we develop new mechanisms of

learning that go beyond reliance on data instances, and train models (e.g., deep neural networks) by integrating declarative logical rules, as well as rich auxiliary models

from relevant tasks. The studies also yield a new set of applications for controllable text generation. In Part III, we show the unified formalism opens up a wide range of

opportunities for extending originally specialized algorithms to solve new problems. In particular, we show how a set of seemingly unrelated problems, including training with fuzzy knowledge, automated data augmentation, and stabilizing GAN training, are essentially the same problem within the standardized framework, corresponding to joint model-experience co-learning, and can all be addressed by simply repurposing existing algorithms in the fertile research area of reinforcement learning. In Part IV, we further operationalize the standardized framework by developing a composable ML toolkit, Texar, that allows users to quickly assemble ML solutions to their problems by putting together standard and reusable building blocks.