Michael Data

Introduced in a 2014 paper, Generative Adversarial Nets are a framework for an ensemble of machine learning models. The original paper is generally model-agnostic but emphasizes neural networks (the nets).

The basic structure consists of a generative model and a discriminative model. The generative model is trained on uninformed noise input. The discriminative model is trained on samples of the generative model and informative training data and performs binary classification between the two.

Game Theory Analogy

Under a game theory framework, the two classifiers compete as adversaries until they reach an equilibrium. When complete, the generative model mimics training data perfectly and the discriminative model has 50% accuracy 1).

Ensemble Analogy

Analogous to boosting in decision trees: If the generator was trained on regular training/holdout data, the discriminator is analogous to a boosted second model in the ensemble, trained on the error residuals of the first model. Whereas models in decision tree ensembles are cooperative, GAN base models function competitively.

Base Models