人工智能原理Lecture 7 自编码器 Autoencoders -PPT精品课件.pptx

Lecture 7: AutoEncoders Artificial Intelligence2Generative Models October 17, 2019 Artificial Intelligence3October 17, 2019Outline?Autoencoders ? Autoencoders ? Denoising Autoencoders ? Contractive Autoencoder ? Deep Autoencoders ? Variational Encoders ? Multimodal Deep Learning Artificial Intelligence4October 17, 2019Autoencoders???EncoderDecoderParameters Artificial Intelligence5Autoencoders October 17, 2019 Artificial Intelligence6October 17, 2019Autoencoders?Feed-forward neural network trained to reproduce its input atthe output layer Artificial Intelligence7Autoencoders October 17, 2019 Artificial Intelligence8Autoencoders October 17, 2019 Artificial Intelligence9Autoencoders October 17, 2019 Artificial Intelligence10October 17, 2019Autoencoders?Loss function for binary inputs ? Cross-entropy error function (reconstruction loss)? Loss function for real-valued inputs ? sum of squared differences (reconstruction loss) ? we use a linear activation function at the output Artificial Intelligence11Autoencoders October 17, 2019 Artificial Intelligence12Example: MNIST October 17, 2019 Artificial Intelligence13Learned Features October 17, 2019 Artificial Intelligence14October 17, 2019Autoencoders??Autoencoder gives low reconstruction error on textexamples from the same distribution as the trainingexamples, but generally high reconstruction error onsamples randomly chosen from the input spaceAutoencoder is a multi-layer neural network. The onlydifference is that the size of its output layer is the sameas the input layer and the objective function 15October 17, 2019 Artificial IntelligenceOptimality of the Linear Autoencoder ? With nonlinear hidden units, we have a nonlinear generalization of PCA 16October 17, 2019 Artificial IntelligenceUndercomplete Representation?Hidden layer is undercomplete ifsmaller than the input layer(bottleneck layer, e.g.dimensionality reduction): ? hidden layer ‘‘compresses’’ the input ? will compress well only for the t