【李宏毅机器学习课件】25.RNN.pdf

Recurrent Neural Network (RNN) Example Application • Slot Filling I would like to arrive Taipei on November 2nd. ticket booking system Destination: Taipei Slot time of arrival: November 2nd Example Application y 1 y 2 Solving slot filling by Feedforward network? Input: a word (Each word is represented as a vector) Taipei x1 x2 1-of-N encoding How to represent each word as a vector? 1-of-N Encoding lexicon = {apple, bag, cat, dog, elephant} The vector is lexicon size. apple = [ 1 0 0 0 0] bag = [ 0 1 0 0 0] Each dimension corresponds to a word in the lexicon cat = [ 0 0 1 0 0] The dimension for the word dog = [ 0 0 0 1 0] is 1, and others are 0 elephant = [ 0 0 0 0 1] Beyond 1-of-N encoding Dimension for “Other” Word hashing apple 0 a-a-a 0 bag 0 a-a-b 0 … … cat 0 a-p-p 1 dog 0 … … 26 X 26 X 26 elephant 0 p-l-e 1 … … … p-p-l 1 “other” 1 … … w = “apple” w = “Gandalf” w = “Sauron