深度学习在自然语言(nlp)的实践-.doc

Practical Neural Networks for NLP Chris Dyer, Yoav Goldberg, Graham Neubig November 1, 2016 EMNLP  Neural Nets and Language Tension: Language and neural nets ? Language is discrete and structured ? Sequences, trees, graphs ? Neural nets represent things with continuous vectors ? Poor “native support” for structure ? The big challenge is writing code that translates between the ? {discrete-structured, continuous} regimes This tutorial is about one framework that lets you use the power of ? neural nets without abandoning familiar NLP algorithms  Outline Part 1 ? Computation graphs and their construction Neural Nets in DyNet ? ? ? ? ? Recurrent neural networks Minibatching Adding new differentiable functions  Outline Part 2: Case Studies ? Tagging with bidirectional RNNs ? ? ? Transition-based dependency parsing Structured prediction meets deep learning  Computation Graphs Deep Learning’s Lingua Franca  expression: y=xAx+b·x+c  expression: y=xAx+b·x+c graph: x  expression: y=xAx+b·x+c graph: A node is a {tensor, matrix, vector, scalar} value x  expression: y=xAx+b·x+c graph: f(u)=u x  An edge represents a function argument (and also an data dependency). They are just y=x Ax+b·x+c pointers to nodes. graph: f(u)=u x  An edge represents a function argument (and also an data dependency). They are just y=x Ax+b·x+c pointers to nodes. A node with an incoming edge is a function of graph: that edge’s tail node. f(u)=u x  An edge represents a function argument (and also an data dependency). They are just y=x Ax+b·x+c pointers to nodes. A node with an incoming edge is a function of graph: that edge’s tail node. A node knows how to compute its value and the value of its derivative w.r.t each argument (edge) @F times a derivative of an arbitrary input . @f(u) @f@(uu)@f@F(u) =?@f@(Fu) ◆ f(u)=u x  expression: y=xAx+b·x+c graph: Functions can be nullary, unary, binary, … n-ary. Often they are unary or binary. f(U,V)=UV f(u)=u A x  expre