Human Effort and Machine Learnability in Computer Aided Translation.pdf

Human Effort and Machine Learnability in Computer Aided Translation Spence Green, Sida Wang, Jason Chuang,* Jeffrey Heer,* Sebastian Schuster, and Christopher D. Manning Computer Science Department, Stanford University {spenceg,sidaw,sebschu,manning}@ *Computer Science Department, University of Washington {jcchuang,jheer}@ Abstract Analyses of computer aided translation typi- cally focus on either frontend interfaces and human effort, or backend translation and machine learnability of corrections. How- ever, this distinction is artificial in prac- tice since the frontend and backend must work in concert. We present the first holis- tic, quantitative evaluation of these issues by contrasting two assistive modes: post- editing and interactive machine translation (MT). We describe a new translator inter- face, extensive modifications to a phrase- based MT system, and a novel objective function for re-tuning to human correc- tions. Evaluation with professional bilin- gual translators shows that post-edit is faster than interactive at the cost of translation quality for French-English and English- German. However, re-tuning the MT sys- tem to interactive output leads to larger, sta- tistically significant reductions in HTER versus re-tuning to post-edit. Analysis shows that tuning directly to HTER results in fine-grained corrections to subsequent machine output. 1 Introduction The goal of machine translation has always been to reduce human effort, whether by partial assistance or by outright replacement. However, preoccupa- tion with the latter—fully automatic translation—at the exclusion of the former has been a feature of the research community since its first nascent steps in the 1950s. Pessimistic about progress during that decade and future prospects, Bar-Hillel (1960, p.3) argued that more attention should be paid to a “machine-post-editor partnership,” whose decisive problem is “the region of optimality in the contin- uum of possible divisions of labor.” Today, wi