This paper discusses neural machine translation (NMT), a new paradigm in the MT field,
comparing the quality of NMT systems with statistical MT by describing three studies using
automatic and human evaluation methods. Automatic evaluation results presented for NMT
are very promising, however human evaluations show mixed results. We report increases in
fluency but inconsistent results for adequacy and post-editing effort. NMT undoubtedly represents
a step forward for the MT field, but one that the community should be careful not to
oversell.