Peer-Reviewed Journal Details
Mandatory Fields
Way A.;Gough N.
2005
March
Machine Translation
Controlled translation in an example-based environment: What do automatic evaluation metrics tell us?
Published
()
Optional Fields
Controlled translation Evaluation Example-based MT Marker Hypothesis
19
1
1
36
This paper presents an extended, harmonised account of our previous work on integrating controlled language data in an Example-based Machine Translation system. Gough and Way in MT Summit pp. 133-140 (2003) focused on controlling the output text in a novel manner, while Gough and Way (9th Workshop of the EAMT, (2004a), pp. 73-81) sought to constrain the input strings according to controlled language specifications. Our original sub-sentential alignment algorithm could deal only with 1:1 matches, but subsequent refinements enabled n:m alignments to be captured. A direct consequence was that we were able to populate the system's databases with more than six times as many potentially useful fragments. Together with two simple novel improvements - correcting a small number of mistranslations in the lexicon, and allowing multiple translations in the lexicon - translation quality improves considerably. We provide detailed automatic and human evaluations of a number of experiments carried out to test the quality of the system. We observe that our system outperforms the rule-based on-line system Logomedia on a range of automatic evaluation metrics, and that the 'best' translation candidate is consistently highly ranked by our system. Finally, we note in a number of tests that the BLEU metric gives objectively different results than other automatic evaluation metrics and a manual evaluation. Despite these conflicting results, we observe a preference for controlling the source data rather than the target translations. © Springer 2006.
0922-6567
10.1007/s10590-005-1403-8
Grant Details