Peer-Reviewed Journal Details
Mandatory Fields
Ma Y.;Way A.
2009
May
ACM Transactions on Asian Language Information Processing
Bilingually motivated word segmentation for statistical machine translation
Published
()
Optional Fields
Alignment Bilingually motivated Phrase-based statistical machine translation Word segmentation
8
2
We introduce a bilingually motivated word segmentation approach to languages where word boundaries are not orthographically marked, with application to Phrase-Based Statistical Machine Translation (PB-SMT). Our approach is motivated from the insight that PB-SMT systems can be improved by optimizing the input representation to reduce the predictive power of translation models. We firstly present an approach to optimize the existing segmentation of both source and target languages for PB-SMT and demonstrate the effectiveness of this approach using a Chinese - English MT task, that is, to measure the influence of the segmentation on the performance of PB-SMT systems. We report a 5.44% relative increase in Bleu score and a consistent increase according to other metrics. We then generalize this method for Chinese word segmentation without relying on any segmenters and show that using our segmentation PB-SMT can achieve more consistent state-of-the-art performance across two domains. There are two main advantages of our approach. First of all, it is adapted to the specific translation task at hand by taking the corresponding source (target) language into account. Second, this approach does not rely on manually segmented training data so that it can be automatically adapted for different domains. © 2009 ACM.
1530-0226
10.1145/1526252.1526255
Grant Details