Paper
Learning When to Translate for Streaming Speech
How to find proper moments to generate partial sentence translation given a streaming speech input? Existing approaches waiting-and-translating for a fixed duration often break the acoustic units in speech, since the boundaries between acoustic units in speech are not even. In this paper, we propose MoSST, a simple yet effective method for translating streaming speech content. Given a usually long speech sequence, we · Authors: Qianqian Dong, Yaoming Zhu, Mingxuan Wang, Lei Li · Published 2021-09-15 · arXiv:2109.07368
- Published
- 2021-09-15
- Authors
- 4
- arXiv
- 2109.07368
- Category
- cs.CL
Abstract
arXiv:2109.07368
Qianqian Dong, Yaoming Zhu, Mingxuan Wang, Lei Li
How to find proper moments to generate partial sentence translation given a streaming speech input? Existing approaches waiting-and-translating for a fixed duration often break the acoustic units in speech, since the boundaries between acoustic units in speech are not even. In this paper, we propose MoSST, a simple yet effective method for translating streaming speech content. Given a usually long speech sequence, we develop an efficient monotonic segmentation module inside an encoder-decoder model to accumulate acoustic information incrementally and detect proper speech unit boundaries for the input in speech translation task. Experiments on multiple translation directions of the MuST-C dataset show that MoSST outperforms existing methods and achieves the best trade-off between translation quality (BLEU) and latency. Our code is available at https://github.com/dqqcasia/mosst.
Related papers
16
-
Peking University
-
Northeastern University
-
Peking University
-
The Chinese University of Hong Kong
-
Westlake University
-
Huazhong University of Science and Technology
-
Peking University
-
The Chinese University of Hong Kong
-
Pazhou Laboratory (Huangpu)