ADDING WORD DURATION INFORMATION TO BIGRAM LANGUAGE MODELS

Pr ( wi wi −1 , wi − 2 , L
ቤተ መጻሕፍቲ ባይዱ
)
The question is how to represent the candidate prosodic features at the word level. There is no obvious and correct way of doing this. So, in order to begin, let each prosodic feature be represented as a single scalar attribute of each word. For example, Pj could be the average pitch period, averaged over all voiced frames in word j; Tj could be the time interval between the end of word j-1 and the end of word j; and Ej could be the rms energy, averaged over word j. When these features are added to the model, the model becomes:
1. INTRODUCTION
Suprasegmental information is generally believed to play an important role in the recognition of speech by human listeners, and there has been a widespread desire and numerous attempts to incorporate this kind of information in ASR systems. (See, for example, R. Gadde, E. Shriberg, A. Stolcke, D. Hakkani-Tür, and G. Tür, "Prosody Modeling for Speech Recognition and Understanding," Presented at the Hub5 Conversational Speech Recognition (LVCSR) Workshop, Linthicum Heights, Maryland, USA, June 1999.F. Alleva, X. Huang, M. Hwang, and L. Jiang, "Can Continuous Speech Recognizers Handle Isolated Speech?," Speech Communication, vol. 26, pp. 183-189, June 1998.A. Stolcke, E. Shriberg, D. Hakkani-Tür, and G. Tür, "Modeling the prosody of hidden events for improved word recognition," Proceedings of the 6th European Conference on Speech Communication and Technology, Budapest, Hungary, September 1999.A. Stolcke, E. Shriberg, D. Hakkani-Tür, G. Tür, Z. Rivlin, and K. Sonmez, "Combining words and speech prosody for automatic topic segmentation," Proceedings of the DARPA Broadcast News Workshop, pp. 61--64, Herndon, VA, USA, 1999..) Unfortunately, these attempts have been only marginally successful, at best. A typical method of incorporating suprasegmental information has been to add suprasegmental measures to the segmental feature vector, and then to model the segmental and suprasegmental information jointly with a hidden Markov model (HMM). An alternative would be to incorporate the suprasegmental information at a higher level, namely in the language model. This paper attempts to motivate such an approach and discusses how one might go about doing this. The N-gram is currently the language model most commonly used in ASR. There are several reasons for this, not the least of which is that it provides generally superior performance. It is also relatively easy to implement and train. These characteristics also make the N-gram model suitable as a means of modeling suprasegmental information. And while the N-gram lacks explicit linguistic structure, much of its power lies in an ability to capture semantic information (by means of the strong correlation of meaning with specific word sequences). Therefore, since suprasegmental information is also strongly tied to meaning, the N-gram would seem to be a natural and promising way to capture it.
ADDING WORD DURATION INFORMATION TO BIGRAM LANGUAGE MODELS
George Doddington1,2, Aravind Ganapathiraju3, Joe Picone3, Yufeng Wu3
1
National Institute of Standards and Technology, 2 SRI International, 3 Mississippi State University
2. N-GRAM LANGUAGE MODELS INCORPORATING SUPRASEGMENTAL FEATURES
Candidate features include the usual prosodic features of pitch, timing and energy, as represented by the pitch period in milliseconds (P), the time interval in seconds (T), and the energy (E). A logarithmic transformation of these is suggested, in order to produce a more normal statistical distribution of feature values. The usual N-gram model is simply the probability of a word given the preceding N-1 words:
ABSTRACT
Suprasegmental information, while generally thought to play an important role in speech recognition by human listeners, has shown little promise in previous attempts to integrate into ASR systems. This paper outlines an approach that will successfully exploit suprasegmental information by modeling duration within the context of N-gram language modeling. Results show that up to half of the variance in wordlevel timing can be explained in terms of a simple bigram duration model. These experiments were conducted using the Switchboard corpus of conversational speech over the telephone. The paper also outlines a way of augmenting the N-gram language model with suprasegmental information.