Speech enhancement for nonstationary noise environment

SPEECH ENHANCEMENT BASED ON TEMPORAL PROCESSINGHynek Hermansky, Eric A. Wan, and Carlos AvendanoOregon Graduate Institute of Science & Technology Department of Electrical Engineering and Applied Physics P.O. Box 91000, Portland, OR 97291Finite Impulse Response (FIR) Wiener-like lters are applied to time trajectories of cubic-root compressed short-term power spectrum of noisy speech recorded over cellular telephone communications. Informal listenings indicate that the technique brings a noticeable improvement to the quality of processed noisy speech while not causing any signi cant degradation to clean speech. Alternative lter structures are being investigated as well as other potential applications in cellular channel compensation and narrowband to wideband speech mapping.ABSTRACTBoll (1979)), i.e., enhanced speech often contains musical noise and the technique typically degrades clean speech.2. CURRENT TECHNIQUE1. INTRODUCTIONThe need for enhancement of noisy speech in telecommunications increases with the spread of cellular telephony. Calls may originate from rather noisy environments such as moving cars or crowded public places. The corrupting noise is often relatively stationary, or at least changing rather slowly. This leads us to investigate the use of RelAtive SpecTrAl (RASTA) processing of speech (Hermansky and Morgan, et. al., 1991, 1994, Hirsch et. al., 1991), which was originally designed to alleviate e ects of convolutional and additive noise in automatic speech recognition (ASR). RASTA does this by band-pass ltering time trajectories of parametric representations of speech in a domain in which the disturbing noisy components are additive. Recently, RASTA was also applied to direct enhancement of noisy speech (Hermansky, Morgan, and Hirsch, 1993). In that case, RASTA ltering was applied to a magnitude (or cubic-root compressed power) of the short-term spectrum of speech while keeping the phase of the original noisy speech. However, applying rather aggressive xed ARMA RASTA lters (designed for suppression of convolutional distortions in ASR) yields results similar to spectral subtraction (see e.g.,This work was sponsored in part by USWEST Advanced Technologies under contract number 9069-111. Appeared in ICASSP 95, Detroit MI,1995RASTA involves nonlinear ltering of the trajectory of the short-term power spectrum (estimated using a 256 point FFT applied with a 64 point overlap Hamming window at a 8kHz sampling rate). Currently, we use linear lters applied to the cubic-root of the estimated power spectrum (Hermansky et. al. (1994)). In this study, we have substituted the ad hoc designed and xed RASTA lters by a bank of non-casual FIR Wiener-like lters. Each lter is designed to optimally map a time window of the noisy speech spectrum of a speci c frequency to a single estimate of the short-term magnitude spectrum of clean speech. For a 256 point FFT we require 129 unique lters.-10 Noisy Short-term Speech Analysis 1 0 FIR filter 10( ).a( )b. Magnitude . & Phase Extraction 129. . .. . .Phasea=2/3 b=3/2Figure 1: Block Diagram of System Let n( ) be the cubic-root estimate of the shorti term power spectrum of noisy speech in frequency bin ( = 1 to 129 and corresponds to an 8ms step). The output of each lter is the following:p k i i k p^i( ) =kj =?MM Xwi( ) n( ? ) ij p kSynthesisEstimate of Clean Speechj ;(1)jwhere ^i( ) is the estimate of the clean speech cubicroot spectrum. FIR lter coe cients i( ) are found such that ^i is the least squares estimate of the cleanp k w psignal i for each frequency bin . The current design has = 10 corresponding to 21 tap noncausal lters. As in the spectral subtraction technique, any negative spectral values after RASTA ltering are substituted by zeros and the noisy speech phase is used for the synthesis. The technique is illustrated in Fig. 1. Initial lters were designed on approximately 2 minutes of speech of one male talker recorded at 8 kHz sampling over a public analog cellular line from a relatively quiet laboratory. The speech was arti cially corrupted by additive noise recorded over a second cellular channel from a) car driving on a freeway with a window closed, b) car driving on a freeway with a window open, and c) busy shopping mall.p i M3. RESULTING OPTIMAL RASTA FILTERSFilter frequency responses are shown in Fig.2 (darker shades represent larger values). The lters are approximately symmetric (i.e., near zero phase). Filters for di erent frequency channels di er. The whole frequency band between 0 and 4 kHz appears to be subdivided into several regions, each characterized by its own RASTA processing.C A -60 dB 0 dBModulation frequency (Hz)Bsponse of typical lters in this region (slice A in Fig. 2) are shown in Fig. 3(A). Typically, lters in this frequency band have a band-pass character, emphasizing modulation frequencies around 6-8 Hz. Comparing to our original ad hoc designed RASTA lter (Hermansky, Morgan, and Hirsch, 1993), the low frequency bandstop is much milder, being only at most 10 dB down from the maximum. Filters for very low frequencies (0-100 Hz) are highgain lters with a rather at frequency response (slice C in Fig. 2 and Fig. 3(C)). Filters in the 150-250 Hz and the 2700-4000 Hz regions are low-gain low-pass lters (slice B in Fig.2) with at least 10 dB attenuation for modulation frequencies above 5 Hz. The low frequency pass-band of these lters are typically below the passband of the high-gain band-pass lters of Fig. 3(A). The frequency response of a 129 tap Wiener lter designed on noisy data is shown by the solid line in Fig. 4. For comparison, the center taps of the RASTA lters designed on magnitude spectrum (i.e. a=b=1 in Fig.1) are shown in the gure by the dashed line. Informal comparisons between the quality of speech processed by the time-domain Wiener lter and the RASTA lters indicates some advantage of the additional lter taps and compressed spectrum used with the new RASTA lters.10| | |62Magnitude (dB)RASTA center taps Wiener filter0 -10 -20 -30 -40| |00 0.5 1 1.5 2 2.5 3 3.5 4Center frequency of analysis channel (kHz)Figure 2: Frequency Response of RASTA Filters|01234Frequency (kHz)00(C)−5−5Figure 4: Wiener lter response and RASTA centertaps.Gain (dB)−10 −10 −15 −15(A) (B)−20 −20 −25 −25 −30 −30 −35 −35 0 0 20 40 20 60 80 40 100 1204. EXPERIMENTS60Modulation Frequency (Hz)Figure 3: Freq. Response of Di erent Filters. The highest gain RASTA lters are applied in the frequency bands of about 300-2300 Hz. Frequency re-The RASTA processing using lters described above has been tested on several recordings of actual noisy cellular communications (out of training set). We have informally observed that the processing brings a noticeable improvement in subjective quality of the recordings. Noise seems to be less disturbing while the quality of speech is not impaired. Further, we have also observed that the processing does not seem to impair the quality of clean speech recordings. A visible reduc-tion of the noise after processing is apparent from the spectrograms shown in Fig. 5.4have been taken towards the investigation of this problem.5.3.1. Cellular Channel Equalization0 0 4sec6.40 0sec6.4Figure 5: Spectrogram showing noisy speech followed bythe same noisy segment after processing.The original 16kHz sampled speech is downsampled to 8kHz and used as the desired response in our training scheme. The input data corresponds to the same speech segments recorded over the cellular channel. Additional white Gaussian noise is added to the input in order to avoid excessive gains in the band edges where cellular speech is not present. Several tests done on clean cellular communications have resulted in a noticeable improvement; however, formal comparisons with other techniques have yet to be done.8kHzkHzA database that includes the e ects of actual ambient noise on cellular speech communications is being collected. A training set of 330 sentences taken from the TIMIT database (complete dr8 set) is played through a portable PC over a public cellular line from di erent locations. A test set generated with 123 randomly chosen TIMIT sentences (dr1-dr7 only) is recorded under the same conditions for later evaluation of our algorithms. Adjacent channel information has been used to train each channel resulting in a set of (Multi-Input-SingleOutput) MISO lters. Initial results with 6 adjacent channels (3 at each side of the frequency of interest) show an improvement in the reduction of perceived noise as well as reduction of residual error at each frequency bin. However, the additional improvement obtained with this structure is noticeable only within the training set (original 2 minute source), while the processing generalizes poorly over di erent speakers and noise types. The e ect of training on large data sets and the e ect of di erent architectures need to be investigated before any conclusions can be drawn. Availability of the original 16kHz sampled speech data has raised the question as to the possibility of recovering a good quality wideband signal from a degraded, 8khz sampled, cellular speech recording. Two steps5. WORK IN PROGRESS 5.1. Data CollectionkHz4 0 0 8 1.8kHz4 0 0 8 1.8kHz4 0 0 1.85.2. Adjacent Channelssecondstrogram of downsampled signal and c) recovered spectrogram. 5.3.2. Wideband Speech ReconstructionFigure 6: a) Spectrogram of the original speech, b) spec-5.3. Cellular to Wideband Speech RecoveryAn approximate wideband from narrowband speech map has been produced by extending the adjacent channel concept. Now the 129 trajectories derived from the 8kHz sampled speech analysis are fed to MISO lters whose outputs correspond to frequencies from 4kHz to 8kHz (i.e., bins 130 to 257) and the signal reconstruction is done at twice the original rate. Training was performed with 9 TIMIT sentences (same speaker) with a time window of 40 ms ( = 2 in (1)). In Fig.6, we show an out of training set original wideband speech spectrogram, the downsampled version of it, and the recovered spectrogram. Within a given talker the high frequency recovery appears to be viable. However, for time signal reconstruction an estimate of the phase ofMthe recovered high frequencies needs to be performed. This is discussed in the following section. Modi cation of the short-term magnitude spectrum of a signal and synthesis with original phase do not, in general, yield a time signal with the desired magnitude spectrum (see e.g., Allen, J.(1977)). To avoid distortion in the synthesized signal, especially when the power spectrum has been severely modi ed (see subsection 5.6) , we are investigating an iterative algorithm (see e.g., Gri n and Lim (1984)) in which the mean squared error between the desired spectrum and the spectrum produced by the synthesized signal is minimized. In noise reduction applications, we have used the noisy signal's phase to perform the initial step in the reconstruction. In the case of wideband speech mapping, a linear map from the available low frequency phase components to the higher frequencies is taken as a rst approximation.5.4. Phase Reconstructionshort-term spectrum of speech. While no formal subjective perceptual tests were carried out, our initial experience indicates a possible advantage of the proposed technique over conventional speech enhancement techniques. Future work will investigate generalization of the technique for additional noise sources and larger training sets. In addition, more formal perceptual evaluations will be undertaken. Related work outlined in section 5 will also be pursued.7. ACKNOWLEDGEMENTSWe would like to thank Mahesh K. Kumashikar for the neural networks training, and CONACYT (Mexico) for partial support of the third author.8. REFERENCES1] J.B. Allen: Short term spectral analysis, synthesis, and modi cation by discrete Fourier transform, IEEE ASSP-25, pp. 235-238, Jun. 1977. 2] S.F. Boll: Suppression of acoustic noise in speech using spectral subtraction, IEEE ASSP-27, pp. 113-120, Apr. 1979. 3] V. Ramamoorthy, N.S. Jayant, R.V. Cox and M.M. Sondhi: Enhancement of ADPCM speech coding with backward-adaptive algorithms for post ltering and noise feedback, IEEE J. Selec. Areas in Comm. Vol. 6, No.2, pp. 364-382, Feb. 1988. 4] H. Hermansky: Perceptual predictive (PLP) analysis of speech, J. Acoust. Soc. Am. 87 pp. 17381752 Apr. 1990. 5] H. G. Hirsch, P. Meyer, and H. Ruehl: Improved speech recognition using high-pass ltering of subband envelopes, Proc. EUROSPEECH '91, pp. 413-416, Genova, 1991 6] H. Hermansky, N. Morgan, A. Bayya, and P. Kohn: Compensation for the e ect of the communication channel in auditory-like analysis of speech (RASTA-PLP) Proc. EUROSPEECH '91, pp. 1367-1370, Genova, 1991. 7] H. Hermansky, N. Morgan, and H.G. Hirsch: Recognition of speech in additive and convolutional noise based on RASTA spectral processing, Proc. ICASSP-93, pp. II-83, II-86, 1993. 8] H. Hermansky, E. Wan, and C. Avendano: Noise suppression in cellular communications, Proc. IVTTA 94, pp. 85-88, Kyoto 1994.5.5. Postprocessing with Envelope EnhancementFirst proposed for enhancement of ADPCM speech coding (see Ramamoorthy, et. al., (1988)), a pre ltering technique for spectral envelope enhancement has been applied in our system. Spectral domain ltering is done by designing a lter at each 8 ms step with an untilted and smoothed 8th order all-pole model spectrum of the particular frame, giving the e ect of enhancing frequencies around formants and suppression elsewhere. Best results have been achieved by pre ltering the compressed power spectrum prior to applying RASTA lters to its trajectories.Non-linear RASTA lters have also been implemented with 3 layer arti cial neural networks. Preliminary results on small training sets show a more aggressive ltering and greater noise reduction. However, musicallike residual noise as well as greater degradation of clean speech is also observed. The iterative synthesis method described above has been used with some success to alleviate the phase distortion introduced by the large amount of spectral modi cation. Current e orts are focused on reducing the adverse e ect introduced by the networks.5.6. Non-linear Filters6. CONCLUSIONSWe have presented a new technique for enhancement of noisy speech in cellular communications, which utilizes Wiener-like RASTA lters on cubic-root compressed。

Y. Ephraim is with the Department of Electrical and Computer Engineering, George Mason University, Fairfax, VA 22030. Email: yephraim@ I. Cohen is with the Department of Electrical Engineering, Technion, Israel Institute of Technology, Haifa 32000, ISRAEL. Email: icohen@ee.technion.ac.il
1
1
enhancement. Thus, no reference noise source is assumed available. This problem is of great interest, and has attracted significant research effort for over fifty years. A successful algorithm may be useful as a preprocessor for speech coding and speech recognition of noisy signals. The perception of a speech signal is usually measured in terms of its quality and intelligibility. The quality is a subjective measure which reflects on individual preferences of listeners. Intelligibility is an objective measure which predicts the percentage of words that can be correctly identified by listeners. The two measures are not correlated. In fact, it is well known that intelligibility can be improved if one is willing to sacrifice quality. This can be achieved, for example, by emphasizing high frequencies of the noisy signal [35]. It is also well known that improving the quality of the noisy signal does not necessarily elevate its intelligibility. On the contrary, quality improvement is usually associated with loss of intelligibility relative to that of the noisy signal. This is due to the distortion that the clean signal undergoes in the process of suppressing the input noise. From a pure information theoretic point of view, such loss in “information” is predicted by the data processing theorem [10]. Loosely speaking, this theorem states that one can never learn from the enhanced signal, more than he can learn from the noisy signal, about the clean signal. A speech enhancement system must perform well for all speech signals. Thus, from the speech enhancement system point of view, its input is a random process whose sample functions are randomly selected by the user. The noise is naturally a random process. Hence, the speech enhancement problem is a statistical estimation problem of one random process from the sum of that process and the noise. Estimation theory requires statistical models for the signal and noise, and a distortion measure which quantifies the similarity of the clean signal and its estimated version. These two essential ingredients of estimation theory are not explicitly available for speech signals. The difficulties are with the lack of a precise model for the speech signal and a perceptually meaningful distortion measure. In addition, speech signals are not strictly stationary. Hence, adaptive estimation techniques, which do not require explicit statistical model for the signal, often fail to track the changes in the underlying statistics of the signal. In this paper we survey some of the main ideas in the area of speech enhancement from a single microphone. We begin in Section 2 by describing some of the most promising statistical models and distortion measures which have been used in designing speech enhancement systems. In Section 3 we present a detailed design example for a speech enhancement system which is based on minimum mean square error estimation of the speech spectral magnitude. This approach integrates several key ideas from Section 2, and has

一种改进的LSA语音增强算法王金明;周坤;尹海明;徐志军【摘要】An improved ln-spectral amplitude (LSA)speech enhancement method was proposed for robust speaker recognition.The LSA feature preserves the naturalness of speech and hence is more suitable for speaker recognition.However,the traditional minimum mean square error-ln-spectral amplitude(MMSE-LSA)method cannot adj ust the gain factor according to input signal to noise ratio (SNR),thus often re-sulting in performance degradation when SNR is low.A recursive averaging algorithm was proposed for es-timating SNR of speech frame efficiently,thereby affording adaptive gain control of the speech enhance-ment system.The improved LSA speech enhancement method can output high fidelity speech signal and minimize the impact on the naturalness of speech signal.Experimental results show that under 5 dB noise, SNR of the new method's output is 1 .3 6 dB better than that of the traditional MMSE-LSA,and perceptual evaluation of speech quality(PESQ)score rises 0.08 on average.The method is also applied to speaker rec-ognition system and the detection cost score is about 3% lower than that of MMSE-LSA.%针对说话人识别的噪声鲁棒性问题,在对数谱最小均方差误差估计算法基础上,采用改进的最小值控制递归平均算法对语音帧信噪比进行估计,通过对前一帧的短时功率谱进行2次平滑和前向多帧最小值搜索,结合语音存在概率估计出当前帧的信噪比,并根据信噪比自适应调整增益因子的大小,对噪声进行消除.构建了一种改进的LSA语音增强方法,使用该方法可以使增强后的语音保持较高的自然度.实验结果表明,与MMSE-LSA算法比较,改进的LSA算法具有更好的语音增强效果,在5 dB各类噪声环境下,其平均信噪比较MMSE-LSA算法提高1．36 dB,主观语音质量评估平均提高8％.将该方法用于说话人识别系统,其检测代价较采用MMSE-LSA算法的系统平均降低3％.【期刊名称】《解放军理工大学学报（自然科学版）》【年(卷),期】2015(000)004【总页数】6页(P310-315)【关键词】语音增强;短时对数谱;最小均方误差;信噪比;说话人识别【作者】王金明;周坤;尹海明;徐志军【作者单位】解放军理工大学通信工程学院，江苏南京 210007;解放军理工大学通信工程学院，江苏南京 210007;解放军理工大学通信工程学院，江苏南京210007;解放军理工大学通信工程学院，江苏南京 210007【正文语种】中文【中图分类】TN912.34目前，说话人识别的实用化面临的最主要问题是识别的鲁棒性问题，尤其是抗噪声鲁棒性。

一种适用于双微阵列的语音增强算法毛维;曾庆宁;龙超【摘要】Considering the limitations of traditional single channel speech enhancement algorithm for noise sup -pression,a dual-mini microphone array consisting of two microphones array is employed.The noisy speech is pro-cessed by the temporal and spatial characteristics of the spatial structure of the array,and a speech enhancement al-gorithm based on dual-mini microphone array is proposed.The speech enhancement algorithm uses the logarithmic minimum mean square error(LogMMSE)to enhance the signal-to-noise(SNR)of the noisy speech signals collect-ed by each channel.Then,the signal in the target source direction is obtained and retained by using the wideband minimum variance distortionless response(MVDR)in the frequency domain,which also suppresses the signal inter-ference in other directions.Finally,an improved intelligibility and improved minimum controlled recursive averagingcombination(IMCRA)Wiener filter for noise estimation to remove residual noise can be used to improve the speech quality.Simulation results show that compared with the traditional single channel speech enhancement algo -rithm,the proposed algorithm has good performance in noise suppression.%考虑到传统单通道语音增强算法对噪声抑制的局限性,采用由两个微型麦克风阵列组成的双微阵列;利用该阵列空间结构的时空域特性对含噪语音进行处理,提出了一种适用于双微阵列的语音增强算法;该增强算法是将各通道采集到的带噪语音信号先使用对数最小均方误差(logarithmic minimum mean squareerror,LogMMSE)提升其信噪比;然后利用频域宽带最小方差无畸变响应(MVDR)通过对目标声源信号的获取,保留目标声源方向的信号并抑制其他方向的信号干扰;最后通过一个改进可懂度结合改进最小控制递归平均(improved minimum controlled recursive average algorithm,IMCRA)噪声估计的维纳滤波器来去除噪声残留提升语音质量.仿真实验结果表明,相比传统的单通道语音增强算法,该算法具有良好的噪声抑制性能.【期刊名称】《科学技术与工程》【年(卷),期】2018(018)010【总页数】5页(P245-249)【关键词】双微阵列;语音增强;对数最小均方误差;最小方差无畸变响应;改进最小控制递归平均;维纳滤波【作者】毛维;曾庆宁;龙超【作者单位】桂林电子科技大学信息与通信学院,桂林541004;桂林电子科技大学信息与通信学院,桂林541004;桂林电子科技大学信息与通信学院,桂林541004【正文语种】中文【中图分类】TP391.42近年来，人工智能技术发展迅速；特别是在如计算机视觉、大数据分析、无人驾驶、自动翻译、垃圾邮件过滤等诸多领域已取得了革命性的突破进展；而语音作为人工智能交互的接口，对其性能的改善和优化起了重要的作用。