We  we abandoned the lapel because they sort of were not too not too hot , not too cold , they were you know , they were  , far enough away that you got more background noise , and  and so forth phd a:  - huh . professor c: but they weren't so close that they got quite the you know , the really good No , th phd a: OK . They were not so far away that they were really good representative distant mikes , phd a:  - huh . professor c: but on the other hand they were not so close that they got rid of all the interference . On the other hand if you only had to have one mike in some ways you could argue the lapel was a good choice , precisely because it 's in the middle . professor c: There 's  , some kinds of junk that you get with these things that you don't get with the lapel  , little mouth clicks and breaths and so forth are worse with these than with the lapel , but given the choice we there seemed to be very strong opinions for  , getting rid of lapels . phd a: The mike number is professor c: So , phd f: your mike number 's written on the back of that unit there . phd f: It - it 's one less than what 's written on the back of your phd a: OK . professor c: And you should do a lot of talking so we get a lot more of your pronunciations . phd f: So what we usually do is  , we typically will have our meetings professor c: Yeah . One thing phd f: Sunil 's here for the summer ? professor c: Sunil 's here for the summer , right .  , so , one thing is to talk about a kick off meeting maybe  , and then just  , I guess  , progress reports individually , and then  , plans for where we go between now and then , pretty much . phd f: I could say a few words about  , some of the  , compute stuff that 's happening around here , so that people in the group know . professor c: Yeah ? phd f: We  So we just put in an order for about twelve new machines , to use as sort of a compute farm . And  , we ordered  , SUN - Blade - one - hundreds , and  , I 'm not sure exactly how long it 'll take for those to come in , but , in addition , we 're running So the plan for using these is , we 're running P - make and Customs here and Andreas has sort of gotten that all  , fixed up and up to speed . And he 's got a number of little utilities that make it very easy to  , run things using P - make and Customs . The simplest thing And I can send an email around or , maybe I should do an FAQ on the web site about it or something .  , professor c: How about an email that points to the FAQ , phd f: there 's a c professor c: you know what I 'm saying ? phd f: Yeah , yeah . And , if you say that and then some job that you want to execute , it will find the fastest currently available machine , and export your job to that machine , and  and run it there and it 'll duplicate your environment . So you can say " run dash command L S " , and , it 'll actually export that LS command to some machine in the institute , and  , do an LS on your current directory . So , substitute LS for whatever command you want to run , and  And that 's a simple way to get started using using this . And , so , soon , when we get all the new machines up , e then we 'll have lots more compute to use . Now th one of the nice things is that  , each machine that 's part of the P - make and Customs network has attributes associated with it .  , attributes like how much memory the machine has , what its speed is , what its operating system , and when you use something like " run command " , you can specify those attributes for your program . For example if you only want your thing to run under Linux , you can give it the Linux attribute , and then it will find the fastest available Linux machine and run it on that . You can control where your jobs go , to a certain extent , all the way down to an individual machine . If there 's already a job running , on some machine that you 're trying to select , your job will get queued up , and then when that resource , that machine becomes available , your job will get exported there . So , there 's a lot of nice features to it and it kinda helps to balance the load of the machines and  , right now Andreas and I have been the main ones using it and we 're  . professor c: So as I understand , you know , he 's using all the machines and you 're using all the machines , phd f: So . Yeah , you know , I I sort of got started using the recognizer just recently and  ,  I fired off a training job , and then I fired off a recognition job and I get this email about midnight from Andreas saying , "  , are you running two trainings simultaneously s my m my jobs are not getting run . grad g: I have I have a question about the  , parallelization ? phd f:   grad g: So , let 's say I have like , a thousand little little jobs to do ? phd f:   grad g: how do I do it with " run command " ?  do phd f: You could write a script  , which called run command on each sub - job grad g:  - huh . A thousand times ? phd f: right ? But you probably wanna be careful with that grad g: OK .  , so , you know , you should you should probably not run more than , say ten jobs yourself at any one time , just because then it would keep other people grad g: Oh , too much file transfer and stuff . phd f: Well it 's not that so much as that , you know , e with if everybody ran fifty jobs at once then it would just bring everything to a halt and , you know , people 's jobs would get delayed , so it 's sort of a sharing thing . phd f: so you should try to limit it to somet sometim some number around ten jobs at a time . So if you had a script for example that had a thousand things it needed to run , you 'd somehow need to put some logic in there if you were gonna use " run command " , to only have ten of those going at a time .  , professor c: I remember I I forget whether it was when the Rutgers or or Hopkins workshop , I remember one of the workshops I was at there were everybody was real excited cuz they got twenty - five machines and there was some kind of P - make like thing that sit sent things out . professor c: So all twenty - five people were sending things to all twenty - five machines phd f:   professor c: and and things were a lot less efficient than if you 'd just use your own machine . phd f: but  , you can also If you have that level of parallelization  , and you don't wanna have to worry about writing the logic in in a Perl script to take care of that , you can use  , P - make grad g: Just do P - make . phd f: and and you basically write a Make file that  , you know your final job depends on these one thousand things , grad g: s   phd f: and when you run P - make , on your Make file , you can give it the dash capital J and and then a number , grad g:   If you " Run command " , that I mentioned before , is doesn't know about other things that you might be running . phd f: So , it would be possible to run a hundred run jobs at once , phd d: Right . But if you use P - make , then , it knows about all the jobs that it has to run phd d:   professor c: So " run command " doesn't use P - make , or ? phd f: It uses " export " underlyingly . But , if you i It 's meant to be run one job at a time ? So you could fire off a thousand of those , and it doesn't know any one of those doesn't know about the other ones that are running . professor c: So why would one use that rather than P - make ? phd f: Well , if you have ,  Like , for example ,  if you didn't wanna write a P - make script and you just had a ,  an HTK training job that you know is gonna take  , six hours to run , and somebody 's using , the machine you typically use , you can say " run command " and your HTK thing and it 'll find another machine , the fastest currently available machine and and run your job there . professor c: Now , does it have the same sort of behavior as P - make , which is that , you know , if you run something on somebody 's machine and they come in and hit a key then it phd f: Yes . And if you specify that , in in one of your attribute lines , then it 'll go to a machine which your job won't be evicted from . phd f: But , the machines that don't have that attribute , if a job gets fired up on that , which could be somebody 's desktop machine , and and they were at lunch , professor c:   phd f: they come back from lunch and they start typing on the console , then your machine will get evicted your job will get evicted from their machine and be restarted on another machine . So which can cause you to lose time , right ? If you had a two hour job , and it got halfway through and then somebody came back to their machine and it got evicted . If you don't want your job to run on a machine where it could be evicted , then you give it the minus the attribute , you know , " no evict " , and it 'll pick a machine that it can't be evicted from . professor c: what what about I remember always used to be an issue , maybe it 's not anymore , that if you if something required if your machine required somebody hitting a key in order to evict things that are on it so you could work , but if you were logged into it from home ? phd f:   professor c: and you weren't hitting any keys ? cuz you were , home ? phd f: Yeah , I I 'm not sure how that works . I don't know whether it monitors the keyboard or actually looks at the console TTY , so maybe if you echoed something to the you know , dev dev console or something . phd f:  ? professor c:  you sort of you 're at home and you 're trying to log in , and it takes forever to even log you in , and you probably go , " screw this " , phd f: Yeah , yeah . phd a: I need a little orientation about this environment and  scr s how to run some jobs here because I never d did anything so far with this X emissions phd f: OK . Yeah , and and also  , Stephane 's a a really good resource for that if you can't find me . Well , why don't we  , Sunil since you 're haven't haven't been at one of these yet , why don't yo you tell us what 's what 's up with you ? Wh - what you 've been up to , hopefully . phd a: yeah , after the submission the what I 've been working on mainly was to take take other s submissions and then over their system , what they submitted , because we didn't have any speech enhancement system in in ours . And then I found that  , if if I combine it with LDA , it gives @ @ improvement over theirs . I just plug in I just take the cepstral coefficients coming from their system and then plug in LDA on top of that . phd a: What I did was I took the LDA filter 's design using clean speech , mainly because the speech is already cleaned up after the enhancement so , instead of using this , narrow narrow band LDA filter that we submitted  , I got new filters . And  , so then after after that I I added  , on - line normalization also on top of that . And that there there also I n I found that I have to make some changes to their time constant that I used because th it has a a mean and variance update time constant and which is not suitable for the enhanced speech , and whatever we try it on with proposal - one . But  , I didn't I didn't play with that time constant a lot , I just t g I just found that I have to reduce the value  , I have to increase the time constant , or reduce the value of the update value . And  , the other other thing what I tried was , I just  , took the baseline and then ran it with the endpoint inf  th information , just the Aurora baseline , to see that how much the baseline itself improves by just supplying the information of the  the w speech and nonspeech . And  , I found that the baseline itself improves by twenty - two percent by just giving the wuh . professor c: can you back up a second , I I I missed something , I guess my mind wandered . Ad - ad When you added the on - line normalization and so forth ,  things got better again ? phd a: Yeah . phd a: With the different time constant I found that  , I didn't get an improvement over not using on - line normalization , professor c: Oh . phd a: But I didn't play it with play play quite a bit to make it better than . The the other thing what I tried was the adding the  , endpoint information to the baseline and that itself gives like twenty - two percent because the the second the new phase is going to be with the endpointed speech . And just to get a feel of how much the baseline itself is going to change by adding this endpoint information , I just , use professor c:  . phd f: So people won't even have to worry about , doing speech - nonspeech then . professor c: G I guess the issue is that people do that anyway , phd f: I see . professor c: and they wanted to see , given that you 're doing that , what what are the best features that you should use . professor c: But but it might be  In some ways it might be better t to rather than giving the endpoints , to have a standard that everybody uses and then interacts with . phd f: So , are people supposed to assume that there is  Are are people not supposed to use any speech outside of those endpoints ? phd a:  phd f: Or can you then use speech outside of it for estimating background noise and things ? phd a: No . Like y you will you will You 'll be given the information about the beginning and the end of speech but the whole speech is available to you . professor c: So it should make the spectral subtraction style things work even better , phd a: Yeah . I found that in s one of the SpeechDat - Car cases , that like , the Spanish one improves by just fifty percent by just putting the endpoint . phd a: Yeah , so professor c: But but phd a: so that is when  , the the qualification criteria was reduced from fifty percent to something like twenty - five percent for well - matched . And  , Yeah , I guess after that , I just went home f I just had a vacation fo for four weeks . phd a: Ye Yeah , and I I came back and I started working on  , some other speech enhancement algorithm .  , so I from the submission what I found that people have tried spectral subtraction and Wiener filtering . phd a: so just to just to fill the space with some f few more speech enhancement algorithms to see whether it improves a lot , I I 've been working on this  , signal subspace approach for speech enhancement where you take the noisy signal and then decomposing the signal s and the noise subspace and then try to estimate the clean speech from the signal plus noise subspace . phd a: So , I 've been actually running some s So far I 've been trying it only on Matlab . phd a: and then I 'll p port it to C and I 'll update it with the repository once I find it it giving any some positive result . professor c: S So you s you So you said one thing I want to jump on for a second . So so now you 're you 're getting tuned into the repository thing that he has here phd a: Yeah . Cuz you did some stuff that you talked about last week , I guess ? phd d:   professor c: where you were also combining something both of you I guess were both combining something from the  , French Telecom system with the u  phd d: Right . professor c: I I don't know whether it was system one or system two , or ? phd d:   phd d: we The main thing that we did is just to take the spectral subtraction from the France Telecom , which provide us some speech samples that are  , with noise removed . So then , one distinction is that  , you were taking the actual France Telecom features and then applying something to phd a: no there is a slight different .   , which are extracted at the handset because they had another back - end blind equalization professor c: Yeah . professor c: What I meant was you had something like cepstra or something , right ? phd a: Yeah , yeah , yeah , yeah . But I guess it 's the s exactly the same thing because on the heads  , handset they just applied this Wiener filter and then compute cepstral features , phd a: Yeah , the cepstral f The difference is like There may be a slight difference in the way phd d: right ? or ? phd a: because they use exactly the baseline system for converting the cepstrum once you have the speech . But  , I th phd d: Yeah , well I think we should  , have a table with all the result because I don't know I  , I don't exactly know what are your results ? But , phd a: OK . Yeah , but so we did this , and another difference I guess is that we just applied  , proposal - one system after this without well , with our modification to reduce the delay of the the LDA filters , phd a:  - huh . phd d: and phd b: And the filter phd d: Well there are slight modifications , but it was the full proposal - one . In your case , if you tried just putting LDA , then maybe on - line normalization ? phd a: Only LDA . So we just tried directly to to just , keep the system as it was and , when we plug the spectral subtraction it improves  , signif significantly .  , but , what seems clear also is that we have to retune the time constants of the on - line normalization . phd d: Because if we keep the value that was submitted  , it doesn't help at all . But , you can still find some kind of optimum somewhere , and we don't know where exactly phd a: Yeah . professor c: So it sounds like you should look at some tables of results or something phd d: Right . professor c: and see where i where the where they were different and what we can learn from it . phd d: because we change it the system to have phd a: Oh yeah ,  the the new LDA filters . There are other things that we finally were shown to improve also like , the sixty - four hertz cut - off . phd d: And , right now if we look at the results , it 's , always better than it seems always better than France Telecom for mismatch and high - mismatch . But , the problem is that it 's not significant , but if you put this in the , mmm , spreadsheet , it 's still worse . I don't think it 's importa important because when they will change their metric , mainly because of  , when you p you plug the  , frame dropping in the baseline system , it will improve a lot HM , and  , phd a: Yeah . phd a: Because the your improvement on HM and  will also go down significantly in the spreadsheet so . phd a:  the well - matched may be the one which is least affected by adding the endpoint information . But they d the everything  is like , but there that 's how they reduce why they reduce the qualification to twenty - five percent or some something on . professor c: But are they changing the weighting ? phd a: no , I guess they are going ahead with the same weighting . professor c: I guess I I haven't been part of the discussion , so , it seems to me that the well - matched condition is gonna be unusual , phd a: Usual . professor c: Because , you don't actually have good matches ordinarily for what any @ @ particular person 's car is like , or phd a: Mmm . phd a: Yeah , but actually the well well the well - matched  ,  the the well - matched condition is not like , the one in TI - digits where  , you have all the training , conditions exactly like replicated in the testing condition also . The well - matched has also some some mismatch in that which is other than the professor c: The well wa matched has mismatch ? phd a: has has also some slight mismatches , unlike the TI - digits where it 's like prefectly matched phd f: Perfect to match . So remind me of what well - matched meant ? phd a: The the well - matched is like professor c: You 've told me many times . phd a: the the well - matched is defined like it 's seventy percent of the whole database is used for training and thirty percent for testing . phd a: It 's it 's phd d: Because it phd a: OK , it 's professor c: Yeah . So , yeah , unless they deliberately chose it to be different , which they didn't because they want it to be well - matched , it is pretty much You know , so it 's so it 's sort of saying if you phd f: It 's it 's not guaranteed though . phd a: the main mismatch is coming from the amount of noise and the silence frames and all those present in the database actually . professor c: Again , if you have enough if you have enough phd a: No yeah , yeah . professor c: So it 's sort of i i it 's sort of saying OK , so you much as you train your dictation machine for talking into your computer , you you have a car , and so you drive it around a bunch and and record noise conditions , or something , and then I don't think that 's very realistic ,  I th phd a:   professor c: I I you know , so I I I you know , I guess they 're saying that if you were a company that was selling the stuff commercially , that you would have a bunch of people driving around in a bunch of cars , and and you would have something that was roughly similar and maybe that 's the argument , but I 'm not sure I buy it , so . We are playing we are also playing , trying to put other spectral subtraction mmm , in the code .  , it would be a very simple spectral subtraction , on the  , mel energies which I already tested but without the  frame dropping actually , and I think it 's important to have frame dropping if you use spectral subtraction . phd f: Is it is spectral subtraction typically done on the after the mel , scaling or is it done on the FFT bins ? phd d: phd f: Does it matter , or ? phd d: I d I don't know . So - some of the proposal , we 're doing this on the bin on the FFT bins , phd f:  . You can do both , but I cannot tell you what 's which one might be better or I phd f:  . phd a: I guess if you want to reconstruct the speech , it may be a good idea to do it on FFT bins .  it may not be very different if you do it on mel warped or whether you do it on FFT . phd d: Well , it gives something different , but I don't know what are the , pros and cons of both . phd a: The other thing is like when you 're putting in a speech enhancement technique , is it like one stage speech enhancement ? Because everybody seems to have a mod two stages of speech enhancement in all the proposals , which is really giving them some improvement . phd a: And So , there 's something that is good about doing it  , to cleaning it up once more . phd a: so we can phd d: So maybe in my implementation I should also try to inspire me from this kind of thing phd a: Yeah . That 's what professor c: Well , the other thing would be to combine what you 're doing . professor c:  maybe one or one or the other of the things that you 're doing would benefit from the other happening first . So , professor c: Right , so he 's doing a signal subspace thing , maybe it would work better if you 'd already done some simple spectral subtraction , or maybe vi maybe the other way around , phd d: Yeah ,   professor c: you know ? phd a: So I 've been thinking about combining the Wiener filtering with signal subspace , phd d:   phd a:  just to see all some some such permutation combination to see whether it really helps or not . professor c: How is it I I guess I 'm ignorant about this , how does  , since Wiener filter also assumes that you 're that you 're adding together the two signals , how is how is that differ from signal subspace ? phd a: The signal subspace ? The professor c: Yeah . phd a: So , the the different the c the the advantage of combining two things is mainly coming from the signal subspace approach doesn't work very well if the SNR is very bad . So essentially you could do simple spectral subtraction , followed by a KL transform , followed by a phd a: Wiener filtering . Yeah , in general , you don't that 's right you don't wanna othorg orthogonalize if the things are noisy .  , that was something that  , Herve and I were talking about with  , the multi - band stuff , that if you 're converting things to from  , bands , groups of bands into cepstral coef you know , local sort of local cepstral coefficients that it 's not that great to do it if it 's noisy . phd a: So that that 's one reason maybe we could combine s some something to improve SNR a little bit , first stage , professor c: Yeah . phd d: What was your point about about colored noise there ? phd a: Oh , the colored noise  phd d: Yeah . phd a: the colored noise the the v the signal subspace approach has  , it it actually depends on inverting the matrices . phd a:  it has a it 's It doesn't behave very well if it is not positive definite ak It works very well with white noise because we know for sure that it has a positive definite . phd a: So the way they get around is like they do an inverse filtering , first of the colo colored noise professor c: Yeah . phd a: and then finally when you reconstruct the speech back , you do this filtering again .  if you sort of you do the s spectral subtraction , that also gets rid phd a: Yeah . professor c: and then you then then add a little bit l noise noise addition  , that sort of what J JRASTA does , in a way . professor c: If you look at what JRASTA doing essentially i i it 's equivalent to sort of adding a little adding a little noise , phd a: Huh ?  - huh . And maybe we well we find some people so that  , agree to maybe work with us , and they have implementation of VTS techniques so it 's  , Vector Taylor Series that are used to mmm ,  f to model the transformation between clean cepstra and noisy cepstra . Well , if you take the standard model of channel plus noise , it 's it 's a nonlinear eh  , transformation in the cepstral domain . phd d: And  , there is a way to approximate this using  , first - order or second - order Taylor Series and it can be used for  , getting rid of the noise and the channel effect . professor c: Who is doing this ? phd d:  w working in the cepstral domain ? So there is one guy in Grenada , phd b: Yeah , in Grenada one of my friend . professor c: Who 's the guy in Grenada ? phd d: phd b: Jose Carlos Segura . professor c: Yeah , so at any rate , you 're looking general , standing back from it , looking at ways to combine one form or another of  , noise removal , with with these other things we have , phd d:   But for sure there 's required to that requires to re - check everything else , and re - optimize the other things professor c: Oh yeah .  , professor c: Well one of the seems like one of the things to go through next week when Hari 's here , phd d: I professor c: cuz Hari 'll have his own ideas too or I guess not next week , phd d:  - huh . professor c: week and a half , will be sort of go through these alternatives , what we 've seen so far , and come up with some game plans .  one would be , you look at a few things very quickly , you pick on something that looks like it 's promising and then everybody works really hard on the same different aspects of the same thing . Another thing would be to have t to to pick two pol two plausible things , and and you know , have t sort of two working things for a while until we figure out what 's better , phd d:   professor c: and then , you know , but , w  , he 'll have some ideas on that too . phd a: The other thing is to ,  Most of the speech enhancement techniques have reported results on small vocabulary tasks . But we we going to address this Wall Street Journal in our next stage , which is also going to be a noisy task so s very few people have reported something on using some continuous speech at all . So , there are some  , I was looking at some literature on speech enhancement applied to large vocabulary tasks and spectral subtraction doesn't seems to be the thing to do for large vocabulary tasks . And it 's Always people have shown improvement with Wiener filtering and maybe subspace approach over spectral subtraction everywhere . But if we if we have to use simple spectral subtraction , we may have to do some optimization to make it work @ @ . professor c: So they 're making there Somebody 's generating Wall Street Journal with additive artificially added noise or something ? phd a: Yeah , yeah . professor c: Sort of a sort of like what they did with TI - digits , and ? phd a: Yeah . professor c: And then they 're they 're  , generating HTK scripts to phd a: Yeah . There are they have there is no I don't know if they are converging on HTK or are using some Mississippi State , professor c: Mis - Mississippi State maybe , phd a: yeah . professor c: well we 've Yeah , it 's true for the additive noise , y artificially added noise we 've always used small vocabulary too . But for n there 's been noisy speech this larv large vocabulary that we 've worked with in Broadcast News . professor c: and some of the focus conditions were noisy and and phd a: It had additive n professor c: But we but we didn't do spectral subtraction . We were doing our funny stuff , right ? We were doing multi multi  , multi - stream and and so forth . professor c: and and  , that we have  , for the  , the quote - unquote noisy data there is just noisy and reverberant actually . And that 's what most o again , most of our work has been done with that , with with  , connected digits . professor c: but  , we have recognition now with some of the continuous speech , large vocabulary continuous speech , using Switchboard  , Switchboard recognizer , phd a: Yeah . You just take the Switchboard trained ? Yeah , professor c: That 's that 's what we 're doing , phd a: yeah . professor c: but we 're hop  , actually  , Dave and I were just talking earlier today about maybe at some point not that distant future , trying some of the techniques that we 've talked about on , some of the large vocabulary data .  , I guess no one had done yet done test one on the distant mike using  , the SRI recognizer and , phd f: I don't not that I know of . professor c: You 'll see a little smoke coming up from the the CPU or something trying to trying to do it , phd f: That 's right professor c: but  , yeah . But , you 're right that that that 's a real good point , that  , we we don't know yeah , what if any of these ta I guess that 's why they 're pushing that in the  in the evaluation . Anything else going on ? at you guys ' end , phd b: I don't have good result , with the inc including the new parameters , professor c: or ? phd b: I don't have good result . phd a: With what what other new p new parameter ? grad g: You 're talking about your voicing ? professor c: Yeah . professor c: seeing as how Sunil , phd b: I tried to include another new parameter to the traditional parameter , professor c: yeah . phd b: that , like , the auto - correlation , the R - zero and R - one over R - zero phd a:   phd b: and another estimation of the var the variance of the difference for of the spec si  , spectrum of the signal and and the spectrum of time after filt mel filter bank . phd b: and you have the on the other side you have the output of the mel filter bank . phd b: You can extend the coefficient of the mel filter bank and obtain an approximation of the spectrum of the signal . phd b: because , suppose we we think that if the variance is high , maybe you have n  , noise . phd b: To to To The idea is to found another feature for discriminate between voice sound and unvoice sound . And I did experiment I need to change to obtain this new feature I need to change the size the window size size . phd b: And I do I did two type of experiment to include this feature directly with the with the other feature and to train a neural network to select it voice - unvoice - silence silence phd a: Unvoiced . It 's neve e e sometime it 's worse , sometime it 's a little bit better , but not significantly . phd a: is it with TI - digits , or with ? phd b: And No , I work with eh , Italian and Spanish basically . phd b: And if I don't y use the neural network , and use directly the feature the results are worse . professor c:  we 've had these discussions before , and and one of the things that struck me was that  , about this line of thought that was particularly interesting to me was that we  whenever you condense things , in an irreversible way , you throw away some information . And , that 's mostly viewed on as a good thing , in the way we use it , because we wanna suppress things that will cause variability for  particular , phonetic units . And so the question is , can we figure out if there 's something we 've thrown away that we shouldn't have . So , when they were looking at the difference between the filter bank and the FFT that was going into the filter bank , I was thinking " oh , OK , so they 're picking on something they 're looking on it to figure out noise , or voice voiced property whatever . But for me sort of the interesting thing was , " well , but is there just something in that difference which is useful ? " So another way of doing it , maybe , would be just to take the FFT  , power spectrum , and feed it into a neural network , phd b: To know professor c: and then use it , you know , in combination , or alone , or or whatever phd f: Wi - with what targets ? phd a: Voiced , unvoiced is like professor c: no . professor c: No the just the same same way we 're using  , the same way that we 're using the filter bank . And , you know , maybe if it 's used in combination , it will get at something that we 're missing . And maybe , you know , using , orth you know , KLT , or  , adding probabilities , all th all the different ways that we 've been playing with , that we would let the essentially let the neural network determine what is it that 's useful , that we 're missing here . professor c: Well , that 's probably why y i it would be unlikely to work as well by itself , but it might help in combination . professor c: But I I I have to tell you , I can't remember the conference , but , I think it 's about ten years ago , I remember going to one of the speech conferences and and  , I saw within very short distance of one another a couple different posters that showed about the wonders of some auditory inspired front - end or something , and a couple posters away it was somebody who compared one to  , just putting in the FFT and the FFT did slightly better . professor c: but again we have these wonderful statistical mechanisms for quantifying that a that variability , and you know , doing something reasonable with it . professor c: So , It - it 's same , you know , argument that 's gone both ways about  , you know , we have these data driven filters , in LDA , and on the other hand , if it 's data driven it means it 's driven by things that have lots of variability , and that are necessarily not necessarily gonna be the same in training and test , so , in some ways it 's good to have data driven things , and in some ways it 's bad to have data driven things . So , phd a: Yeah , d professor c: part of what we 're discovering , is ways to combine things that are data driven than are not . professor c: so anyway , it 's just a thought , that that if we if we had that maybe it 's just a baseline  , which would show us " well , what are we really getting out of the filters " , or maybe i i probably not by itself , but in combination , phd d:   professor c: you know , maybe there 's something to be gained from it , and let the But , you know , y you 've only worked with us for a short time , maybe in a year or two you w you will actually come up with the right set of things to extract from this information . phd a: What one one  p one thing is like what before we started using this VAD in this Aurora , the th what we did was like , I I guess most of you know about this , adding this additional speech - silence bit to the cepstrum and training the  on that . phd a: That is just a binary feature and that seems to be improving a lot on the SpeechDat - Car where there is a lot of noise but not much on the TI - digits . So , a adding an additional feature to distin to discriminate between speech and nonspeech was helping . phd d: Wait I I 'm sorry ? phd a: Yeah , we actually added an additional binary feature to the cepstrum , just the baseline . Well , in in the case of TI - digits it didn't actually give us anything , because there wasn't any f anything to discriminate between speech , phd d: Yeah . But anyway the question is even more , is within speech , can we get some features ? Are we drop dropping information that can might be useful within speech , phd a: OK . professor c: And it 's particularly more relevant now since we 're gonna be given the endpoints . phd a: There was a paper in ICASSP this ICASSP over the  extracting some higher - order  , information from the cepstral coefficients and I forgot the name . Some is some harmonics I don't know , I can I can pull that paper out from ICASSP . phd a: It wa it was taking the ,  It was about finding the higher - order moments of Yeah . professor c: Yeah , phd a: And I 'm not sure about whether it is the higher - order moments , or professor c: cumulants , yeah . professor c: Yeah , but again You could argue that th that 's exactly what the neural network does . professor c: So n neural network  , is in some sense equivalent to computing , you know , higher - order moments of what you phd a: trying to f to Moments , yeah . I can I can just  , share a little bit Sunil hasn't hasn't heard about  , what I 've been doing . grad g: so , I told you I was I was I was getting prepared to take this qualifier exam . So basically that 's just , trying to propose  , your next your your following years of of your PHD work , trying trying to find a project to to define and and to work on . So , I 've been , looking into , doing something about r  , speech recognition using acoustic events . So , the idea is you have all these these different events , for example voicing , nasality , R - coloring , you know burst or noise , frication , that kinda stuff , building robust  , primary detectors for these acoustic events , and using the outputs of these robust detectors to do speech recognition .  , and , these these primary detectors , will be , inspired by , you know , multi - band techniques , doing things , similar to Larry Saul 's work on , graphical models to to detect these these , acoustic events . And , so I I been I been thinking about that and some of the issues that I 've been running into are , exactly what what kind of acoustic events I need , what  , what acoustic events will provide a a good enough coverage to in order to do the later recognition steps . And , also , once I decide a set of acoustic events , h how do I how do I get labels ? Training data for for these acoustic events . And , then later on down the line , I can start playing with the the models themselves , the the primary detectors .  , so , I kinda see like , after after building the primary detectors I see  , myself taking the outputs and feeding them in , sorta tandem style into into a  , Gaussian mixtures  back - end , and doing recognition . grad g: professor c: By by the way , the voiced - unvoiced version of that for instance could tie right in to what Carmen was looking at . professor c: you know , if you if a multi - band approach was helpful as as I think it is , it seems to be helpful for determining voiced - unvoiced , grad g:   And so , this this past week  , I 've been  , looking a little bit into  , TRAPS  , and doing doing TRAPS on on these e events too , just , seeing seeing if that 's possible .  , and  , other than that , I was kicked out of I - house for living there for four years . professor c: or , no ? grad g: well , s s som something like that . professor c: Suni - i d ' you v did  did you find a place ? phd a: no professor c: Is that out of the way ? phd a: not yet .  , yesterday I called up a lady who ha who will have a vacant room from May thirtieth and she said she 's interviewing two more people . And then , you 're coming back  phd a: i  , I I p I plan to be here on thirty - first . professor c: Thirty - first , phd a: Yeah , well if there 's a house available or place to professor c: OK . professor c: They 're available , and they 'll be able to get you something , so worst comes to worst we 'll put you up in a hotel for for for a while phd a: Yeah . grad e: You know , if you 're in a desperate situation and you need a place to stay , you could stay with me for a while . professor c: Do y wanna say anything about You you actually been  , last week you were doing this stuff with Pierre , you were you were mentioning . Is that that something worth talking about , or ? grad e: it 's Well , it I don't think it directly relates .  , well , so , I was helping a speech researcher named Pierre Divenyi and he 's int He wanted to  , look at  , how people respond to formant changes , I think . So he he created a lot of synthetic audio files of vowel - to - vowel transitions , and then he wanted a psycho - acoustic  , spectrum . And he wanted to look at  , how the energy is moving over time in that spectrum and compare that to the to the listener tests . And to  he he t wanted to track the peaks so he could look at how they 're moving . I found the roots of the  , LPC polynomial to , track the peaks in the , PLP LPC spectra . phd a: well there is aligned spectral pairs , is like the the Is that the aligned s professor c: It 's a r root LPC , of some sort . phd a: Oh , it 's like line sp professor c: Except I think what they call line spectral pairs they push it towards the unit circle , don't they , phd a: Yeah , yeah , yeah , yeah . But what we 'd used to do w when I did synthesis at National Semiconductor twenty years ago , the technique we were playing with initially was was taking the LPC polynomial and and  , finding the roots . It wasn't PLP cuz Hynek hadn't invented it yet , but it was just LPC , and  , we found the roots of the polynomial , And th When you do that , sometimes they 're f they 're what most people call formants , sometimes they 're not . professor c:  Formant tracking with it can be a little tricky cuz you get these funny values in in real speech , phd f: So you just You typically just get a few roots ? professor c: but . phd f: something like that ? professor c: And it depends on the order that you 're doing , but . So , if @ @ Every root that 's Since it 's a real signal , the LPC polynomial 's gonna have real coefficients . So I think that means that every root that is not a real root is gonna be a c complex pair , phd f:   So for each And if you look at that on the unit circle , one of these one of the members of the pair will be a positive frequency , one will be a negative frequency , I think . So I just So , f for the I 'm using an eighth - order polynomial and I 'll get three or four of these pairs professor c: Yeah .  for real speech in real then what you end up having is , like I say , funny little things that are don't exactly fit your notion of formants all that well . grad e: Mmm , professor c: And and what  in in what we were doing , which was not so much looking at things , it was OK phd d: I professor c: because it was just a question of quantization .  , we were just you know , storing It was We were doing , stored speech , quantization . professor c: But but  , in your case  , you know phd d: Actually you have peaks that are not at the formant 's positions , but they are lower in energy grad e: But there 's some of that , yes . phd f: If this is synthetic speech can't you just get the formants directly ?  h how is the speech created ? grad e: It was created from a synthesizer , and  phd f: Wasn't a formant synthesizer was it ? professor c: I bet it it might have may have been grad e: I d d this professor c: but maybe he didn't have control over it or something ? grad e: In in fact w we we could get , formant frequencies out of the synthesizer , as well . And , w one thing that the , LPC approach will hopefully give me in addition , is that I I might be able to find the b the bandwidths of these humps as well . grad e: but I don't think there 's a g a really good reason not to  , get the formant frequencies from the synthesizer instead . professor c: Yeah , so the actual So you 're not getting the actual formants per se . professor c: You 're getting something that is is  , af strongly affected by the PLP model . So it 's a little It 's It 's It 's sort of sort of a different thing . i Ordinarily , in a formant synthesizer , the bandwidths as well as the ban  , formant centers are phd f: Yeah . professor c: that 's Somewhere in the synthesizer that was put in , as as what you grad e:   professor c: But but yeah , you view each complex pair as essentially a second - order section , which has , band center and band width , and  ,  But . So , yeah , you 're going back today and then back in a week I guess , phd a: Yeah


I 'm sick of being the one to sort of go through and say , " Well , what do you think about this ? " You wanna ? phd d: Yeah . phd f: Should we take turns ? You want me to run it today ? professor b: Yeah . Let 's see , maybe we should just get a list of items things that we should talk about .  , I guess there 's the usual updates , everybody going around and saying , you know , what they 're working on , the things that happened the last week . phd f: This is what what do you ,  what 's in the paper there ? phd c: So it 's the paper that describe basically the , system that were proposed for the Aurora . phd f: The one that we s we submitted the last round ? phd c: Right , yeah . phd f: Where where 's it gonna be this year ? phd c: It 's , Aalborg in Denmark . Then , whhh well , I 've been working on on t mainly on on - line normalization this week .  , second thing is , the training of , on - line normalization with two different means , one mean for the silence and one for the speech .  , and so I have two recursions which are controlled by the , probability of the voice activity detector . professor b: So do you maybe make errors in different places ? Different kinds of errors ? phd c: I didn't look , more closely . Well , eh , there is one thing that we can observe , is that the mean are more different for for C - zero and C - one than for the other coefficients . And Yeah , it the C - one is There are strange strange thing happening with C - one , is that when you have different kind of noises , the mean for the the silence portion is can be different . phd c: So when you look at the trajectory of C - one , it 's has a strange shape and I was expecting th the s that these two mean helps , especially because of the the strange C - ze C - one shape , which can like , yo you can have , a trajectory for the speech and then when you are in the silence it goes somewhere , but if the noise is different it goes somewhere else . phd c: So which would mean that if we estimate the mean based on all the signal , even though we have frame dropping , but we don't frame ev  , drop everything , but  , this can hurts the estimation of the mean for speech , and Mmm .  , a third thing is , that instead of t having a fixed time constant , I try to have a time constant that 's smaller at the beginning of the utterances to adapt more quickly to the r something that 's closer to the right mean . phd c: well , if it 's higher than a certain threshold , I keep it to this threshold to still , adapt , the mean when if the utterance is , long enough to to continue to adapt after , like , one second professor b:   It seems pretty phd f: Wasn't there some experiment you were gonna try where you did something differently for each ,  I don't know whether it was each mel band or each , FFT bin or someth There was something you were gonna  , some parameter you were gonna vary depending on the frequency . I don't know if that was phd c: I guess it was I don't know . u Maybe it 's this this idea of having different on - line normalization , tunings for the different MFCC 's . And then it was something about , some and then somebody said " yeah , it does seem like , you know , C - zero is the one that 's , you know , the major one " or , s I can't remember exactly what it was now . S  , it 's very important to normalize C - zero and much less to normalize the other coefficients . And we I think , we kind of know that normalizing C - one doesn't help with the current scheme . In my idea , I I was thinking that the the the reason is maybe because of these funny things that happen between speech and silence which have different means . But maybe it 's not so so easy to professor b: I I really would like to suggest looking , a little bit at the kinds of errors . I know you can get lost in that and go forever and not see too much , but sometimes , phd c:   professor b: but but , just seeing that each of these things didn't make things better may not be enough . It may be that they 're making them better in some ways and worse in others , phd c: Yeah . professor b: or increasing insertions and decreasing deletions , or or , you know , helping with noisy case but hurting in quiet case . And if you saw that then maybe you it would something would occur to you of how to deal with that . I 've been playing a little bit with some kind of thresholding , and , mmm , as a first experiment , I think I Yeah . Well , what I did is t is to take ,  to measure the average no , the maximum energy of s each utterance and then put a threshold Well , this for each mel band . Then put a threshold that 's fifteen DB below well , a couple of DB below this maximum , professor b:   phd c: So I was adding a white noise energy , that 's fifteen DB below the maximum energy of the utterance . When we look at at the , MFCC that result from this , they are a lot more smoother .  , when we compare , like , a channel zero and channel one utterance  , so a clean and , the same noisy utterance well , there is almost no difference between the cepstral coefficients of the two . And the result that we have in term of speech recognition , actually it 's not it 's not worse , it 's not better neither , but it 's , kind of surprising that it 's not worse phd f:  . phd c: because basically you add noise that 's fifteen DB just fifteen DB below the maximum energy . phd c: And at least phd f: So why does that m smooth things out ? I don't I don't understand that . Right ? phd c: It 's I think , it 's whitening This the portion that are more silent , professor b: Cuz it 's phd c: as you add a white noise that are has a very high energy , it whitens everything phd f: Huh . phd c: and and the high - energy portion of the speech don't get much affected anyway by the other noise . And and professor b: So , again , if you trained in one kind of noise and tested in the same kind of noise , you 'd you know , given enough training data you don't do b do badly . The reason that we d that we have the problems we have is because it 's different in training and test . professor b: so when you whiten it , then it 's like you the the only noise to to first order , the only th noise that you have is white noise and you 've added the same thing to training and test . professor b:  phd f: So would that be similar to , like , doing the smoothing , then , over time or ? phd c:   professor b: Well , it 's a kind of smoothing , phd c: I think it 's I think it 's different . professor b: but phd c: It 's it 's something that yeah , that affects more or less the silence portions because phd f:   phd c: Well , anyway , the sp the portion of speech that ha have high energy are not ch a lot affected by the noises in the Aurora database . phd c: If if you compare th the two shut channels of SpeechDat - Car during speech portion , it 's n n n the MFCC are not very different . And , professor b: Yeah , but you 're still getting more recognition errors , phd c:  professor b: which means that the differences , even though they look like they 're not so big , are are hurting your recognition . So , but in this case I I really expect that maybe the the two these two stream of features , they are very different .  , and maybe we could gain something by combining them professor b: Well , the other thing is that you just picked one particular way of doing it . phd c: or professor b: first place it 's fifteen DB , down across the utterance . phd f: So what was the what was the threshold part of it ? Was the threshold , how far down ? professor b: Yeah . How does it go ? If it if if the peak value 's above some threshold , then you add the noise ? Or if it 's below s phd c: I systematically add the noise , but the , noise level is just some kind of threshold below the peak . professor b: So then afterwards a log is taken , and that 's so sort of why the the little variation tends to go away . And I don't know , maybe a constant noise addition would would be fine also , or  professor b: Or or not constant but but , varying over time in fact is another way to go .  professor b: Were you using the the normalization in addition to this ?  , what was the rest of the system ? phd c:  Yeah . A third thing is that , I play a little bit with the ,  finding what was different between , And there were a couple of differences , like the LDA filters were not the same .  , so when we put s some noise compensation the , LDA filter that that 's derived from noisy speech is not more anymore optimal .  , if we use the the old LDA filter ,  the LDA filter that was in the proposal , we have , like , eighty - two point seven percent recognition rate , on noisy speech when the system is trained on clean speech . But and when we use the filter that 's derived from clean speech we jumped so from eighty - two point seven to eighty - five point one , which is a huge leap . So now the results are more similar , and I don't I will not , I think , investigate on the other differences , which is like the number of MFCC that we keep and other small things that we can I think optimize later on anyway . But on the other hand if everybody is trying different kinds of noise suppression things and so forth , it might be good to standardize on the piece that we 're not changing . Right ? So if there 's any particular reason to ha pick one or the other ,  Which which one is closer to what the proposal was that was submitted to Aurora ? Are they they both ? Well ,  phd c: I think Yeah . I think th th  , the new system that I tested is , I guess , closer because it doesn't have it have less of of France Telecom stuff , phd d: You mean the phd c: I phd d: The whatever you , tested with recently . phd d: Yeah ? professor b: Well , no , I I 'm I Yeah , you 're trying to add in France Telecom . Right ? Or phd d: The number of cepstral coefficients is what ? professor b: Cep phd c:   So , I think we 'd wanna standardize there , wouldn't we ? phd c: Yeah , yeah . phd c: I think we were gonna work with with this or this new system , or with phd d: so the the right now , the the system that is there in the what we have in the repositories , with uses fifteen . So , we haven't w we have been always using , fifteen coefficients , phd c: Yeah . Then professor b: I think as long as you guys agree on it , it doesn't matter . phd d: mmm professor b: I think we have a maximum of sixty , features that we 're allowed . Ma - maybe we can  , at least , I 'll t s run some experiments to see whether once I have this noise compensation to see whether thirteen and fifteen really matters or not . phd d: Never tested it with the compensation , but without , compensation it was like fifteen was s slightly better than thirteen , phd c: Yeah . phd d: You know , always for the matched condition , you always get a slightly better performance for log energy than C - zero . phd d: But not for  , for matched and the clean condition both , you get log energy  you get a better performance with log energy . phd d: Well , maybe once we have this noise compensation , I don't know , we have to try that also , whether we want to go for C - zero or log energy . phd f: So do you have more , Stephane , or ? phd c: that 's it , I think . phd f: How about you , Barry ? grad a: still working on my my quals preparation stuff .  , so I 'm I 'm thinking about , starting some , cheating experiments to , determine the ,  the relative effectiveness of , some intermediate categories that I want to classify . So , for example , if I know where voicing occurs and everything , I would do a phone  , phone recognition experiment , somehow putting in the the ,  the perfect knowledge that I have about voicing . So , in particular I was thinking , in in the hybrid framework , just taking those LNA files , and , setting to zero those probabilities that ,  that these phones are not voicing . So say , like , I know this particular segment is voicing , I would say , go into the corresponding LNA file and zonk out the the posteriors for , those phonemes that , are not voiced , phd f:   And so this would be a useful thing , to know in terms of , like , which which ,  which of these categories are are good for , speech recognition . grad a: So , that 's I hope to get those ,  those experiments done by by the time quals come come around in July . phd f: So do you just take the probabilities of the other ones and spread them out evenly among the the remaining ones ? grad a: Yeah . I I I was thinking OK , so just set to set to some really low number , the the non - voiced , phones . phd f: wh are you gonna do digits grad a: Yeah , m  , well , I 'm gonna f work with TIMIT phd f: or ? With TIMIT . So where do the outputs of the net go into if you 're doing phone recognition ? grad a: Oh . So maybe , Chronos phd f: An - and you 're gonna the you 're gonna do phone recognition with that ? grad a: or Phone recognition . And , another thing would be to extend this to , digits or something where I can look at whole words . grad a: And I would be able to see , not just , like , phoneme events , but , inter - phoneme events . So , like , this is from a stop to to a vo a vocalic phd f:   phd f:  Let 's see , I haven't done a whole lot on anything related to this this week . Well , in my lunch talk last week I I said I 'd tried phase normalization and gotten garbage results using that l  , long - term mean subtraction approach . And also I 've been talking to , Andreas and Thilo about the , SmartKom language model and about coming up with a good model for , far mike use of the SmartKom system . So I 'm gonna be working on , implementing this mean subtraction approach in the far - mike system for the SmartKom system ,  . And , one of the experiments we 're gonna do is , we 're gonna , train the a Broadcast News net , which is because that 's what we 've been using so far , and , adapt it on some other data .  , An - Andreas wants to use , data that resembles read speech , like these digit readings , because he feels that the SmartKom system interaction is not gonna be exactly conversational . grad g: S so actually I was wondering , how long does it take to train that Broadcast News net ? professor b: The big one takes a while . professor b: So but , you know , you can get I don't know if you even want to run the big one , in the in the final system , cuz , you know , it takes a little while to run it . So , you can scale it down by I 'm sorry , it was two , three weeks for training up for the large Broadcast News test set training set . professor b: i so if you trained on half as much and made the net , half as big , then it would be one fourth the amount of time grad g: OK . Also , I guess we had we 've had these , little di discussions I guess you ha haven't had a chance to work with it too much about about ,  m other ways of taking care of the phase . So , I I guess that was something I could say would be that we 've talked a little bit about grad g:   professor b: you just doing it all with complex arithmetic and ,  and not not , doing the polar representation with magnitude and phase . But it looks like there 's ways that one could potentially just work with the complex numbers and and and in principle get rid of the effects of the average complex spectrum . But grad g: And , actually , regarding the phase normalization So I did two experiments , and one is So , phases get added , modulo two pi , and because you only know the phase of the complex number t t to a value modulo two pi . And so I thought at first , that , what I should do is unwrap the phase because that will undo that .  , but I actually got worse results doing that unwrapping using the simple phase unwrapper that 's in Matlab than I did not unwrapping at all . So I 'm I 'm still hopeful that that  , we we don't even know if the phase is something the average phase is something that we do want to remove . But , at least in principle it looks like there 's there 's , a couple potential ways to do it . So you work with the complex numbers and then when you get the spectrum the average complex spectrum  , actually divide it out , as opposed to taking the log and subtracting . And , actually I was talking to Dick Karp about it a little bit , and and and , since I got thinking about it , and and , so one thing is that y you 'd have to do , I think ,  we may have to do this on a whiteboard , but I think you have to be a little careful about scaling the numbers that you 're taking the complex numbers that you 're taking the log of because the Taylor expansion for it has , you know , a square and a cube , and and so forth . And and so if if you have a a number that is modulus , you know , very different from one It should be right around one , if it 's cuz it 's a expansion of log one one minus epsilon or o is is one plus epsilon , or is it one plus ? Well , there 's an epsilon squared over two and an epsilon cubed over three , grad g: OK . But that 's not a big deal cuz it 's the log of of K times a complex number , then you can just that 's the same as log of K plus log of the complex number . How about you , Sunil ? phd d: So , I 've been , implementing this , Wiener filtering for this Aurora task . And then I ran the whole recognition experiment with Italian and I got , like , worse results than not using it . And then it looks like I have some problem in the way there is some some very silly bug somewhere . I was looking at the spectrograms that I got and it 's , like w it 's it 's very horrible . Like , when I professor b: I I missed the v I 'm sorry , I was I was distracted . phd d: And it it it gave , like I just got the signal out and it it was OK . So , I plugged it in somewhere and then  , it 's like I had to remove some part and then plugging it in somewhere . So , it was real  , I thought it was all fine and then I ran it , and I got something worse than not using it . So , I was like I 'm trying to find where the m m problem came , professor b:  - huh . And , the other thing , was ,  Well , Hynek showed up one suddenly on one day and then I was t talking wi professor b: Right . So I was actually that day I was thinking about d doing something about the Wiener filtering , and then Carlos matter of stuff . And then he gave me a whole bunch of filters what Carlos used for his , thesis and then that was something which came up . And then ,  So , I 'm actually , thinking of using that also in this , W Wiener filtering because that is a m modified Wiener filtering approach , where instead of using the current frame , it uses adjacent frames also in designing the Wiener filter . So instead of designing our own new Wiener filters , I may just use one of those Carlos filters in in this implementation professor b:   phd d: and see whether it it actually gives me something better than using just the current f current frame , which is in a way , something like the smoothing the Wiener filter professor b:   phd d: but @ @ S so , I don't know , I was h I 'm I 'm I 'm , like that so that is the next thing . Once this I once I sort this pro  , problem out maybe I 'll just go into that also . So , I , like , plugged some groupings for computing this eigen  , s values and eigenvectors . So just I just @ @ some small block of things which I needed to put together for the subspace approach . phd f: What what is VTS again ? phd d: New phd e: Eh , Vectorial Taylor Series . I think I ask you that every single meeting , don't I ? phd e: What ? phd f: I ask you that question every meeting . phd e: If Well professor b: It 's good to have some , cases of the same utterance at different different times . And we put everything , the result is better , but it 's not better than the result that we have without VTS . So that @ @ given that you 're using the VAD also , the effect of the VTS is not so far phd e: Is not . professor b: Do you How much of that do you think is due to just the particular implementation and how much you 're adjusting it ? Or how much do you think is intrinsic to ? phd e: Pfft . I don't know because phd c: Are you still using only the ten first frame for noise estimation phd e: Hhh , phd c: or ? Or i ? phd e: I do the experiment using only the f onl eh , to use on only one fair estimation of the noise . And , well , it 's a little bit better but not n phd c: Maybe you have to standardize this thing also , noise estimation , because all the thing that you are testing use a different They all need some some noise noise spectra phd d: Mmm .  , given that we 're going to have for this test at least of  , boundaries , what if initially we start off by using known sections of nonspeech for the estimation ? phd c:   professor b: first place ,  even if ultimately we wouldn't be given the boundaries , this would be a good initial experiment to separate out the effects of things .  , how much is the poor you know , relatively , unhelpful result that you 're getting in this or this or this is due to some inherent limitation to the method for these tasks and how much of it is just due to the fact that you 're not accurately finding enough regions that that are really n noise ? phd d: Mmm . So maybe if you tested it using that , you 'd have more reliable stretches of nonspeech to do the estimation from and see if that helps . Well , we have If this if this is the noise signal , in the log domain , we have something like this . And the idea of these methods is to n given a ,  professor b:  phd e: How do you say ? I will read because it 's better for my English . I i given is the estimate of the PDF of the noise signal when we have a , a statistic of the clean speech and an statistic of the noisy speech . So , the top equation is is is phd e: No , this in the it 's this is the log domain . professor b: Which is which is the log domain ? phd e: Is the T is egual is equal to , log of professor b: And but Y is what ? Y of the spectrum phd e: this this is this professor b: or ? phd e: and this is this . phd c: p s this professor b: No , is that power spectrum ? Is it ? phd c: Yeah . And professor b: So that 's  phd e: This is the noisy Yeah , it 's professor b: OK . professor b: OK , so you have power spectrum added there and down here you have you you put the depends on T , but b all of this is just you just mean phd e: w o Yeah . phd e: Yeah , maybe professor b: o phd e: But , n Well , y we can expre we can put this expression professor b: X times one plus , N  , N N N minus X ? phd e: The Yeah . professor b: And then ,  So that 's log of X plus log of one plus ,  phd e: And the noise signal . phd e: Well , if we apply the log , we have E is n phd c: Mmm . phd d: and phd e: log E is equal , oh , to log of X plus N . phd e: And , well , phd d: And , log of phd e: we can say that E is equal to log of , exponential of X plus exponential of N . phd d: Well , if E restricts It is y phd e: Well , this is this is in the ti the time domain . phd e: and we can put u that n the log domain log of X omega , but , well , in the time domain we have an exponential .  , it 's just if X and N are variables Right ? phd d: What is ,  ? professor b: The the the log of X plus N is not the same as the log of E to the X plus E to the N . But this i Well , I don't Well , professor b: Maybe we can take it off - line , phd e: maybe professor b: but I I don't know . Well , the expression that appear in the in the paper , is ,  phd d: The log the Taylor series expansion for log one plus N by X is professor b: OK . phd c: Is it the first - order expansion ? phd e: is X professor b: I i phd d: Yeah , the first one . If if you take log X into log one plus N by X , and then expand the log one plus N by X into Taylor series phd c: Yeah . phd e: Now , this is the and then phd c: Yeah , but the the second expression that you put is the first - order expansion of the nonlinear relation between phd e: Not exactly . Well , we have pfft , em Well , we can put that X is equal I is equal to log of , mmm professor b: That doesn't follow . Because cuz the log of a sum is not the same as th phd e: The top ? professor b: as phd e: Yeah , yeah , yeah , yeah , yeah . phd e: But we can  , we we know that , for example , the log of E plus B is equal to log of E plus log to B . So you could s phd c: What is that ? phd e: And we can , put this inside . phd e: And then we can , professor b: N no , phd e: you know professor b: but phd e: Yeah . professor b: The  , just more generally here , if you say " log of , A plus B " , the log of log of A plus B is not or A plus B is not the , log of E to the A plus E to the B . professor b: Right ? Cuz you cuz you up here you have the A plus B phd e: No . I say if I apply log , I have , log of E is equal to log of ,  in this side , is equal to log of X professor b: Plus N . professor b: And then how do you go from there to the ? phd e: This is right . OK , so let 's  , C equals A plus B , phd c: It 's log o of capital Y . It 's just by definition that the individual that the ,  So , capital X is by definition the same as E to the little X because she 's saying that the little X is is the ,  is the log . professor b: I think these things are a lot clearer when you can use fonts different fonts there phd e: Oh , yes . phd e: And now I can do it ,  pfff ! I can put log of EX plus log professor b: Oh . So now once you get that that one , then you then you do a first or second - order , or something , Taylor series expansion of this . And for that , well , the goal is to obtain ,  est estimate a PDF for the noisy speech when we have a a statistic for clean speech and for the noisy speech . Mmm ? And when w the way to obtain the PDF for the noisy speech is well , we know this statistic and we know the noisy st well , we can apply first order of the vector st Taylor series of the of the of well , the order that we want , increase the complexity of the problem . phd e: And then when we have a expression , for the mean and variance of the noisy speech , we apply a technique of minimum mean - square estimation professor b:   phd e: to obtain the expected value of the clean speech given the this statistic for the noisy speech professor b:   phd e: We can expre we can put that the PDF for the clean test , probability of the clean speech is equal to professor b: Yeah . professor b: So , how h how much in in the work they reported , how much noisy speech did you need to get , good enough statistics for the to get this mapping ? phd e: I don't know exactly . professor b: Cuz I think what 's certainly characteristic of a lot of the data in this test is that , you don't have the the training set may not be a a great estimator for the noise in the test set . And what are you using for the noisy ? Y y doing that strictly phd e: Of the noise I estimate the noises wi professor b:   professor b: And and you and you train it up entirely from , nonspeech sections in the test ? phd c:  . The first experiment that I do it is solely to calculate the , mmm well , this value professor b: Yeah . phd e: the compensation of the dictionary o one time using the the noise at the f beginning of the sentence .  , because well , the VTS methods In fact the first thing that I do is to to obtain , an expression for E probability e expression of of E . That mean that the VTS mmm , with the VTS we obtain ,  well , we we obtain the means for each Gaussian and the variance . phd e: Because we can write  , we can write that the estimation of the clean speech is equal at an expected value of the clean speech conditional to , the noise signal the probability f of the the statistic of the clean speech and the statistic of the noise . phd e: And we can put that this is equal to the estimated value of E minus a function that conditional to E to the T to the noise signal . Well , this is this function is the the term after develop this , the term that we we take . phd e: And I can put that this is equal to the noise signal minus Well , I put before this name ,  And I can calculate this . professor b: What is the first variable in that probability ? phd e: this is the Gaussian .  , if we have clean speech we have the dictionary for the clean speech , we have a probability f of our our weight for each Gaussian . phd e: And for calculate this , I have an I I can develop an expression that is phd d: It 's overlapping . I can calculate I can I calculated this value , with the statistic of the noisy speech that I calculated before with the VTS approximation .  , with the , nnn when I develop this in s Taylor Taylor series , I can't , calculate the mean and the variance of the for each of the Gaussian of the dictionary for the noisy speech . phd e: If I never do an estimat a newer estimation of the noise , this mean as mean and the variance are fixed . phd e: And for each s  , frame of the speech the only thing that I need to do is to calculate this in order to calculate the estimation of the clean speech given our noisy speech . professor b: So , I 'm I 'm not following this perfectly but , I Are you saying that all of these estimates are done using , estimates of the probability density for the noise that are calculated only from the first ten frames ? And never change throughout anything else ? phd e: Yeah . phd e: And the other estimation is when I do the  on - line estimation , I change the means and variance of th for the noisy speech professor b: Yeah ? phd e: each time that I detect noise . professor b: So you estimated , f completely forgetting what you had before ?  , or is there some adaptation ? phd e: no , no , no . It 's not completely No , it 's I am doing something like an adaptation of the noise . Now do we know , either from their experience or from yours , that , just having , two parameters , the the mean and variance , is enough ? Yeah .  , I know you don't have a lot of data to estimate with , but but ,  phd e: I estimate mean and variance for each one of the Gaussian of the codebook . phd e: I don't know i professor b: And you and and it 's ,  right , it 's only it 's only one Wait a minute . This is what 's the dimensionality of the Gaussian ? This is phd e: it 's in after the mel filter bank . Yeah , maybe maybe you don't have a phd e: the original paper say that only one Gaussian for the noise . But , no no paper is is a Bible , phd e: Yeah , maybe isn't the right thing . professor b: The question is , whether it would be helpful , i particularly if you used if you had more So , suppose you did This is almost cheating . But if y suppose you use the real boundaries that that you were in fact were given by the VAD and so forth or I I guess we 're gonna be given even better boundaries than that . And you look you take all o all of the nonspeech components in an utterance , so you have a fair amount . professor b: So first question would be to what extent i are the errors that you 're still seeing based on the fact that you have poor boundaries for the , nonspeech ? And the second question might be , given that you have good boundaries , could you do better if you used more parameters to characterize the noise ?  . Also another question might be  , they are doing they 're using first term only of the vector Taylor series ? phd e: Yeah . professor b: if you do a second term does it get too complicated cuz of the nonlinearity ? phd e: Yeah . phd e: Oh , it 's it 's the for me it 's the first time that I am working with VTS . phd e:  professor b: w we haven't had anybody work with it before , so it 's interesting to get your get your feedback about it . phd e: It 's another type of approximation because i because it 's a statistic statistic approximation to remove the noise . phd f: professor b: They prefer to have them on phd f: and the professor b: just so that they 're continuing to get the distant , information