Actually , I think he he redirected it to everybody also so  the PDA mikes  have a big bunch of energy at at  five hertz  where this came up was that  I was showing off these wave forms that we have on the web and and  I just sort of hadn't noticed this , but that the major , major component in the wave in the second wave form in that pair of wave forms is actually the air conditioner . I I have to be more careful about using that as a as a as a good illustration , in fact it 's not , of  of the effects of room reverberation . And then we had this other discussion about  whether this affects the dynamic range , cuz I know , although we start off with thirty two bits , you end up with  sixteen bits and you know , are we getting hurt there ? But  Dan is pretty confident that we 're not , that that quantization error is not is still not a significant factor there . So there was a question of whether we should change things here , whether we should change a capacitor on the input box for that or whether we should phd b: Yeah , he suggested a smaller capacitor , right ? professor a: Right . But then I had some other  thing discussions with him phd b: For the P D professor a: and the feeling was once we start monk monkeying with that , many other problems could ha happen . professor a: A simple thing to do is he he he has a I forget if it this was in that mail or in the following mail , but he has a a simple filter , a digital filter that he suggested . professor a:  The other thing that I don't know the answer to , but when people are using Feacalc here ,  whether they 're using it with the high - pass filter option or not . So when we 're doing all these things using our software there is  if it 's if it 's based on the RASTA - PLP program , which does both PLP and RASTA - PLP  then  there is an option there which then comes up through to Feacalc which  allows you to do high - pass filtering and in general we like to do that , because of things like this and it 's it 's pretty it 's not a very severe filter . Doesn't affect speech frequencies , even pretty low speech frequencies , at all , but it 's phd b: What 's the cut - off frequency it used ? professor a: Oh . I don't know I wrote this a while ago phd b: Is it like twenty ? professor a: Something like that .  I think there 's some effect above twenty but it 's it 's it 's it 's mild . So ,  it probably there 's probably some effect up to a hundred hertz or something but it 's it 's pretty mild . I don't know in the in the STRUT implementation of the stuff is there a high - pass filter or a pre pre - emphasis or something in the phd f:  . We we we want to go and check that in i for anything that we 're going to use the P D A mike for .  He says that there 's a pretty good roll off in the PZM mikes so we don't need need to worry about them one way or the other but if we do make use of the cheap mikes ,  we want to be sure to do that that filtering before we process it . And then again if it 's  depending on the option that the our our software is being run with , it 's it 's quite possible that 's already being taken care of .  the thing is it was since I was talking about reverberation and showing this thing that was noise , it wasn't a good match , but it certainly was still  an indication of the fact that you get noise with distant mikes .  It 's just not a great example because not only isn't it reverberation but it 's a noise that we definitely know what to do . professor a: So , it doesn't take deep a new bold new methods to get rid of  five hertz noise , so . So it was it was a bad example in that way , but it 's it still is it 's the real thing that we did get out of the microphone at distance , so it wasn't it w it w wasn't wrong it was inappropriate . So  , but  , Yeah , someone noticed it later pointed it out to me , and I went " oh , man .  So I think we 'll change our our picture on the web , when we 're @ @ . One of the things I was  , I was trying to think about what what 's the best way to show the difference an and I had a couple of thoughts one was , that spectrogram that we show is O K , but the thing is the eyes  and the the brain behind them are so good at picking out patterns from from noise that in first glance you look at them it doesn't seem like it 's that bad  because there 's many features that are still preserved . So one thing to do might be to just take a piece of the spec  of the spectrogram where you can see that something looks different , an and blow it up , and have that be the part that 's just to show as well .  Another , I was thinking of was  taking some spectral slices , like  like we look at with the recognizer , and look at the spectrum or cepstrum that you get out of there , and the the  , the reverberation  does make it does change that . grad c: W w what d what do you mean ? professor a: Well ,   all the recognizers look at frames . professor a: So it 's , yeah , at one point in time or  twenty over twenty milliseconds or something , you have a spectrum or a cepstrum . And if you look at phd b: You could just you could just throw up , you know ,  the  some MFCC feature vectors . You know , one from one , one from the other , and then , you know , you can look and see how different the numbers are . Well , that 's why I saying either Well , either spectrum or cepstrum phd b: I 'm just kidding . professor a: but but I think the thing is you wanna  phd b: I don't mean a graph . Yeah , at first I had a remark why I am wondering why the PDA is always so far . Since the last meeting we 've we 've tried to put together  the clean low - pass  downsampling , upsampling ,  the new filter that 's replacing the LDA filters , and also the  delay issue so that We considered th the the delay issue on the for the on - line normalization . professor a: But it 's not wer worse and it 's better better latency , phd f: It 's not professor a: right ? phd f: Yeah . It seems better when we look at the mismatched case but I think we are like like cheated here by the th this problem that  in some cases when you modify slight slightly modify the initial condition you end up completely somewhere air somewhere else in the in the space , the parameters . For Italian is at seventy - eight percent recognition rate on the mismatch , and this new system has eighty - nine . I don't I don't think it means that the new system is more robust professor a:  - huh . professor a: Well , the test would be if you then tried it on one of the other test sets , if if it was phd f: Y professor a: Right . professor a: So then if you take your changes phd f: It 's similar for other test sets professor a: and then phd f: but  from this se seventy - eight  percent recognition rate system , I could change the transition probabilities for the the first  and it will end up to eighty - nine also . phd f: By using point five instead of point six , point four as in the the HTK script . phd b:  th the only difference is you change the self - loop transition probability by a tenth of a percent phd f: Yeah . From point phd f: Even tenth of a percent ? phd b: I I 'm sorry phd f: Well , we tried we tried point one , phd b: f for point from You change at point one phd f: yeah . professor a: Oh ! phd b: and n not tenth of a percent , one tenth , phd f:  . phd b: alright ?  so from point five so from point six to point five and you get ten percent better . phd b: And it 's I think it 's what you basically hypothesized in the last meeting about  it just being very phd f:   phd b: and I think you mentioned this in your email too it 's just very  phd f: Mmm , yeah . phd b: you know get stuck in some local minimum and this thing throws you out of it I guess . professor a: Well , what 's what are according to the rules what what are we supposed to do about the transition probabilities ? Are they supposed to be point five or point six ? phd b: I think you 're not allowed to Yeah . But changing it to point five I think is which gives you much better results , but that 's not allowed . phd f: Yeah , but even if you use point five , I 'm not sure it will always give you the better results phd b: Yeah . We only tested it on the the medium mismatch , phd f: on the other training set ,  . phd b: right ? You said on the other cases you didn't notice phd f: Yeah . I think the reason is , yeah , I not I it was in my mail I think also , is the fact that the mismatch is trained only on the far microphone . Well , in for the mismatched case everything is  using the far microphone training and testing , whereas for the highly mismatched , training is done on the close microphone so it 's it 's clean speech basically so you don't have this problem of local minima probably and for the well - match , it 's a mix of close microphone and distant microphone and Well . phd b: I did notice  something phd f: So th I think the mismatch is the more difficult for the training part . phd b: Somebody , I think it was Morgan , suggested at the last meeting that I actually count to see how many parameters and how many frames . phd b: And there are  almost one point eight million frames of training data and less than forty thousand parameters in the baseline system . phd b: I did one quick experiment just to make sure I had everything worked out and I just  f for most of the  For for all of the digit models , they end up at three mixtures per state . And so I just did a quick experiment , where I changed it so it went to four and  it it it didn't have a r any significant effect at the  medium mismatch and high mismatch cases and it had it was just barely significant for the well - matched better . And I think also just seeing what we saw  in terms of the expected duration of the silence model ? when we did this tweaking of the self - loop ? The silence model expected duration was really different . phd b: And so in the case where  it had a better score , the silence model expected duration was much longer . I think you know if we make a better silence model I think that will help a lot too  for a lot of these cases so but one one thing I I wanted to check out before I increased the  number of mixtures per state was  in their default training script they do an initial set of three re - estimations and then they built the silence model and then they do seven iterations then the add mixtures and they do another seven then they add mixtures then they do a final set of seven and they quit . Seven seems like a lot to me and it also makes the experiments go take a really long time  to do one turn - around of the well matched case takes like a day . phd b: And so you know in trying to run these experiments I notice , you know , it 's difficult to find machines , you know , compute the run on . And so one of the things I did was I compiled HTK for the Linux machines professor a:   phd b: cuz we have this one from IBM that 's got like five processors in it ? professor a: Right . phd b: and so now I 'm you can run stuff on that and that really helps a lot because now we 've got you know , extra machines that we can use for compute . And if I 'm do running an experiment right now where I 'm changing the number of iterations ? from seven to three ? phd d:   And so if we can get away with just doing three , we can do many more experiments more quickly . And if it 's not a a huge difference from running with seven iterations , you know , we should be able to get a lot more experiments done . But if we can you know , run all of these back - ends f with many fewer iterations and on Linux boxes we should be able to get a lot more experimenting done . So I wanted to experiment with cutting down the number of iterations before I increased the number of Gaussians . professor a: But they 're not making things worse and we have reduced latency , right ? phd f: Yeah . But actually  actually it seems to do a little bit worse for the well - matched case and we just noticed that Yeah , actually the way the final score is computed is quite funny . It 's not a weighted mean of word error rate , it 's a weighted mean of improvements . Which means that actually the weight on the well - matched is Well I well what what What happened is that if you have a small improvement or a small if on the well - matched case it will have  huge influence on the improvement compared to the reference because the reference system is is is quite good for for the well - ma well - matched case also . phd b: So it it weights the improvement on the well - matched case really heavily compared to the improvement on the other cases ? phd f: No , but it 's the weighting of the of the improvement not of the error rate . Yeah , and it 's hard to improve on the on the best case , cuz it 's already so good , right ? phd f: Yeah but what  is that you can have a huge improvement on the H HMK 's ,  like five percent  absolute , and this will not affect the final score almost  this will almost not affect the final score because this improvement because the improvement  relative to the the baseline is small professor a: So they do improvement in terms of  accuracy ? rather than word error rate ? phd f:  .  improvement ? professor a: So phd f: No , it 's compared to the word er it 's improvement on the word error rate , professor a: OK . professor a: So if you have  ten percent error and you get five percent absolute  improvement then that 's fifty percent . So what you 're saying then is that if it 's something that has a small word error rate , then  a even a relatively small improvement on it , in absolute terms , will show up as quite quite large in this . Sure , but when we think about the weighting , which is point five , point three , point two , it 's on absolute on on relative figures , professor a: Yeah . That 's why I 've been saying we should be looking at word error rate  and and not not at at accuracies . phd b: it 's not it 's not that different , right ?  , just subtract the accuracy . professor a: Yeah but you 're but when you look at the numbers , your sense of the relative size of things is quite different . professor a: If you had ninety percent  correct and five percent , five over ninety doesn't look like it 's a big difference , but five over ten is is big . professor a: So just when we were looking at a lot of numbers and getting sense of what was important . professor a: What 's a little bit ? Like phd f: Like , it 's difficult to say because again  I 'm not sure I have the  phd b: Hey Morgan ? Do you remember that Signif program that we used to use for testing signi ? Is that still valid ? I I 've been using that . I just use my old one from ninety - two or whatever professor a: Yeah , I 'm sure it 's not that different but but he he  he was a little more rigorous , as I recall . s phd f:  well we start from ninety - four point sixty - four , and we go to ninety - four point O four . For Finnish , we start to ninety - three point eight - four and we go to ninety - three point seventy - four . And for Spanish we are we were at ninety - five point O five and we go to ninety - three - s point sixty one . professor a: And is that wh what do you know what piece you 've done several changes here . Because nnn , well  we don't have complete result , but the filter So the filter with the shorter delay hurts on Italian well - matched , which And , yeah . And the other things , like  downsampling , upsampling , don't seem to hurt and the new on - line normalization , neither . If we saw that making a small change like , you know , a tenth , to the self - loop had a huge effect , can we really make any conclusions about differences in this stuff ? phd f:   phd f: I think we can be completely fooled by this thing , but I don't know . There is first this thing , and then the yeah , I computed the  like , the confidence level on the different test sets . professor a: But OK , so you these these degradations you were talking about were on the well - matched case phd f: So . Do the does the new filter make things  better or worse for the other cases ? phd f: Yeah . OK , so  I guess the argument one might make is that , " Yeah , if you looked at one of these cases and you jiggle something and it changes then  you 're not quite sure what to make of it . But when you look across a bunch of these and there 's some some pattern , so eh h here 's all the if if in all these different cases it never gets better , and there 's significant number of cases where it gets worse , then you 're probably hurting things , I would say . So   at the very least that would be a reasonably prediction of what would happen with with a different test set , that you 're not jiggling things with . What I was asking , though , is  are what 's what 's the level of communication with  the O G I gang now , about this and phd f: Well , we are exchanging mail as soon as we we have significant results . We are working on our side on other things like  also trying a sup spectral subtraction but of of our own , another spectral substraction . It 's going professor a: Is there any further discussion about this this idea of of having some sort of source code control ? phd f: Yeah . As soon as we have something that 's significant and that 's better than than what was submitted , we will fix fix the system and But we 've not discussed it it it this yet , yeah . Sounds like a great idea but but I think that that  he 's saying people are sort of scrambling for a Eurospeech deadline . Anybo - anybody in the in this group do doing anything for Eurospeech ? phd f: S professor a: Or , is that what is that phd f: Yeah we are We are trying to to do something with the Meeting Recorder digits , professor a: Right . phd f: and , well , some people from OGI are working on a paper for this , but there is also the  special session about th Aurora which is  which has an extended deadline . professor a: So the deadline When 's the deadline ? When 's the deadline ? phd f:  ? I think it 's the thirteenth of May . So , I I think that you could certainly start looking at at the issue  but but  I think it 's probably , on s from what Stephane is saying , it 's it 's unlikely to get sort of active participation from the two sides until after they 've phd b: Well I could at least Well , I 'm going to be out next week but I could try to look into like this  CVS over the web . That seems to be a very popular way of people distributing changes and over , you know , multiple sites and things professor a:   phd b: so maybe if I can figure out how do that easily and then pass the information on to everybody so that it 's you know , as easy to do as possible and and people don't it won't interfere with their regular work , then maybe that would be good . And if you 're interested in using CVS , I 've set it up here , phd b: Oh great . grad c:  j phd b: I used it a long time ago but it 's been a while so maybe I can ask you some questions . Maybe you and I can talk a little bit at some point about coming up with a better  demonstration of the effects of reverberation for our web page , cuz  the    , actually the the  It made a good good audio demonstration because when we could play that clip the the the really obvious difference is that you can hear two voices and in the second one and only hear phd b: Maybe we could just like , talk into a cup . professor a: No , it sound it sounds pretty reverberant , but  you can't when you play it back in a room with a you know a big room , nobody can hear that difference really . professor a: They hear that it 's lower amplitude and they hear there 's a second voice , grad c:  - huh . professor a:  but  that actually that makes for a perfectly good demo because that 's a real obvious thing , that you hear two voices . But for the the visual , just , you know , I 'd like to have   , you know , the spectrogram again , grad c: Yeah . professor a: because you 're you 're you 're visual  abilities as a human being are so good you can pick out you know , you you look at the good one , you look at the cru the screwed up one , and and you can see the features in it without trying to @ @ phd b: I noticed that in the pictures . phd b: I thought " hey , you know th " I My initial thought was " this is not too bad ! " professor a: Right . But you have to you know , if you look at it closely , you see " well , here 's a place where this one has a big formant   formant maj major formants here are are moving quite a bit . professor a: So  you could that 's why I was thinking , in a section like that , you could take a look look at just that part of the spectrogram and you could say " Oh yeah . The main thing that struck me in looking at those two spectrograms was the difference in the high frequencies . It looked like for the one that was farther away , you know , it really everything was attenuated professor a: Right . Since you 're getting all this indirect energy , then a lot of it does have have  reduced high frequencies . But  the other thing is the temporal courses of things really are changed , and and  we want to show that , in some obvious way . " but I I just  After after  they were put in there I didn't really look at them anymore , cuz I just they were different . So maybe we can just substitute one of these wave forms and  then do some kind of zoom in on the spectrogram on an interesting area . professor a: The other thing that we had in there that I didn't like was that  the most obvious characteristic of the difference  when you listen to it is that there 's a second voice , and the the the the the  cuts that we have there actually don't correspond to the full wave form . It 's just the first I think there was something where he was having some trouble getting so much in , or . But it it 's  it 's the first six seconds or something of it and it 's in the seventh or eighth second or something where @ @ the second voice comes in . So we we would like to actually see the voice coming in , too , I think , since that 's the most obvious thing when you listen to it . phd f: So the thing that we did is just to add spectral subtraction before this , the Wall  process , which contains LDA on - line normalization . phd f: And so we started to look at at  things like this , which is , well , it 's Yeah . And the sentence contain only one word , which is " Due " And it can't clearly be seen . phd f: Where is the word ? phd b: This is this is , grad e:  . phd f: This is a plot of C - zero ,  when we don't use spectral substraction , and when there is no on - line normalization . phd f:  then when we apply mean normalization it looks like the second figure , though it is not . phd f: and And then the third figure is what happens when we apply mean normalization and variance normalization . What we can clearly see is that on the speech portion the two channel come becomes very close , but also what happens on the noisy portion is that the variance of the noise is professor a:   phd b: Can I ask  what does variance normalization do ? w What is the effect of that ? professor a: Normalizes the variance . phd f: So it phd b: No , I understand that , phd f: You you get an estimate of the standard deviation . phd f:  phd b: No , I understand what it is , but  , what does it what 's what is phd f: Yeah but . professor a: Well , because everything  If you have a system based on Gaussians , everything is based on means and variances . professor a: So if there 's an overall reason You know , it 's like  if you were doing  image processing and in some of the pictures you were looking at ,  there was a lot of light  and and in some , there was low light , phd b:   professor a: And the variance is just sort of like the next moment , you know ? So  what if  one set of pictures was taken  so that throughout the course it was went through daylight and night    ten times , another time it went thr  i is , you know , how how much how much vari phd b: Oh , OK . I guess a better example would be how much of the light was coming in from outside rather than artificial light . So if it was a lot if more was coming from outside , then there 'd be the bigger effect of the of the of the change in the So every mean every all all of the the parameters that you have , especially the variances , are going to be affected by the overall variance . professor a: And so , in principle , you if you remove that source , then , you know , you can phd b: I see . So would the major effect is that you 're gonna get is by normalizing the means , professor a: That 's the first order but thing , phd b: but it may help First - order effects . professor a: but then the second order is is the variances phd b: And it may help to do the variance . professor a: because , again , if you if you 're trying to distinguish between E and B phd b: OK . professor a: if it just so happens that the E 's were a more you know , were recorded when when the energy was was was larger or something , phd b:    professor a: or the variation in it was larger ,  than with the B 's , then this will be give you some some bias . phd b:  professor a: So the it 's removing these sources of variability in the data that have nothing to do with the linguistic component . And it and this professor a: i is if If you have a good voice activity detector , isn't isn't it gonna pull that out ? phd f: Yeah . Well what it it shows is that , yeah , perhaps a good voice activity detector is is good before on - line normalization and that 's what  we 've already observed . phd b: I don't know , it seems like this would be a lot easier than this signal to work with . What I notice is that , while I prefer to look at the second figure than at the third one , well , because you clearly see where speech is . phd f: But the problem is that on the speech portion , channel zero and channel one are more different than when you use variance normalization where channel zero and channel one become closer . phd b: But for the purposes of finding the speech phd f: And Yeah , but here phd b: You 're more interested in the difference between the speech and the nonspeech , phd f: Yeah . For I th I think that it perhaps it shows that  the parameters that the voice activity detector should use  have to use should be different than the parameter that have to be used for speech recognition . phd f: Well , y professor a: So you can do that by doing the voi voice activity detection . You also could do it by spect  spectral subtraction before the variance normalization , right ? phd f: Yeah , but it 's not clear , yeah . phd f: the the number that at that are here are recognition experiments on Italian HM and  with these two kinds of parameters . phd b: Where 's th phd f: But the fact is that the voice activity detector doesn't work on channel one . phd b: Where at what stage is the voice activity detector applied ? Is it applied here or a after the variance normalization ? phd f:  ? professor a: Spectral subtraction , I guess . Is it applied all the way back here ? phd f: It 's applied the  on , yeah , something like this , phd b: Maybe that 's why it doesn't work for channel one . professor a: Can I phd f: So we could perhaps do just mean normalization before VAD . Can I ask a ,  a sort of top - level question , which is  " if if most of what the OGI folk are working with is trying to integrate this other other  spectral subtraction , why are we worrying about it ? " phd f:   phd f: It 's just  Well it 's another They are trying to u to use the  the Ericsson and we 're trying to use something something else . When we do spectral subtraction , actually , I think that this is the the two last figures . phd b: Speech is more what ? phd f: Well , the difference between the energy of the speech and the energy of the n spectral subtrac subtracted noise portion is is larger . phd f: Well , if you compare the first figure to this one Actually the scale is not the same , but if you look at the the numbers  you clearly see that the difference between the C - zero of the speech and C - zero of the noise portion is larger .  but what happens is that after spectral subtraction , you also increase the variance of this of C - zero . And what they did at OGI is just  they don't use on - line normalization , for the moment , on spectral subtraction and I think Yeah . So yeah , we 're working on the same thing but I think  with different different system and professor a: Right .  , i the Intellectually it 's interesting to work on things th  one way or the other phd f:   professor a: but I 'm I 'm just wondering if  on the list of things that there are to do , if there are things that we won't do because we 've got two groups doing the same thing . phd b: if  you know , if if if you work on something else and and you 're waiting for them to give you spectral subtraction  it 's hard to know whether the effects that you get from the other experiments you do will carry over once you then bring in their spectral subtraction module . I don't know if that 's true or not , but I could see how phd f: Mmm .  , we still evidently have a latency reduction plan which which isn't quite what you 'd like it to be . And then  weren't issues of of having a a second stream or something ? That was Was it There was this business that , you know , we we could use up the full forty - eight hundred bits , and phd f: Yeah . And they are t I think they want to work on the second stream also , but more with some kind of multi - band or , well , what they call TRAP or generalized TRAP . Do you remember when the next meeting is supposed to be ? the next  phd f: It 's  in June . Yeah , the other thing is that you saw that that mail about  the VAD V A Ds performing quite differently ? That that  So  . This there was this experiment of  " what if we just take the baseline ? " phd f: Mmm . professor a: set  of features , just mel cepstra , and you inc incorporate the different V A And it looks like the the French VAD is actually  better significantly better . If the use the small VAD I th I think it 's on I think it 's easy to do better because it doesn't work at all . He Actually , I think that he say with the good VAD of from OGI and with the Alcatel VAD . phd f: Yeah but I it 's  I think you were talking about the other mail that used VAD on the reference features . professor a:  it was enough better that that it would  account for a fair amount of the difference between our performance , actually . phd f: And perhaps we can easily improve if if we put like mean normalization before the before the VAD . professor a: H Hynek will be back in town  the week after next , back back in the country . phd d: Also is Stephane was thinking that maybe it was useful to f to think about  voiced - unvoiced phd f:   phd f: Yeah , my feeling is that  actually when we look at all the proposals , ev everybody is still using some kind of spectral envelope professor a: Right . phd f: Yeah , well , not pitch , but to look at the  fine at the at the high re high resolution spectrum . We don't necessarily want to find the the pitch of the of the sound but  Cuz I have a feeling that when we look when we look at the just at the envelope there is no way you can tell if it 's voiced and unvoiced , if there is some It 's it 's easy in clean speech because voiced sound are more low frequency and . phd f:  there is the first formant , which is the larger and then voiced sound are more high frequencies cuz it 's frication and professor a: Right . When you have noise there is no  if if you have a low frequency noise it could be taken for for voiced speech and . professor a: but but phd b: Isn't there some other phd f: S phd b:  d phd f: So I think that it it would be good Yeah , yeah , well , go go on . phd b: I was just gonna say isn't there aren't aren't there lots of ideas for doing voice activity , or speech - nonspeech rather ,  by looking at  , you know ,  I guess harmonics or looking across time professor a: Well , I think he was talking about the voiced - unvoiced , though , phd f: Mmm . phd b:  w ah you know ,  even with the voiced - non voiced - unvoiced phd f: Mmm . phd f:  yeah , so yeah , I think if we try to develop a second stream well , there would be one stream that is the envelope and the second , it could be interesting to have that 's something that 's more related to the fine structure of the spectrum . We were thinking about like using ideas from from Larry Saul , have a good voice detector , have a good , well , voiced - speech detector , that 's working on on the FFT and  professor a: U phd f: Larry Saul could be an idea . We were are thinking about just kind of  taking the spectrum and computing the variance of of the high resolution spectrum and things like this .  we had a guy here some years ago who did some work on  making use of voicing information  to help in reducing the noise . phd f: Yeah ? professor a: So what he was doing is basically y you you do estimate the pitch . professor a: And  you from that you you estimate or you estimate fine harmonic structure , whichev ei either way , it 's more or less the same . But  the thing is that  you then can get rid of things that are not i if there is strong harmonic structure , you can throw away stuff that 's that 's non - harmonic . professor a: And that that is another way of getting rid of part of the noise phd f: Yeah . It was kind of like RASTA was taking care of convolutional stuff phd f: Mmm . We 've actually back when Chuck was here we did some voiced - unvoiced  classification using a bunch of these , phd f: But professor a: and and  works OK . professor a: But the thing is that you can't given the constraints of this task , we can't , in a very nice way , feed forward to the recognizer the information the probabilistic information that you might get about whether it 's voiced or unvoiced , where w we can't you know affect the the  distributions or anything . phd b: Didn't the head dude send around that message ? Yeah , I think you sent us all a copy of the message , where he was saying that I I 'm not sure , exactly , what the gist of what he was saying , but something having to do with the voice activity detector and that it will that people shouldn't put their own in or something . professor a: I guess what you could do , maybe this would be w useful , if if you have if you view the second stream , yeah , before you before you do KLT 's and so forth , if you do view it as probabilities , and if it 's an independent So , if it 's if it 's  not so much envelope - based by fine - structure - based ,  looking at harmonicity or something like that ,  if you get a probability from that information and then multiply it by you know , multiply by all the voiced outputs and all the unvoiced outputs , you know , then use that as the phd f:   professor a:  take the log of that or  pre pre  pre - nonlinearity , phd f: Yeah . i if professor a:  and do the KLT on the on on that , phd f: Yeah . And then that would be phd f: Yeah , well , I was not thinking this yeah , this could be an yeah So you mean have some kind of probability for the v the voicing professor a: R Right . If you have a tandem system and then you have some kind of it can be pretty small net phd f:   professor a: and the and and you use the thing is to use information primarily that 's different as you say , it 's more fine - structure - based than than envelope - based phd f:   professor a:  so then it you you you can pretty much guarantee it 's stuff that you 're not looking at very well with the other one , and  then you only use for this one distinction . professor a: And and so now you 've got a probability of the cases , and you 've got  the probability of the finer  categories on the other side . professor a: if they really are from independent information sources then they should have different kinds of errors phd f:   professor a: and roughly independent errors , and it 's a good choice for phd f:   Because , yeah , well , spectral subtraction is good and we could u we could use the fine structure to to have a better estimate of the noise but still there is this issue with spectral subtraction that it seems to increase the variance of of of professor a: Yeah . phd f:  Well it 's this musical noise which is annoying if you d you do some kind of on - line normalization after . Spectral subtraction and on - line normalization don't seem to to go together very well . I professor a: Or if you do a spectral subtraction do some spectral subtraction first and then do some on - line normalization then do some more spectral subtraction  , maybe maybe you can do it layers or something so it doesn't doesn't hurt too much or something . professor a: But it but  , anyway I think I was sort of arguing against myself there by giving that example phd f: Yeah . professor a:   cuz I was already sort of suggesting that we should be careful about not spending too much time on exactly what they 're doing In fact if you get if you go into  a  harmonics - related thing it 's definitely going to be different than what they 're doing and   phd f:   I know that when have people have done  sort of the obvious thing of taking  your feature vector and adding in some variables which are pitch related or  that it hasn't my impression it hasn't particularly helped . professor a: But I think  that 's that 's a question for this  you know extending the feature vector versus having different streams . Was it nois noisy condition ? the example that you you just professor a: And and it may not have been noisy conditions . I I don't remember the example but it was it was on some DARPA data and some years ago and so it probably wasn't , actually phd f:   But we were thinking , we discussed with Barry about this , and perhaps thinking we were thinking about some kind of sheet cheating experiment where we would use TIMIT professor a:  - huh . phd f: and see if giving the d  , this voicing bit would help in in terms of  frame classification . professor a: Why don't you why don't you just do it with Aurora ? phd f: Mmm . professor a: Just any i in in each in each frame phd f: Yeah , but but B but we cannot do the cheating , this cheating thing . Cuz we don't have Well , for Italian perhaps we have , but we don't have this labeling for Aurora . professor a: But you could  you can you can align so that It 's not perfect , but if you if you know what was said and phd b: But the problem is that their models are all word level models . phd b: You So you could find out where the word boundaries are but that 's about it . grad e: S But we could use  the the noisy version that TIMIT , which you know , is similar to the the noises found in the TI - digits  portion of Aurora . phd f: Well , I guess I guess we can we can say that it will help , but I don't know . If this voicing bit doesn't help , I think we don't have to to work more about this because professor a:  . professor a: and different ta it was probably Resource Management or something , I think you were getting something like still eight or nine percent error on the voicing , as I recall . professor a: what that said is that , sort of , left to its own devices , like without the a strong language model and so forth , that you would you would make significant number of errors just with your  probabilistic machinery in deciding phd b: It also professor a: one oh phd b: Yeah , the though I think  there was one problem with that in that , you know , we used canonical mapping so our truth may not have really been true to the acoustics . Well back twenty years ago when I did this voiced - unvoiced stuff , we were getting more like ninety - seven or ninety - eight percent correct in voicing . professor a: And we did this complex feature selection thing where we looked at all the different possible features one could have for voicing and and and  and exhaustively searched all size subsets and and  for for that particular speaker and you 'd find you know the five or six features which really did well on them . professor a: And then doing doing all of that we could get down to two or three percent error . professor a: So I would I would believe that  it was quite likely that  looking at envelope only , that we 'd be significantly worse than that . phd f: And the all the the SpeechCorders ? what 's the idea behind ? Cuz they they have to Oh , they don't even have to detect voiced spe speech ? professor a: The modern ones don't do a a simple switch . phd f: They just work on the code book professor a: They work on the code book excitation . They try they they try every every possible excitation they have in their code book and find the one that matches best . One of the ideas that we had come up with last week for things to try to improve the system  . Actually I I s we didn't I guess I wrote this in after the meeting b but the thought I had was  looking at the language model that 's used in the HTK recognizer , which is basically just a big loop , grad e:   phd b: and then that can be either go to silence or go to another digit , which That model would allow for the production of infinitely long sequences of digits , right ? professor a: Right . I thought " well I 'm gonna just look at the what actual digit strings do occur in the training data . phd b: And the interesting thing was it turns out that there are no sequences of two - long or three - long digit strings in any of the Aurora training data . So it 's either one , four , five , six ,  up to eleven , and then it skips and then there 's some at sixteen . So I I just for the heck of it , I made a little grammar which  , you know , had it 's separate path for each length digit string you could get . So there was a one - long path and there was a four - long and a five - long professor a:   phd b: So it was you know , I I didn't have any weights of these paths or I didn't have anything like that . phd b: And I played with tweaking the word transition penalties a bunch , but I couldn't go anywhere . I thought " well if I only allow " Yeah , I guess I should have looked at to see how often there was a mistake where a two - long or a three - long path was actually put out as a hypothesis . phd b: So to do that right you 'd probably want to have allow for them all but then have weightings and things


What are we talking about today ? phd e: well , first there are perhaps these  Meeting Recorder digits that we tested . professor a: And for one thing that that sure shows the difference between having a lot of  training data or not , phd e: Of data ? Yeah . professor a: the  The best kind of number we have on the English  on near microphone only is is  three or four percent . professor a: And  it 's significantly better than that , using fairly simple front - ends on on the   , with the SRI system . professor a: So I th I think that the  But that 's that 's using  a a pretty huge amount of data , mostly not digits , of course , but but then again Well , yeah . In fact , mostly not digits for the actual training the H M Ms whereas  in this case we 're just using digits for training the H M phd e: Yeah . professor a: Did anybody mention about whether the the SRI system is a is is doing the digits  the wor as a word model or as  a sub s sub - phone states ? phd e: I guess it 's it 's  allophone models , professor a: Yeah . There is one difference Well , the SRI system the result for the SRI system that are represented here are with adaptation . So there is It 's their complete system and including on - line  unsupervised adaptation . phd e: And if you don't use adaptation , the error rate is around fifty percent worse , I think , if I remember . professor a: It 's tha it 's that much , huh ? phd e: Nnn . professor a: But but  what what I think I 'd be interested to do given that , is that we we should  take I guess that somebody 's gonna do this , right ? is to take some of these tandem things and feed it into the SRI system , right ? phd e: Yeah . Our back - end is is fairly simple but until now , well , the attempts to improve it or have fail Ah , well ,   what Chuck tried to to to do professor a: Yeah , but he 's doing it with the same data , right ?  so to So there 's there 's there 's two things being affected . One is that that , you know , there 's something simple that 's wrong with the back - end . professor a:  I I don't know if he got to the point of playing with the  number of Gaussians yet phd e:   So , yeah , we could retrain some of these tandem on on huge professor a: Well , you could do that , but I 'm saying even with it not with that part not retrained , just just using having the H M Ms much better H M phd e: Ah , yeah . But what would be interesting to see also is what what perhaps it 's not related , the amount of data but the  recording conditions . Because it 's probably not a problem of noise , because our features are supposed to be robust to noise . phd e: It 's not a problem of channel , because there is  normalization with respect to the channel . What what is the problem that you 're trying to explain ? phd e: The the fact that the result with the tandem and Aurora system are  so much worse . professor a: I  but I 'm I 'm almost certain that it it  , that it has to do with the  amount of training data . professor a: But but having a huge If if you look at what commercial places do , they use a huge amount of data .  , ordinarily you would say " well , given that you have enough occurrences of the digits , you can just train with digits rather than with , you know " phd e:   professor a: But the thing is , if you have a huge in other words , do word models But if you have a huge amount of data then you 're going to have many occurrences of similar  allophones . professor a: So it 's  I I think it has to be that , because , as you say , this is , you know , this is near - microphone , phd e:   Now , some of it could be the fact that  let 's see , in the in these multi - train things did we include noisy data in the training ? phd e: Yeah . Well , actually we see that the clean train for the Aurora proposals are are better than the multi - train , professor a: It is if Yeah . phd e: Well , o I guess what I meant is that well , let 's say if we if we add enough data to train on the  on the Meeting Recorder digits , I guess we could have better results than this . What I meant is that perhaps we can learn something  from this , what 's what 's wrong  what what is different between TI - digits and these digits and professor a: What kind of numbers are we getting on TI - digits ? phd e: It 's point eight percent , so . professor a: So in the actual TI - digits database we 're getting point eight percent , phd e: Yeah . professor a: and here we 're getting three or four three , let 's see , three for this ? phd e:   Sure , but  ,  point eight percent is something like double  or triple what people have gotten who 've worked very hard at doing that . professor a: And and also , as you point out , there 's adaptation in these numbers also . So if you , you know , put the ad adap take the adaptation off , then it for the English - Near you get something like two percent . professor a: It 's , you know , we used a simple HTK system with a modest amount of data . And this is a a , you know , modern  system  has has a lot of nice points to it . professor a: But to me it just it just meant a practical point that  if we want to publish results on digits that that people pay attention to we probably should  Cuz we 've had the problem before that you get show some nice improvement on something that 's that 's  ,  it seems like too large a number , and   people don't necessarily take it so seriously . So the three point four percent for this  is is  So why is it It 's an interesting question though , still . Why is why is it three point four percent for the d the digits recorded in this environment as opposed to the  point eight percent for for for the original TI - digits database ?  . So ignore ignoring the the the SRI system for a moment , phd e: I I I don't I   professor a: just looking at the TI - di the  tandem system , if we 're getting point eight percent , which , yes , it 's high . It 's , you know , it it 's not awfully high , phd e:   professor a: Right ?  , there 's even though it 's close - miked there 's still there really is background noise . And  I suspect when the TI - digits were recorded if somebody fumbled or said something wrong or something that they probably made them take it over . professor a: It was not  there was no attempt to have it be realistic in any in any sense at all . TI - digit is it 's very , very clean and it 's like studio recording professor a:   But professor a: It 's I think it 's it 's the indication it 's harder .  , they 're much much better , but still you 're getting something like one point three percent for  things that are same data as in T TI - digits the same same text . And  , I 'm sure the same same system would would get , you know , point point three or point four or something on the actual TI - digits . professor a: Which I find sort of interesting cause I think this is closer to   it 's still read . But I still think it 's much closer to to what what people actually face ,  when they 're they 're dealing with people saying digits over the telephone . I don't think   , I 'm sure they wouldn't release the numbers , but I don't think that  the  the the companies that that do telephone speech get anything like point four percent on their digits . I 'm I 'm I 'm sure they get  , for one thing people do phone up who don't have   Middle America accents and it 's a we we it 's it 's it 's US . professor a: Did we end up giving up on on , any Eurospeech submissions , phd e: But professor a: or ? I know Thilo and Dan Ellis are are submitting something , but  . I I guess e the only thing with these the Meeting Recorder and , well , So , I think , yeah I think we basically gave up . Now , actually for the for the Aur -  phd e: But professor a: we do have stuff for Aurora , right ? Because because we have ano an extra month or something . professor a: and we have We don't we don't have to flood it with papers . Perhaps the point is that we 've been working on is , yeah , we have put the  the good VAD in the system and it really makes a huge difference . I think , yeah , this is perhaps one of the reason why our system was not not the best , because with the new VAD , it 's very the results are similar to the France Telecom results and perhaps even better sometimes . The problem is that it 's very big and we still have to think how to where to put it and  , professor a:   phd e: because it it well , this VAD  either some delay and we if we put it on the server side , it doesn't work , because on the server side features you already have LDA applied from the f from the terminal side and so you accumulate the delay so the VAD should be before the LDA which means perhaps on the terminal side and then smaller and professor a: So wha where did this good VAD come from ? phd e: So . So it 's the network trained it 's the network with the huge amounts on hidden of hidden units , and  nine input frames compared to the VAD that was in the proposal which has a very small amount of hidden units and fewer inputs . Yeah , but they had to get rid of it because of the space , didn't they ? phd e: Yeah . But the abso assumption is that we will be able to make a VAD that 's small and that works fine .  ,  i if if there 's a if if I I don't know what the thinking was amongst the the the the ETSI folk but  if everybody agreed sure let 's use this VAD and take that out of there phd e:   They just want , apparently they don't want to fix the VAD because they think there is some interaction between feature extraction and and VAD or frame dropping But they still want to just to give some  requirement for this VAD because it 's it will not be part of they don't want it to be part of the standard . But I was thinking that that  s " Sure , there may be some interaction , phd e: Nnn . professor a: but I don't think we need to be stuck on using our or OGI 's VAD .  I designed a new a new filter because when I designed other filters with shorter delay from the LDA filters , there was one filter with fif sixty millisecond delay and the other with ten milliseconds professor a: Right . phd e: and  Hynek suggested that both could have sixty - five sixty - s I think it 's sixty - five . Both should have sixty - five because professor a: You didn't gain anything , right ? phd e: Yeah . phd e: Yeah , and then we 've started to work with this of  voiced - unvoiced stuff . phd e: And next week I think we will perhaps try to have  a new system with   MSG stream also see what what happens . phd d: No , I w I begin to play with Matlab and to found some parameter robust for voiced - unvoiced decision . And we they we found that maybe w is a classical parameter , the sq the variance between the  FFT of the signal and the small spectrum of time we after the  mel filter bank . So , basically we wa want to look at something like the ex the ex excitation signal and professor a: Right . phd d: The the mix of the two , noise and unnoise , and the signal is this . professor a: Well , I 'm s  There 's None of these axes are labeled , so I don't know what this What 's this axis ? phd d:  this is  this axis is nnn , " frame " . professor a: And what 's th what this ? phd d: this is  energy , log - energy of the spectrum . Of the this is the variance , the difference between the spectrum of the signal and FFT of each frame of the signal and this mouth spectrum of time after the f may fit for the two , professor a: For this one . phd d: Pity , but I don't have two different professor a: And presumably when there 's a a phd e: So this should the the the t voiced portions . This is trying to obtain with LPC model the spectrum but using Matlab without going factor and s professor a: No pre - emphasis ? Yeah . So now I wonder  , do you want to I know you want to get at something orthogonal from what you get with the smooth spectrum  . But if you were to really try and get a voiced - unvoiced , do you do you want to totally ignore that ?  , do you do you  , clearly a a very big very big cues for voiced - unvoiced come from  spectral slope and so on , right ? phd e:   phd d: if s @ @ val value is indicative that is a voice frame and it 's low values professor a: Yeah . Well , you probably want  , certainly if you want to do good voiced - unvoiced detection , you need a few features . But , you know , people look at at slope and  first auto - correlation coefficient , divided by power . professor a: Or or   there 's  I guess we prob probably don't have enough computation to do a simple pitch detector or something ?  with a pitch detector you could have a have a an estimate of of what the phd e: Mmm . Or maybe you could you just do it going through the P FFT 's figuring out some  probable  harmonic structure . phd d: you have read up and you have a paper , the paper that you s give me yesterday . But Yeah , but it 's not it 's , yeah , it 's it 's another problem . phd e: What 's this again ? Is it the mel - filters ? phd d: Yeah like this . phd e: and what we clearly see is that in some cases , and it clearly appears here , and the the harmonics are resolved by the f Well , there are still appear after mel - filtering , professor a:   phd e: and it happens for high pitched voice because the width of the lower frequency mel - filters is sometimes even smaller than the pitch . phd e: And so what happens is that this  , add additional variability to this envelope and  professor a: Yeah . phd e: so we were thinking to modify the mel - spectrum to have something that that 's smoother on low frequencies . What I was talking about was just , starting with the FFT you could you could  do a very rough thing to estimate estimate  pitch . professor a: And   , given you know , given that ,  you could   come up with some kind of estimate of how much of the low frequency energy was was explained by by   those harmonics . And and so if you if you just you know subtracted off  your guess of the harmonics then something like this would end up with quite a bit lower energy in the first fifteen hundred hertz or so and and our first kilohertz , even . professor a: And  if was  noisy , the proportion that it would go down would be if it was if it was unvoiced or something . What 's up with you ? grad b:  our t I went to talk with  Mike Jordan this this week professor a:   grad b:  and  shared with him the ideas about  extending the Larry Saul work and  I asked him some questions about factorial H M so like later down the line when we 've come up with these these feature detectors , how do we how do we  you know ,  model the time series that that happens  and and we talked a little bit about factorial H M Ms and how  when you 're doing inference or w when you 're doing recognition , there 's like simple Viterbi stuff that you can do for for these H M and the  the great advantages that  a lot of times the factorial H M Ms don't  don't over - alert the problem there they have a limited number of parameters and they focus directly on on  the sub - problems at hand so you can imagine  five or so parallel  features  transitioning independently and then at the end you you  couple these factorial H M Ms with  with  undirected links  based on based on some more data . grad b: So he he seemed he seemed like really interested in in  in this and said said this is this is something very do - able and can learn a lot and  yeah , I 've just been continue reading  about certain things . grad b:  thinking of maybe using   m modulation spectrum stuff to  as features  also in the in the sub - bands professor a:   grad b: because it seems like the modulation  spectrum tells you a lot about the intelligibility of of certain  words and stuff So ,  . And  so I 've been looking at Avendano 's work and   I 'll try to write up in my next stat status report a nice description of what he 's doing , but it 's it 's an approach to deal with reverberation or that the aspect of his work that I 'm interested in the idea is that  normally an analysis frames are  too short to encompass reverberation effects  in full . You miss most of the reverberation tail in a ten millisecond window and so you you 'd like it to be that  the reverberation responses  simply convolved  in , but it 's not really with these ten millisecond frames cuz you j But if you take , say , a two millisecond  window I 'm sorry a two second window then in a room like this , most of the reverberation response is included in the window and the then it  then things are l more linear . It is it is more like the reverberation response is simply c convolved and  and you can use channel normalization techniques like  in his thesis he 's assuming that the reverberation response is fixed . He just does  mean subtraction , which is like removing the DC component of the modulation spectrum and that 's supposed to d  deal  deal pretty well with the  reverberation and  the neat thing is you can't take these two second frames and feed them to a speech recognizer  so he does this  method training trading the  the spectral resolution for time resolution and  come ca  synthesizes a new representation which is with say ten second frames but a lower s  frequency resolution . I guess it 's these are called " time frequency representations " and h he 's making the the time sh  finer grained and the frequency resolution  less fine grained . grad c: s so I 'm I guess my first stab actually in continuing his work is to  re - implement this this thing which  changes the time and frequency resolutions cuz he doesn't have code for me . grad c: Oh , and  , another f first step is  , so the the way I want to extend his work is make it able to deal with a time varying reverberation response  and  we don't really know how fast the  the reverberation response is varying the Meeting Recorder data  so  we we have this  block least squares  imp echo canceller implementation and  I want to try finding the the response , say , between a near mike and the table mike for someone using the echo canceller and looking at the echo canceller taps and then see how fast that varies from block to block . grad c: That should give an idea of how fast the reverberation response is changing . S so  y you do I think you read some of the the zeros as O 's and some as zeros . grad c: Is there a particular way we 're supposed to read them ? phd e: There are only zeros here . " O " " O " " O " " O " " O " " O " and " zero " are two ways that we say that digit . professor a: So it 's grad b: Ha ! phd e: But professor a: so it 's i phd e: Perhaps in the sheets there should be another sign for the if we want to the the guy to say " O " or professor a: No . professor a:  in digit recognition we 've done before , you have you have two pronunciations for that value , " O " and " zero " . phd e: But it 's perhaps more difficult for the people to prepare the database then , if because here you only have zeros professor a: No , they just write phd e: and and people pronounce " O " or zero professor a: they they write down OH . phd e: Yeah but if the sh the sheet was prepared with a different sign for the " O " .  , you 'd have to tell them " OK when we write this , say it tha " , phd e: OK . professor a: you know , and you just They just want people to read the digits as you ordinarily would phd e:   Is this a change from the last batch of of  forms ? Because in the last batch it was spelled out which one you should read . It was it was spelled out , and they decided they wanted to get at more the way people would really say things . professor a: That 's also why they 're they 're bunched together in these different groups . Actually , let me just s since since you brought it up , I was just it was hard not to be self - conscious about that when it after we since we just discussed it . But I realized that that  when I 'm talking on the phone , certainly , and and saying these numbers , I almost always say zero . So that that that 's the habit I 'm in , but some people say " O " and grad b: Yeah I normally say " O " cuz it 's easier to say