professor b: So what what from what grad a: Hello ? professor b: Whatever we say from now on , it can be held against us , right ? phd e: That 's right . So I I the the problem is that I actually don't know how th these held meetings are held , if they are very informal and sort of just people are say what 's going on phd e: Yeah . phd e: We just sorta go around and people say what 's going on , what 's the latest  professor b: Yeah . So I guess that what may be a reasonable is if I  first make a report on what 's happening in Aurora in general , at least what from my perspective . professor b: And and  so , I I think that Carmen and Stephane reported on  Amsterdam meeting , phd d:  o professor b: which was kind of interesting because it was for the first time we realized we are not friends really , but we are competitors . phd e: right ? that they were trying to decide ? professor b: There is a plenty of there 're plenty of issues . phd e: Like the voice activity detector , professor b: Well and what happened was that they realized that if two leading proposals , which was French Telecom Alcatel , and us both had  voice activity detector . And I said " well big surprise ,  we could have told you that n n n four months ago , except we didn't because nobody else was bringing it up " . professor b: Obviously French Telecom didn't volunteer this information either , cuz we were working on mainly on voice activity detector for past  several months phd e: Right . professor b: I said " well yeah , you are absolutely right ,  if I wish that you provided better end point at speech because  or at least that if we could modify the recognizer ,  to account for these long silences , because otherwise  that that th that wasn't a correct thing . " And so then ev ev everybody else says " well we should we need to do a new eval evaluation without voice activity detector , or we have to do something about it " . Because  but in that case ,  we would like to change the  the algorithm because  if we are working on different data , we probably will use a different set of tricks . professor b: But unfortunately nobody ever officially can somehow acknowledge that this can be done , because French Telecom was saying " no , no , no , now everybody has access to our code , so everybody is going to copy what we did . " Yeah well our argument was everybody ha has access to our code , and everybody always had access to our code . We thought that people are honest , that if you copy something and if it is protected protected by patent then you negotiate , or something , phd e: Yeah . professor b: But And French Telecom was saying " no , no , no , phd e:   professor b: there is a lot of little tricks which  sort of  cannot be protected and you guys will take them , " which probably is also true .  , you know , it might be that people will take   th the algorithms apart and use the blocks from that . But I somehow think that it wouldn't be so bad , as long as people are happy abou    honest about it . professor b: And I think they have to be honest in the long run , because winning proposal again  what will be available th is will be a code . So the  the people can go to code and say " well listen this is what you stole from me " phd e:   The biggest problem of course is that f that Alcatel French Telecom cl claims " well we fulfilled the conditions . " And e and other people don't feel that , because they so they now decided that that is the whole thing will be done on well - endpointed data , essentially that somebody will endpoint the data based on clean speech , because most of this the SpeechDat - Car has the also close speaking mike and endpoints will be provided . professor b: And  we will run again still not clear if we are going to run the if we are allowed to run   new algorithms , but I assume so .  but since  u u n u at least our experience is that only endpointing a a mel cepstrum gets  gets you twenty - one percent improvement overall and twenty - seven improvement on SpeechDat - Car phd e:  . professor b: So they agreed that  there will be a twenty - five percent improvement required on on  h u m bad mis badly mismatched phd e: But wait a minute , I thought the endpointing really only helped in the noisy cases . professor b: Yeah but you have the same prob  MFCC basically has an enormous number of  insertions . professor b: And so , so now they want to say " we we will require fifty percent improvement only for well matched condition , and only twenty - five percent for the serial cases . professor b: And  and they almost agreed on that except that it wasn't a hundred percent agreed . And so last time  during the meeting , I just  brought up the issue , I said " well you know  quite frankly I 'm surprised how lightly you are making these decisions because this is a major decision . For two years we are fighting for fifty percent improvement and suddenly you are saying " oh no we we will do something less " , but maybe we should discuss that . And everybody said " oh we discussed that and you were not a mee there " and I said " well a lot of other people were not there because not everybody participates at these teleconferencing c things . " However , there is only ten or fifteen lines , so people can't even con you know participate . " Immediately Nokia  raised the question and they said " oh yeah we agree this is not good to to  dissolve the   the  the criterion . professor b: So now officially , Nokia is   complaining and said they they are looking for support ,  I think QualComm is  saying , too " we shouldn't abandon the fifty percent yet . professor b: Next Wednesday we are going to have  another  teleconferencing call , so we 'll see what  where it goes . phd e: So what about the issue of  the weights on the for the different systems , the well - matched , and medium - mismatched and professor b: Yeah , that 's what that 's a g very good  point , because David says " well you know we ca we can manipulate this number by choosing the right weights anyways . professor b:  yeah , if of course if you put a zero  weight zero on a mismatched condition , or highly mismatched then then you are done . So phd e: And they 're the staying the same ? professor b: Well , of course people will not like it . Now What is happening now is that I th I think that people try to match the criterion to solution . But it 's should happen at a point where everybody feels comfortable that we did all what we could . professor b: Basically , I think that that this test was a little bit bogus because of the data and  essentially there were these arbitrary decisions made , and and everything . So what we are doing at OGI now is    working basically on our parts which we I think a little bit neglected , like noise separation .  so we are looking in ways is in  which  with which we can provide better initial estimate of the mel spectrum basically , which would be a l  , f more robust to noise , and so far not much  success . professor b: We tried  things which  a long time ago Bill Byrne suggested , instead of using Fourier spectrum , from Fourier transform , use the spectrum from LPC model . Their argument there was the LPC model fits the peaks of the spectrum , so it may be m naturally more robust in noise . And I thought " well , that makes sense , " but so far we can't get much much out of it . professor b:  we may try some standard techniques like spectral subtraction and phd e: You haven't tried that yet ? professor b: not not not much . professor b: like for instance  Dennis Klatt was suggesting  the one way to  deal with noisy speech is to add noise to everything . phd e: Oh ! professor b: So that makes  th any additive noise less addi less a a effective , phd e: I see . It was kind of like one of these things , you know , but if you think about it , it 's actually pretty ingenious . So well , you know , just take a take a spectrum and and and add of the constant , C , to every every value . And if if then if this data becomes noisy , it b it becomes eff effectively becomes less noisy basically . professor b: But of course you cannot add too much noise because then you 'll s then you 're clean recognition goes down , but  it 's yet to be seen how much , it 's a very simple technique . professor b: Yes indeed it 's a very simple technique , you just take your spectrum and and use whatever is coming from FFT , add constant , phd e:  . That that Or the other thing is of course if you have a spectrum , what you can s start doing , you can leave start leaving out the p the parts which are   low in energy and then perhaps  one could try to find a a all - pole model to such a spectrum . Because a all - pole model will still try to to to put the the continuation basically of the of the model into these parts where the issue set to zero . phd e:  ! What is that ? professor b: Ah , you don't know about TRAPS ! grad a:  . phd e: The TRAPS sound familiar , I but I don't professor b: Yeah  tha This is familiar like sort of because we gave you the name , but , what it is , is that normally what you do is that you recognize  speech based on a shortened spectrum . professor b: Essentially L P - LPC , mel cepstrum , everything starts with a spectral slice .  so if you s So , given the spectrogram you essentially are sliding sliding the spectrogram along the  f frequency axis phd e:   professor b: So you can say " well you can also take the time trajectory of the energy at a given frequency " , and what you get is then , that you get a p vector . Namely you can say i it I will I will say that this vector will eh will will describe the phoneme which is in the center of the vector . professor b: And you so you classi so it 's a very different vector , very different properties , we don't know much about it , but the truth is phd e:  . But you have many of those vectors per phoneme , professor b: Well , so you get many decisions . professor b: Because if you run this  recognition , you get you still get about twenty percent error  twenty percent correct . professor b: on on like for the frame by frame basis , so   so it 's much better than chance . But  the latest  observation  is that you you you are you can get quite a big advantage of using two critical bands at the same time . Because there are some reasons I can I could talk about , will have to tell you about things like masking experiments which     yield critical bands , and also experiments with release of masking , which actually tell you that something is happening across critical bands , across bands . And phd e: Well how do you how do you  convert this  energy over time in a particular frequency band into a vector of numbers ? professor b: It 's     time T - zero is one number , time t phd e: Yeah but what 's the number ? Is it just the professor b: It 's a spectral energy , logarithmic spectral energy , phd e: it 's just the amount of energy in that band from f in that time interval . professor b: And that 's what that 's what I 'm saying then , so this is a this is a starting vector . professor b: for instance a question is like " how correlated are the elements of this vector ? " Turns out they are quite correlated , because  , especially the neighboring ones , right ? They they represent the same almost the same configuration of the vocal tract . professor b: Then the question is  " can you describe elements of this vector by Gaussian distributions " , or to what extent ? Because  And and and so on and so on . professor b: But  is the is the critical band the right   dimension ? So we somehow made arbitrary decision , " yes " . Then but then now we are thinking a lot how to  how to use at least the neighboring band because that seems to be happening This I somehow start to believe that 's what 's happening in recognition . Cuz a lot of experiments point to the fact that people can split the signal into critical bands , but then oh   so you can you are quite capable of processing a signal in   independently in individual critical bands . But at the same time you most likely pay attention to at least neighboring bands when you are making any decisions , you compare what 's happening in in this band to what 's happening to the band to to to the to the neighboring bands . That 's why the articulatory events , which  F F Fletcher talks about , they are about two critical bands . professor b: You need to you need to compare it to something else , what 's happening but it 's what 's happening in the in the close neighborhood . So if you are making decision what 's happening at one kilohertz , you want to know what 's happening at nine hundred hertz and it and maybe at eleven hundred hertz , but you don't much care what 's happening at three kilohertz . phd e: So it 's really w It 's sort of like saying that what 's happening at one kilohertz depends on what 's happening around it . But it 's but for but for instance , th   what what  humans are very much capable of doing is that if th if they are exactly the same thing happening in two neighboring critical bands , recognition can discard it . grad a: Hey ! professor b: Hey ! OK , we need us another another voice here . professor b: And so so so for instance if you d if you a if you add the noise that normally masks masks the  the the signal right ? phd e:   professor b: and you can show that in that if the if you add the noise outside the critical band , that doesn't affect the the decisions you 're making about a signal within a critical band . If the noise is modulated , with the same modulation frequency as the noise in a critical band , the amount of masking is less . professor b: So the s m masking curve , normally it looks like sort of I start from from here , so you you have  no noise then you you you are expanding the critical band , so the amount of maching is increasing . And when you e hit a certain point , which is a critical band , then the amount of masking is the same . professor b: But , if you if you if you modulate the noise , the masking goes up and the moment you start hitting the another critical band , the masking goes down . So essentially essentially that 's a very clear indication that that that cognition can take   into consideration what 's happening in the neighboring bands . But if you go too far in a in a if you if the noise is very broad , you are not increasing much more , so so if you if you are far away from the signal  from the signal f  the frequency at which the signal is , then the m even the when the noise is co - modulated it it 's not helping you much . So things like this we are kind of playing with with with the hope that perhaps we could eventually u use this in a in a real recognizer . phd e: But you probably won't have anything before the next time we have to evaluate , professor b: Probably not . phd e: right ? professor b: Well , maybe , most likely we will not have anything which c would comply with the rules . professor b: latency currently chops the require  significant  latency amount of processing , phd e:   professor b: because  we don't know any better , yet , than to use the neural net classifiers ,  and  and  TRAPS . professor b: Though the the work which  everybody is looking at now aims at s trying to find out what to do with these vectors , so that a g simple Gaussian classifier would be happier with it . professor b: or to what extent a Gaussian classifier should be unhappy  that , and how to Gaussian - ize the vectors , and phd e:  . Then Sunil is    asked me f for one month 's vacation and since he did not take any vacation for two years , I had no I didn't have heart to tell him no . professor b: And  phd e: Is he getting married or something ? professor b:  well , he may be looking for a girl , for for I don't I don't I don't ask . Well , I 've known other friends who they they go to Ind - they go back home to India for a month , they come back married , professor b: Yeah . professor b: and then of course then what happened with Narayanan was that he start pushing me that he needs to get a PHD because they wouldn't give him his wife . And she 's very pretty and he loves her and so so we had to really phd e: So he finally had some incentive to finish , professor b: Oh yeah . phd e: huh ? professor b: Sort of figured that That was a  that he  he told me the day when we did very well at our NIST evaluations of speaker recognition , the technology , and he was involved there . So I I said " well , yeah , OK " so he took another another three quarter of the year but  he was out . phd e: huh ? professor b: So I wouldn't surprise me if he has a plan like that , though though  Pratibha still needs to get out first . professor b: And S and Satya needs to get out very first because he 's he already has  four years served , though one year he was getting masters . phd e: So have the  when is the next  evaluation ? June or something ? professor b: Which ? Speaker recognition ? phd e: No , for  Aurora ? professor b:  there , we don't know about evaluation , next meeting is in June . But I  , yeah , what I think would be of course extremely useful , if we can come to our next meeting and say " well you know we did get fifty percent improvement . If if you are interested we eventually can tell you how " , but  we can get fifty percent improvement . Do you know what the new baseline is ? Oh , I guess if you don't have professor b: Twenty - two t twenty twenty - two percent better than the old baseline . But I assume that it will be similar , I don't I I don't see the reason why it shouldn't be . professor b: Cuz if it is worse , then we will raise the objection , phd e: Yeah . professor b: we say " well you know how come ? " Because eh if we just use our voice activity detector , which we don't claim even that it 's wonderful , it 's just like one of them . professor b: We get this sort of improvement , how come that we don't see it on on on on your endpointed data ? phd c: Yeah . phd c: because the voice activity detector that I choosed is something that cheating , it 's using the alignment of the speech recognition system , professor b: Yeah . C yeah  phd c: and only the alignment on the clean channel , and then mapped this alignment to the noisy channel . professor b: Well David told me David told me yesterday or Harry actually he told Harry from QualComm and Harry  brought up the suggestion we should still go for fifty percent he says are you aware that your system does only thirty percent  comparing to to endpointed baselines ? So they must have run already something . But  we think that we we didn't say the last word yet , that we have other other things which we can try . Because Nokia was objecting , with  QualComm 's we basically supported that , we said " yes " . "  The Guenter Hirsch who d doesn't speak for Ericsson anymore because he is not with Ericsson and Ericsson may not may withdraw from the whole Aurora activity because they have so many troubles now . phd e: Where 's  Guenter going ? professor b: Well Guenter is already he got the job  already was working on it for past two years or three years phd e:   professor b: he got a job  at some some Fachschule , the technical college not too far from Aachen . phd e:  ! professor b: So it 's like professor u university professor phd e:   professor b: you know , not quite a university , not quite a sort of it 's not Aachen University , but it 's a good school and he he 's happy .  ! professor b: And he well , he was hoping to work  with Ericsson like on t  like consulting basis , but right now he says says it doesn't look like that anybody is even thinking about speech recognition . But this is being now discussed right now , and it 's possible that  that that it may get through , that we will still stick to fifty percent . Which event es essentially I think that we should be happy with because that that would mean that at least people may be forced to look into alternative solutions phd c:   professor b: but not phd c: Which would mean like sixty percent over the current baseline , which is professor b: Yeah . professor b: Is it like sort of is How did you come up with this number ? If you improve twenty by twenty percent the c the f the all baselines , it 's just a quick c comp co computation ? phd c: Yeah . phd c: Yeah , because it de it depends on the weightings professor b: Yeah , yeah . How 's your documentation or whatever it w what was it you guys were working on last week ? phd c: Yeah , finally we we 've not finished with this . phd d: Ma - nec to need a little more time to improve the English , and maybe s to fill in something some small detail , something like that , phd c:   Well , we have a document that explain a big part of the experiments , phd d: Necessary to to include the bi the bibliography . phd e: So have you been running some new experiments ? I I thought I saw some jobs of yours running on some of the machine phd c: Yeah . We 've fff done some strange things like removing C - zero or C - one from the the vector of parameters , and we noticed that C - one is almost not useful at all . phd e: Eh Is this in the baseline ? or in  phd c: In the No , in the proposal . professor b: So we were just discussing , since you mentioned that , in it w phd c:   professor b: driving in the car with Morgan this morning , we were discussing a good experiment for b for beginning graduate student who wants to run a lot of who wants to get a lot of numbers on something phd c:   professor b: which is , like , " imagine that you will you will start putting every co any coefficient , which you are using in your vector , in some general power . professor b: So if you put it in a s square root , that effectively makes your model half as efficient . professor b: And and  i i i but it 's the mean is an exponent of the whatever , the the this Gaussian function . phd e: You 're compressing the range , professor b: So you 're compressing the range of this coefficient , so it 's becoming less efficient . Morgan was @ @ and he was he was saying well this might be the alternative way how to play with a with a fudge factor , you know ,  in the phd e: Oh . professor b: And I said " well in that case why don't we just start compressing individual elements , like when when because in old days we were doing when when people still were doing template matching and Euclidean distances , we were doing this liftering of parameters , right ? phd e:  - huh . professor b: and it 's highly affected by  frequency response of the of the recording equipment and that sort of thing , phd c:   professor b:  Bell Labs had he this   r raised cosine lifter which still I think is built into H HTK for reasons n unknown to anybody , but but  we had exponential lifter , or triangle lifter , basic number of lifters . But so they may be a way to to fiddle with the f with the f phd e: Insertions . professor b: Insertions , deletions , or the the giving a relative  basically modifying relative importance of the various parameters . professor b: The only of course problem is that there 's an infinite number of combinations and if the if you s if y phd e: Oh . You need like a some kind of a professor b: Yeah , you need a lot of graduate students , and a lot of computing power . phd e: You need to have a genetic algorithm , that basically tries random permutations of these things . If you were at Bell Labs or I d d I shouldn't be saying this in on on a mike , right ? Or I  IBM , that 's what maybe that 's what somebody would be doing . professor b: Oh ,  the places which have a lot of computing power , so because it is really it 's a p it 's a it 's it will be reasonable search phd e:   professor b:  but I wonder if there isn't some way of doing this  search like when we are searching say for best discriminants . phd e: You know actually , I don't know that this wouldn't be all that bad . phd e: right ? And then these exponents are just applied to that professor b: Absolutely . Each each phd e: And is this something that you would adjust for training ? or only recognition ? professor b: For both , you would have to do . phd e: So you 'd actually professor b: Because essentially you are saying "  this feature is not important " . professor b: Or less important , so that 's th that 's a that 's a painful one , yeah . phd e: So for each  set of exponents that you would try , it would require a training and a recognition ? professor b: Yeah . You just may n may need to c  give  less weight to to  a mod  a component of the model which represents this particular feature . So if you Instead of altering the feature vectors themselves , you you modify the the the Gaussians in the models . You modify the Gaussian in the model , but in the in the test data you would have to put it in the power , but in a training what you c in a training  in trained model , all you would have to do is to multiply a model by appropriate constant . But why if you 're if you 're multi if you 're altering the model , why w in the test data , why would you have to muck with the  cepstral coefficients ? professor b: Because in  test in  test data you ca don't have a model . professor b: That is true , but w  , so what you want to do You want to say if  obs you if you observe something like Stephane observes , that C - one is not important , you can do two things . professor b: If you have a trained trained recognizer , in the model , you know the the the the component which I  di dimension wh phd e:   All of the all of the mean and variances that correspond to C - one , you put them to zero . But what I 'm proposing now , if it is important but not as important , you multiply it by point one in a model . professor b: But but but phd e: But what are you multiplying ? Cuz those are means , right ? grad a: You 're multiplying the standard deviation ? phd e:  you 're grad a: So it 's professor b: I think that you multiply the I would I would have to look in the in the math ,  how how does the model  phd e: I think you professor b: Yeah . phd e: Yeah , I think you 'd have to modify the standard deviation or something , so that you make it wider or narrower . professor b: Effectively you , you know y in f in front of the of the model , you put a constant . grad a: right ? professor b: And and and phd e: So by making th the standard deviation narrower ,  your scores get worse for professor b: Yeah . By making it narrower , phd e: Right ? professor b:  y your phd e:  there 's you 're you 're allowing for less variance . Because see what you are fitting is the multidimensional Gaussian , right ? phd e:   professor b: It 's a it has it has  thirty - nine dimensions , or thirteen dimensions if you g ignore deltas and double - deltas . professor b: So in order if you in order to make dimension which which Stephane sees  less important ,    not not useful , less important , what you do is that this particular component in the model you can multiply by w you can you can basically de - weight it in the model . But you can't do it in a in a test data because you don't have a model for th   when the test comes , but what you can do is that you put this particular component in and and you compress it . phd e: Couldn't you just do that to the test data and not do anything with your training data ? professor b: That would be very bad , because  your t your model was trained  expecting  , that wouldn't work . After you train the model , you sort of y you could do you could do still what I was proposing initially , that during the training you you compress C - one that becomes then it becomes less important in a training . professor b: But if you have if you want to run e ex extensive experiment without retraining the model , you don't have to retrain the model . But after , you wh when you are doing this parametric study of importance of C - one you will de - weight the C - one component in the model , and you will put in the you will compress the this component in a in the test data . phd e: Could you also if you wanted to if you wanted to try an experiment  by leaving out say , C - one , couldn't you , in your test data ,  modify the all of the C - one values to be  way outside of the normal range of the Gaussian for C - one that was trained in the model ? So that effectively , the C - one never really contributes to the score ? phd c:   professor b: No , that would be a severe mismatch , phd e: Do you know what I 'm say professor b: right ? what you are proposing ? N no you don't want that . But what if you set if to the mean of the model , then ? And it was a cons you set all C - ones coming in through your test data , you you change whatever value that was there to the mean that your model had . professor b: I see what you are sa saying , phd c: Right ? grad a: Saying .   , no , the If you set it to a mean , that would No , you can't do that . phd e: Oh , that 's true , right , yeah , because you you have phd c: Wait . Which professor b: Because that would be a really f fiddling with the data , phd e: Yeah .  professor b: But what you can do , I 'm confident you ca phd e:  professor b: well , I 'm reasonably confident and I putting it on the record , right ?  y people will listen to it for for centuries now , is what you can do , is you train the model  with the with the original data . So what you will do is that a component in the model for C - one , you will divide it by by two . Then if you think that some component is more is more important then th th th it then then   i it is , based on training , then you  multiply this particular component in the model by by by phd e: You 're talking about the standard deviation ? professor b: yeah . professor b: Yeah , multiply this component  i it by number b larger than one , phd e:   phd c: Yeah , but , at the phd e: But don't you have to do something to the mean , also ? professor b: No . phd c: But I think it 's  the The variance is on on the denominator in the in the Gaussian equation . If you want to decrease the importance of a c parameter , you have to increase it 's variance . phd e: And now you 're you 're you 're changing that by squaring it . phd e: Do you see what  ? phd c: I think What I see What could be done is you don't change your features , which are computed once for all , professor b:  - huh . phd c: And then if you want to decrease the importance of C - one you just take the variance of the C - one component in the in the model and increase it if you want to decrease the importance of C - one or decrease it phd e: Yeah . Yeah , but  , but it 's it 's i it 's do - able , phd c: Well . phd e: to get  this  this the effect I think that you 're talking about , professor b:   phd e: Yeah , because if you had a huge variance , you 're dividing by a large number , you get a very small contribution . grad a: Doesn't matter phd c: Yeah , it becomes more flat grad a: Right . grad a: Yeah , the sharper the variance , the more more important to get that one right . phd e: Yeah , you know actually , this reminds me of something that happened  when I was at BBN . phd e: And this particular pitch algorithm  when it didn't think there was any voicing , was spitting out zeros . So we were getting  when we did clustering , we were getting groups  of features professor b: p Pretty new outliers , interesting outliers , right ? phd e: yeah , with with a mean of zero and basically zero variance . phd e: So , when ener when anytime any one of those vectors came in that had a zero in it , we got a great score . phd e: So if you have very small variance you get really good scores when you get something that matches . So that 's a way , yeah , yeah That 's a way to increase the yeah , n That 's interesting . phd e: You you have a step where you you modify the models , make a d copy of your models with whatever variance modifications you make , and rerun recognition . phd e: That could be set up fairly easily I think , and you have a whole bunch of you know professor b: Chuck is getting himself in trouble . Huh ! grad a: Didn't you say you got these  HTK 's set up on the new Linux boxes ? phd e: That 's right . professor b: Hey ! phd e: In fact , and and they 're just t right now they 're installing  increasing the memory on that  the Linux box . professor b: And Chuck is sort of really fishing for how to keep his computer busy , grad a: Right . professor b: that 's yeah , that 's a good thing grad a: That 's right . professor b: because then y you just write the " do " - loops and then you pretend that you are working while you are sort of you c you can go fishing . Then you are sort of in this mode like all of those ARPA people are , right ? phd e: Yeah . professor b: since it is on the record , I can't say  which company it was , but it was reported to me that  somebody visited a company and during a d during a discussion , there was this guy who was always hitting the carriage returns  on a computer . professor b: So after two hours  the visitor said " wh why are you hitting this carriage return ? " And he said " well you know , we are being paid by a computer ty  we are we have a government contract . " It was in old days when there were  of PDP - eights and that sort of thing . phd e: Oh , my gosh ! So he had to make it look like professor b: Because so they had a they literally had to c monitor at the time at the time on a computer how much time is being spent I i i or on on this particular project . phd e: Have you ever seen those little  It 's it 's this thing that 's the shape of a bird and it has a red ball and its beak dips into the water ? professor b: Yeah , I know , right . professor b: It would be similar similar to I knew some people who were  that was in old Communist  Czechoslovakia , right ? so we were watching for American airplanes , coming to spy on on  on us at the time , phd e:   professor b: so there were three guys   stationed in the middle of the woods on one l lonely  watching tower , pretty much spending a year and a half there because there was this service right ? And so they very quickly they made friends with local girls and local people in the village phd e: Ugh ! professor b: and phd e: Yeah . professor b: and so but they there was one plane flying over s always   above , and so that was the only work which they had . They like four in the afternoon they had to report there was a plane from Prague to Brno Basically f flying there , phd e: Yeah . professor b: so they f very q f first thing was that they would always run back and and at four o ' clock and and quickly make a call , " this plane is   passing " then a second thing was that they they took the line from this u u post to   a local pub . And they but third thing which they made , and when they screwed up , they finally they had to p the the p the pub owner to make these phone calls because they didn't even bother to be there anymore . At least they were sort of smart enough that they looked if the plane is flying there , right ? And the pub owner says " oh my four o ' clock , OK , quickly p pick up the phone , call that there 's a plane flying . professor b: There was no plane for some reason , phd e: And there wasn't ? professor b: it was downed , or and so they got in trouble . phd e: And we 'll just professor b: Well , at least go test the s test the  assumption about C - C - one  to begin with . It might be that  phd e: Yeah , so the first set of  variance weighting vectors would be just you know one modifying one and leaving the others the same . professor b: Because you see , what is happening here in a in a in a in such a model is that it 's tells you yeah what has a low variance  is  is  is more reliable , phd e: That would be one set of experiment professor b: right ? How do we phd e: Wh - yeah , when the data matches that , then you get really professor b: Yeah . professor b: How do we know , especially when it comes to noise ? phd e: But there could just naturally be low variance . professor b: Yeah ? phd e: Because I Like , I 've noticed in the higher cepstral coefficients , the numbers seem to get smaller , right ? So d phd c: They t phd e: just naturally . professor b: Yeah that 's why  people used these lifters were inverse variance weighting lifters basically that makes   Euclidean distance more like  Mahalanobis distance with a diagonal covariance when you knew what all the variances were over the old data . Turns out that  the variance decreases at least at fast , I believe , as the index of the cepstral coefficients . professor b: So typically what happens is that you you need to weight the  weight the higher coefficients more than  the lower coefficients . When we talked about Aurora still I wanted to m make a plea  encourage for  more communication between between   different  parts of the distributed   center .  even when there is absolutely nothing to to s to say but the weather is good in Ore - in in Berkeley . I 'm sure that it 's being appreciated in Oregon and maybe it will generate similar responses down here , like ,  phd c: We can set up a webcam maybe . phd e: Is the  if we mail to " Aurora - inhouse " , does that go up to you guys also ? professor b: I don't think so . professor b: We should definitely set up phd e: Yeah we sh Do we have a mailing list that includes  the OGI people ? professor b: Yeah . And then we also can send the the dis to the same address right , and it goes to everybody phd e:   professor b: Because what 's happening naturally in research , I know , is that people essentially start working on something and they don't want to be much bothered , right ? but what the the then the danger is in a group like this , is that two people are working on the same thing and i c of course both of them come with the s very good solution , but it could have been done somehow in half of the effort or something . reasonably  good one , because he 's doing it for Intel , but I trust that we have  rights to  use it  or distribute it and everything . phd e:  ! professor b: u s And so so  we we will make sure that at least you can see the software and if if if if it is of any use . He says " well you know it 's very difficult to collaborate if you are working with supposedly the same thing , in quotes , except which is not s is not the same . professor b: Which which   one is using that set of hurdles , another one set is using another set of hurdles . professor b: Yeah because Intel paid us  should I say on a microphone ?  some amount of money , not much . And they wanted  to to have software so that they can also play with it , which means that it has to be in a certain environment phd e:  . professor b: they use actu actually some Intel libraries , but in the process , Lucash just rewrote the whole thing because he figured rather than trying to f make sense  of  including ICSI software  not for training on the nets phd e:  . professor b: but I think he rewrote the the the or so maybe somehow reused over the parts of the thing so that so that the whole thing , including MLP , trained MLP is one piece of  software . professor b: Yeah ? grad a: I remember when we were trying to put together all the ICSI software for the submission . He said that it was like it was like just so many libraries and nobody knew what was used when , and and so that 's where he started and that 's where he realized that it needs to be needs to be   at least cleaned up , grad a: Yeah . Well , the the only thing I would check is if he does he use Intel math libraries , professor b:  e ev phd c: because if it 's the case , it 's maybe not so easy to use it on another architecture . professor b: n not maybe Maybe not in a first maybe not in a first ap approximation because I think he started first just with a plain C C or C - plus - plus or something before phd c: Ah yeah . I never checked carefully these sorts of professor b: I know there was some issues that initially of course we d do all the development on Linux but we use we don't have we have only three     s SUNs and we have them only because they have a SPERT board in . In that way Intel succeeded with us , because they gave us too many good machines for very little money or nothing . phd e: The way that it works is each person goes around in turn , and  you say the transcript number and then you read the digits , the the strings of numbers as individual digits . phd e: So you don't say " eight hundred and fifty " , you say " eight five oh " , and so forth


professor c: so you get to phd d: Yeah , I will try to explain the thing that I did this this week during this week . phd d: Well eh you know that I work I begin to work with a new feature to detect voice - unvoice . phd d: What I trying two MLP to to the with this new feature and the fifteen feature  from the eh bus base system phd e: The the mel cepstrum ? phd d: No , satly the mes the Mel Cepstrum , the new base system the new base system . phd d: And I 'm trying two MLP , one one that only have t three output , voice , unvoice , and silence , professor c:   And I tried to do some experiment of recognition with that and only have result with with the MLP with the three output . professor c: What what feeds the  the three - output net ? phd d: Voice , unvoice , and si professor c: No no , what feeds it ? What features does it see ? phd d: The feature the input ? The inputs are the fifteen the fifteen  bases feature . And the other three features are R , the variance of the difference between the two spectrum , professor c:  - huh . phd d: the variance of the auto - correlation function , except the the first point , because half the height value is R - zero professor c:   professor c: You wouldn't do like R - one over R - zero or something like that ?  usually for voiced - unvoiced you 'd do yeah , you 'd do something you 'd do energy phd d: Yeah . professor c: but then you have something like spectral slope , which is you get like R - one ov over R - zero or something like that . phd d: Auto - correlation ? Yes , yes , the variance of the auto - correlation function that uses that professor c: Ye - Well that 's the variance , but if you just say " what is "  , to first order ,  yeah one of the differences between voiced , unvoiced and silence is energy . phd d: Yeah , I I 'll The spectral shape , professor c: Yeah , and so R - one over R - zero is what you typically use for that . No , I don't use that I can't use professor c: No , I 'm saying that 's what people us typically use . professor c: See , because it because this is this is just like a single number to tell you  " does the spectrum look like that or does it look like that " . professor c: So if it 's if it 's  if it 's low energy  but the but the spectrum looks like that or like that , it 's probably silence . professor c:  but if it 's low energy and the spectrum looks like that , it 's probably unvoiced . professor c: So if you just if you just had to pick two features to determine voiced - unvoiced , you 'd pick something about the spectrum like  R - one over R - zero ,  and R - zero phd d:  -  , OK . professor c: or i i you know you 'd have some other energy measure and like in the old days people did like  zero crossing counts .  , phd d: Bec - because the result are a little bit better but we have in a point that everything is more or less the similar more or less similar . professor c: Right , but it seemed to me that what you were what you were getting at before was that there is something about the difference between the original signal or the original FFT and with the filter which is what and the variance was one take  on it . Then in that case , if you have two nets , Alright , and this one has three outputs , and this one has f phd d:   professor c: if you were to sum up the probabilities for the voiced and for the unvoiced and for the silence here , we 've found in the past you 'll do better at voiced - unvoiced - silence than you do with this one . professor c: So  phd d: No phd e: So you 're saying take the features that go into the voiced - unvoiced - silence net and feed those into the other one , as additional inputs , rather than having a separate professor c: w W well that 's another way . professor c: That wasn't what I was saying but yeah that 's certainly another thing to do . No I was just trying to say if you b if you bring this into the picture over this , what more does it buy you ? phd e: Mmm . professor c: And what I was saying is that the only thing I think that it buys you is  based on whether you feed it something different . And so the kind of thing that that she was talking about before , was looking at something  ab  something  about the difference between the the   log FFT  log power  and the log magnitude  F F - spectrum  and the   filter bank . professor c: And so the filter bank is chosen in fact to sort of integrate out the effects of pitch and she 's saying you know trying So the particular measure that she chose was the variance of this m of this difference , but that might not be the right number . professor c: Right ?  maybe there 's something about the variance that 's that 's not enough or maybe there 's something else that that one could use , but I think that , for me , the thing that that struck me was that  you wanna get something back here , so here 's here 's an idea .  What about it you skip all the all the really clever things , and just fed the log magnitude spectrum into this ? phd d: Ah I 'm sorry . professor c: This is f You have the log magnitude spectrum , and you were looking at that and the difference between the filter bank and and c c computing the variance . professor c: What if you stopped being clever ? And you just took this thing in here because it 's a neural net and neural nets are wonderful phd d:   professor c: and figure out what they can what they most need from things , and  that 's what they 're good at . professor c: So  you 're you 're you 're trying to be clever and say what 's the statistic that should we should get about this difference but  in fact , you know maybe just feeding this in or or feeding both of them in phd e:  . professor c: you know , another way , saying let it figure out what 's the what is the interaction , especially if you do this over multiple frames ? phd d:   professor c: Then you have this over time , and and both kinds of measures and  you might get  something better . phd e: So so don't  don't do the division , but let the net have everything .  , it seems to me , if you have exactly the right thing then it 's better to do it without the net because otherwise you 're asking the net to learn this you know , say if you wanted to learn how to do multiplication . professor c:  you could feed it a bunch of s you could feed two numbers that you wanted to multiply into a net and have a bunch of nonlinearities in the middle and train it to get the product of the output and it would work . But , it 's kind of crazy , cuz we know how to multiply and you you 'd be you know much lower error usually if you just multiplied it out . phd e: How long does it take , Carmen , to train up one of these nets ? phd d: Oh , not too much . grad a: What are what are your f  frame error rates for for this ? phd d: Eh fifty - f six  no , the frame error rate ? grad a: O phd d: Fifty - six I think . grad a: Fif - fifty - six percent accurate for v voice - unvoice phd d: The accuracy . No for , yes f I don't remember for voice - unvoice , grad a: Oh , OK . phd d: But I think that fifty - five was for the when the output are the fifty - six phone . phd d: That I look in the with the other nnn the other MLP that we have are more or less the same number . professor c: I think at the frame level for fifty - six that was the kind of number we were getting for for   reduced band width  stuff . phd d: I think that I I I think that for the other one , for the three output , is sixty sixty - two , sixty three more or less .  , if you 're getting fifty - six percent over here ,  that 's in noise also , right ? phd d: Yeah , yeah , yeah . If you 're getting fifty - six here , try adding together the probabilities of all of the voiced phones here and all of the unvoiced phones phd d: will be professor c: and see what you get then . phd d: Well I don't know , but I th I I think that we I have the result more or less . I don't I 'm not sure but I remember @ @ that I can't show that . professor c: OK , but that 's a That is a a good check point , you should do that anyway , phd d: Yeah . professor c: OK ? Given this this  regular old net that 's just for choosing for other purposes ,  add up the probabilities of the different subclasses and see see how well you do . But phd e: The targets for the neural net , they come from forced alignments ? phd d: no . Well there 's gonna be it looks like there 's gonna be a noisy  some large vocabulary noisy stuff too . phd e: Oh ! professor c: Yeah , so the   , the issue is whether people make a decision now based on what they 've already seen , or they make it later . And one of the arguments for making it later is let 's make sure that whatever techniques that we 're using work for something more than than connected digits . phd e: When are they planning When would they do that ? professor c: Mmm , I think late  I think in the summer sometime . phd d: and also mmm I H Hynek last week say that if I have time I can to begin to to study well seriously the France Telecom proposal professor c:   phd d: to look at the code and something like that to know exactly what they are doing because maybe that we can have some ideas professor c:   Look insi look i carefully what they are doing with the program @ @ and I begin to to work also in that . But the first thing that I don't understand is that they are using R - the  log energy that this quite I don't know why they have some constant in the expression of the lower energy . professor c: Oh , at the front it says  " log energy is equal to the rounded version of sixteen over the log of two " phd d: This Yeah . professor c: Well , this is natural log , and maybe it has something to do with the fact that this is I I have no idea . phd e: Is that some kind of base conversion , or ? professor c: Yeah , that 's what I was thinking , but but  , then there 's the sixty - four , I don't know . phd d: Because maybe they 're the threshold that they are using on the basis of this value phd e: Experimental results . phd d: I don't know exactly , because well th I thought maybe they have a meaning . phd e: So they 're taking the number inside the log and raising it to sixteen over log base two . If we ignore the sixteen , the natural log of t one over the natural log of two times the natu I don't know . Well , maybe somebody 'll think of something , phd e:  professor c: but this is  It may just be that they they want to have for very small energies , they want to have some kind of a phd d: Yeah , the e The effect I don't @ @ I can understand the effect of this , no ? because it 's to to do something like that . professor c: Well , it says , since you 're taking a natural log , it says that when when you get down to essentially zero energy , this is gonna be the natural log of one , which is zero . professor c: So it 'll go down to  to the natural log being So the lowest value for this would be zero . I I will look to try if I move this parameter in their code what happens , maybe everything is Maybe they tres hole are on basis of this . professor c:   it they they probably have some fi particular s fixed point arithmetic that they 're using , phd d: I don't know . professor c: and then it just phd e: Yeah , I was just gonna say maybe it has something to do with hardware , professor c: Yeah . I think you 're supposed to on this stuff anyway , and and so maybe that puts it in the right realm somewhere . phd e: Well it just , yeah , puts it in the right range , or professor c: Yeah . I think , given at the level you 're doing things in floating point on the computer , I don't think it matters , would be my guess , phd d:   OK , and wh when did Stephane take off ? He took off phd d: I think that Stephane will arrive today or tomorrow . professor c: Oh , he was gone these first few days , and then he 's here for a couple days before he goes to Salt Lake City . I I don't know if there are many people who are going to ICASSP phd d: Yeah . phd e: Do have Have people sort of stopped going to ICASSP in recent years ? professor c: people are less consistent about going to ICASSP and I think it 's still it 's still a reasonable forum for students to to present things .  , it 's I think for engineering students of any kind , I think it 's it 's if you haven't been there much , it 's good to go to ,  to get a feel for things , a range of things , not just speech . But I think for for sort of dyed - in - the - wool speech people ,  I think that ICSLP and Eurospeech are much more targeted . And then there 's these other meetings , like HLT and and  ASRU phd e:  professor c: so there 's there 's actually plenty of meetings that are really relevant to to  computational  speech processing of one sort or another . So  Wanna talk a little bit about what we were talking about this morning ? grad a: Oh !   Yeah . I I guess some of the progress , I I 've been getting a getting my committee members for the quals . And  so far I have Morgan and Hynek , Mike Jordan , and I asked John Ohala and he agreed . Then  I talked a little bit about  continuing with these dynamic ev  acoustic events , and  we 're we 're we 're thinking about a way to test the completeness of a a set of  dynamic  events .  , completeness in the in the sense that  if we if we pick these X number of acoustic events , do they provide sufficient coverage for the phones that we 're trying to recognize or or the f the words that we 're gonna try to recognize later on . And so Morgan and I were  discussing  s  s a form of a cheating experiment where we get  we have   a chosen set of features , or acoustic events , and we train up a hybrid  system to do phone recognition on TIMIT . So i i the idea is if we get good phone recognition results , using  these set of acoustic events , then  that that says that these acoustic events are g sufficient to cover a set of phones , at least found in TIMIT .  so i it would be a a measure of " are we on the right track with with the the choices of our acoustic events " . And also , just  working on my  final project for Jordan 's class ,  which is professor c: Actually , let me grad a: Yeah . The the other thing I was suggesting , though , is that given that you 're talking about binary features , maybe the first thing to do is just to count and  count co - occurrences and get probabilities for a discrete  cuz that 'd be pretty simple because it 's just Say , if you had ten ten events ,  that you were counting ,  each frame would only have a thousand possible values for these ten bits , and  so you could make a table that would say , if you had thirty - nine phone categories , that would be a thousand by thirty - nine , and just count the co - occurrences and divide them by the the    occ  count the co - occurrences between the event and the phone and divide them by the number of occurrences of the phone , and that would give you the likelihood of the of the event given the phone . And  then just use that in a very simple  and  you could  do phone recognition then and  wouldn't have any of the issues of the  training of the net or  , it 'd be on the simple side , but phd e:   professor c:   you know , if   the example I was giving was that if if you had  onset of voicing and and end of voicing as being two kinds of events , then if you had those a all marked correctly , and you counted co - occurrences , you should get it completely right . So   If you just do this by counting , then you should be able to find out in a pretty straightforward way whether you have a sufficient  set of events to to do the kind of level of of  classification of phones that you 'd like . And then the other thing that we were discussing was was  OK , how do you get the your training data . professor c: Cuz  the Switchboard transcription project   you know was half a dozen people , or so working off and on over a couple years , and  similar similar amount of data to what you 're talking about with TIMIT training . So , it seems to me that the only reasonable starting point is  to automatically translate the  current TIMIT markings into the markings you want . And  it won't have the kind of characteristic that you 'd like , of catching funny kind of things that maybe aren't there from these automatic markings , phd e:   professor c: but but  it 's  phd e: It 's probably a good place to start . professor c: Yeah and a short short amount of time , just to again , just to see if that information is sufficient to  determine the phones . phd e: Yeah , you could even then to to get an idea about how different it is , you could maybe take some subset and you know , go through a few sentences , mark them by hand and then see how different it is from you know , the canonical ones , professor c: Right . phd e: just to get an idea a rough idea of h if it really even makes a difference . professor c:   my my guess would be that this is since TIMIT 's read speech that this would be less of a big deal , phd e:   professor c: And the other thing would be , say , if you had these ten events , you 'd wanna see , well what if you took two events or four events or ten events or t and you know , and and hopefully there should be some point at which having more information doesn't tell you really all that much more about what the phones are . professor c: you could , but the thing is , what he 's talking about here is a  a translation to a per - frame feature vector , so there 's no sequence in that , I think . I think it 's just a phd e: Unless you did like a second pass over it or something after you 've got your professor c: Yeah , but we 're just talking about something simple here , yeah , to see if phd e: Yeah . The idea is with a with a very simple statistical structure , could you could you  at least verify that you 've chosen features that are sufficient . professor c: OK , and you were saying something starting to say something else about your your class project , or ? grad a: Oh . grad a: So for my class project I 'm  I 'm tinkering with  support vector machines ? something that we learned in class , and   basically just another method for doing classification . And so I 'm gonna apply that to  compare it with the results by  King and Taylor who did  these  using recurrent neural nets , they recognized  a set of phonological features  and made a mapping from the MFCC 's to these phonological features , so I 'm gonna do a similar thing with with support vector machines and see if phd e: So what 's the advantage of support vector machines ? What grad a:  . grad a: and  so if you if you give it less data it still does a reasonable job in learning the the patterns . grad a:  and  professor c: I guess it yeah , they 're sort of succinct , and and they  grad a: Yeah . phd e: Does there some kind of a distance metric that they use or how do they for cla what do they do for classification ? grad a:  . So , the the simple idea behind a support vector machine is  , you have you have this feature space , right ? and then it finds the optimal separating plane ,  between these two different  classes , phd e:   grad a: and  and so  , what it i at the end of the day , what it actually does is it picks those examples of the features that are closest to the separating boundary , and remembers those phd e:   So , given these  these features , or or these these examples , critical examples , which they call support f support vectors , then  given a new example , if the new example falls  away from the boundary in one direction then it 's classified as being a part of this particular class phd e: Oh . phd e: So why save the examples ? Why not just save what the boundary itself is ? grad a:   professor c: You know , it it goes back to nearest - neighbor sort of thing , phd e:   professor c: right ?  , i i if is it eh w When is nearest - neighbor good ? Well , nearest - neighbor good is good if you have lots and lots of examples .  but of course if you have lots and lots of examples , then it can take a while to to use nearest - neighbor . So a long time ago people talked about things where you would have  a condensed nearest - neighbor , where you would you would you would pick out  some representative examples which would  be sufficient to represent to to correctly classify everything that came in . professor c: I I think s I think support vector stuff sort of goes back to to that kind of thing . So rather than doing nearest neighbor where you compare to every single one , you just pick a few critical ones , and professor c: Yeah . professor c: And th the You know ,  neural net approach  or Gaussian mixtures for that matter are sort of fairly brute force kinds of things , where you sort of you predefine that there is this big bunch of parameters and then you you place them as you best can to define the boundaries , and in fact , as you know , these things do take a lot of parameters and and  if you have  only a modest amount of data , you have trouble  learning them .  , so I I guess the idea to this is that it it is reputed to  be somewhat better in that regard . I it can be a a reduced  parameterization of of the the model by just keeping certain selected examples . professor c: But I don't know if people have done sort of careful comparisons of this on large tasks or anything . grad b: S do you get some kind of number between zero and one at the output ? grad a: Actually you don't get a you don't get a nice number between zero and one .  ,  there are there are pap Well , basically , it 's it 's  you you get a distance measure at the end of the day , and then that distance measure is is  is translated to a zero or one . professor c: But that 's looking at it for for classification for binary classification , grad a: That 's for classification , right . professor c: right ? phd e: And you get that for each class , you get a zero or a one . grad a: You have the distances to work with , professor c: Cuz actually Mississippi State people did use support vector machines for   speech recognition and they were using it to estimate probabilities . Yeah , they they had a had a way to translate the distances into into probabilities with the with the simple   sigmoidal function . professor c: Yeah , and d did they use sigmoid or a softmax type thing ? grad a:  Yeah , professor c: And didn't they like exponentiate or something grad a: there 's some there 's like one over one plus the exponential or something like that . professor c: and then divide by the sum of them , or ? Oh it i Oh , so it is a sigmoidal . phd e: Did the did they get good results with that ? professor c: they 're OK , I I don't I don't think they were earth earth shattering , but I think that  this was a couple years ago , phd e:  . professor c: I remember them doing it at some meeting , and and  I don't think people were very critical because it was interesting just to to try this and you know , it was the first time they tried it , so so the you know , the numbers were not incredibly good phd e:  . grad b:  s So Barry , if you just have zero and ones , how are you doing the speech recognition ? grad a: Oh I 'm not do I 'm not planning on doing speech recognition with it . grad a: So  for example , this this  feature set called the  sound patterns of English  is just a bunch of  binary valued features . Let 's say , is this voicing , or is this not voicing , is this sonorants , not sonorants , and stuff like that . phd e: Did you find any more mistakes in their tables ? grad a: Oh !  I haven't gone through the entire table , yet . Yeah , yesterday I brought Chuck the table and I was like , " wait , this is Is the mapping from N to to this phonological feature called  " coronal " , is is should it be shouldn't it be a one ? or should it should it be you know coronal instead of not coronal as it was labelled in the paper ? " So I ha haven't hunted down all the all the mistakes yet , professor c:  - huh . grad a: but professor c: But a as I was saying , people do get probabilities from these things , grad b: OK . professor c: and and  we were just trying to remember how they do , but people have used it for speech recognition , and they have gotten probabilities . professor c: There 's you have you have the paper , right ? The Mississippi State paper ? grad a:   professor c: Yeah , if you 're interested y you could look , grad b: And OK . grad a: yeah , our phd e: So in your in in the thing that you 're doing ,  you have a vector of ones and zeros for each phone ? grad a:    phd e: Is that what you 're grad a: Right , Right , right f so for every phone there is there is a  a vector of ones and zeros f  corresponding to whether it exhibits a particular phonological feature or not . And so when you do your wh I 'm what is the task for the class project ? To come up with the phones ? grad a:  phd e: or to come up with these vectors to see how closely they match the phones , grad a: Oh . Right ,  to come up with a mapping from  MFCC 's or s some feature set ,  to  w to whether there 's existence of a particular phonological feature . grad a: And  yeah , basically it 's to learn a mapping from from the MFCC 's to  phonological features . C phd e: I guess  ,  I 'm not sure what you what you 're what you get out of your system . Do you get out a  a vector of these ones and zeros and then try to find the closest matching phoneme to that vector , grad a:   Just it 's it 's basically it 's it 's really simple , basically a detection of phonological features . grad a: and  cuz the  So King and and Taylor  did this with  recurrent neural nets , phd e: Yeah . grad a: and this i their their idea was to first find a mapping from MFCC 's to  phonological features phd e:   grad a: and then later on , once you have these phonological features , then  map that to phones . grad a: So I 'm I 'm sort of reproducing phase one of their stuff . I wo did they compare that  , what if you just did phone recognition and did the reverse lookup . phd e: So you recognize a phone and which ever phone was recognized , you spit out it 's vector of ones and zeros . phd e:   professor c: That 's probably not what he 's going to do on his class project . professor c: So  have you had a chance to do this  thing we talked about yet with the   phd e: Insertion penalty ? professor c:  . No actually I was going a different That 's a good question , too , but I was gonna ask about the the  changes to the data in comparing PLP and mel cepstrum for the SRI system . Well what I 've been " Changes to the data " , I 'm not sure I professor c: Right . So we talked on the phone about this , that that there was still a difference of a of a few percent phd e: Yeah . And I was asking if you were going to do redo it  for PLP with the normalization done as it had been done for the mel cepstrum . phd e: What I 've been doing is  trying to figure out it just seems to me like there 's a  well it seems like there 's a bug , because the difference in performance is it 's not gigantic but it 's big enough that it it seems wrong . professor c: Yeah , I agree , but I thought that the normalization difference was one of the possibilities , phd e: and Yeah , but I don't I 'm not professor c: right ? phd e: Yeah , I guess I don't think that the normalization difference is gonna account for everything . phd e: So what I was working on is  just going through and checking the headers of the wavefiles , to see if maybe there was a  a certain type of compression or something that was done that my script wasn't catching . phd e: Which would you know cause it to perform OK , but  , you know , the the models would be all messed up . So I was going through and just double - checking that kind of think first , to see if there was just some kind of obvious bug in the way that I was computing the features . phd e: Looking at all the sampling rates to make sure all the sampling rates were what eight K , what I was assuming they were , professor c: Yeah . So I was doing that first , before I did these other things , just to make sure there wasn't something professor c: Although really , a couple three percent  difference in word error rate  could easily come from some difference in normalization , I would think . But phd e: Yeah , and I think , hhh I 'm trying to remember but I think I recall that Andreas was saying that he was gonna run sort of the reverse experiment .  which is to try to emulate the normalization that we did but with the mel cepstral features . professor c: Yeah , he 's probably off at at  his meeting now , phd e: Yeah , he 's gone now . But yeah phd e: But professor c: the I sh think they should be roughly equivalent ,   again the Cambridge folk found the PLP actually to be a little better . professor c:  the other thing I wonder about was whether there was something just in the the bootstrapping of their system which was based on but maybe not , since they phd e: Yeah see one thing that 's a little bit  I was looking I 've been studying and going through the logs for the system that  Andreas created . And  his  the way that the S R I system looks like it works is that it reads the wavefiles directly ,  and does all of the cepstral computation stuff on the fly . phd e: And , so there 's no place where these where the cepstral files are stored , anywhere that I can go look at and compare to the PLP ones , so whereas with our features , he 's actually storing the cepstrum on disk , and he reads those in . phd e: But it looked like he had to give it  even though the cepstrum is already computed , he has to give it  a front - end parameter file . Which talks about the kind of  com computation that his mel cepstrum thing does , professor c:  - huh . phd e: so i I I don't know if that it probably doesn't mess it up , it probably just ignores it if it determines that it 's already in the right format or something but the the the two processes that happen are a little different . I 've been  , I 've been working with  Jeremy on his project and then I 've been trying to track down this bug in  the ICSI front - end features . phd e: So one thing that I did notice , yesterday I was studying the  the  RASTA code professor c:  - huh . phd e: and it looks like we don't have any way to  control the frequency range that we use in our analysis . We basically it looks to me like we do the FFT ,  and then we just take all the bins and we use everything . We don't have any set of parameters where we can say you know , " only process from you know a hundred and ten hertz to thirty - seven - fifty " . professor c: Yeah , I don't think it 's in there , I think it 's in the    the filters . So , the F F T is on everything , but the filters  , for instance , ignore the the lowest bins and the highest bins . And what it does is it it copies phd e: The the filters ? Which filters ? professor c:  The filter bank which is created by integrating over F F T bins . Yeah , it 's bark scale , and it 's it it  it actually copies the   the second filters over to the first . So the first filters are always and you can s you can specify a different number of  features different number of filters , I think , as I recall . So you can specify a different number of filters , and whatever   you specify , the last ones are gonna be ignored . Y you can't do it without I think changing the number of filters , but phd e: I saw something about  that looked like it was doing something like that , but I didn't quite understand it . So maybe professor c: Yeah , so the idea is that the very lowest frequencies and and typically the veriest highest frequencies are kind of junk . professor c: And so  you just for continuity you just approximate them by by the second to highest and second to lowest . professor c: And and so if you h phd e: But so the but that 's a fixed  thing ? professor c: Yeah , I think that 's a fixed thing . phd e: There 's nothing that lets you professor c: But see see my point ? If you had If you had ten filters , then you would be throwing away a lot at the two ends . professor c: And if you had if you had fifty filters , you 'd be throwing away hardly anything . professor c: I don't remember there being an independent way of saying " we 're just gonna make them from here to here " . professor c: But I I I don't know , it 's actually been awhile since I 've looked at it . phd e: Yeah , I went through the Feacalc code and then looked at you know just calling the RASTA libs and thing like that . And I didn't I couldn't see any wh place where that kind of thing was done . But  I didn't quite understand everything that I saw , professor c: Yeah , see I don't know Feacalc at all . I guess for some particular database you might find that you could tune that and tweak that to get that a little better , but I think that in general it 's not that critical . professor c: You can You can throw away stuff below a hundred hertz or so and it 's just not going to affect phonetic classification at all . phd e: Another thing I was thinking about was  is there a I was wondering if there 's maybe  certain settings of the parameters when you compute PLP which would basically cause it to output mel cepstrum . So that , in effect , what I could do is use our code but produce mel cepstrum and compare that directly to professor c: Well , it 's not precisely . professor c:  what you can do is  you can definitely change the the filter bank from being  a  trapezoidal integration to a a a triangular one , phd e:   professor c: And some people have claimed that they got some better performance doing that , so you certainly could do that easily . But the fundamental difference , there 's other small differences phd e: There 's a cubic root that happens , right ? professor c: Yeah , but , you know , as opposed to the log in the other case .  the fundamental d d difference that we 've seen any kind of difference from before , which is actually an advantage for the P L P i  , I think , is that the the smoothing at the end is auto - regressive instead of being cepstral  , from cepstral truncation . professor c: and that 's that 's why when people started getting databases that had a little more noise in it , like like   Broadcast News and so on , that 's why c Cambridge switched to PLP I think . professor c: So  That 's a difference that I don't think we put any way to get around , since it was an advantage . professor c: but we did eh we did hear this comment from people at some point , that  it  they got some better results with the triangular filters rather than the trapezoidal . phd e: Yeah just it just seems like this kind of behavior could be caused by you know s some of the training data being messed up . phd e: You know , you 're sort of getting most of the way there , but there 's a So I started going through and looking One of the things that I did notice was that the  log likelihoods coming out of the log recognizer from the PLP data were much lower , much smaller , than for the mel cepstral stuff , and that the average amount of pruning that was happening was therefore a little bit higher for the PLP features . professor c: Oh - huh ! phd e: So , since he used the same exact pruning thresholds for both , I was wondering if it could be that we 're getting more pruning . professor c: Oh ! He he He used the identical pruning thresholds even though the s the range of p of the likeli phd e: Yeah . phd e: Yeah , professor c: I would think that you might wanna do something like  you know , look at a few points to see where you are starting to get significant search errors . Well , what I was gonna do is I was gonna take  a couple of the utterances that he had run through , then run them through again but modify the pruning threshold and see if it you know , affects the score . But  you could  if if if that looks promising you could , you know , r  run the overall test set with a with a few different  pruning thresholds for both , phd e: So . professor c: and presumably he 's running at some pruning threshold that 's that 's  , you know gets very few search errors phd e: Right .  , yeah , generally in these things you you turn back pruning really far , professor c: and phd e: so I I didn't think it would be that big a deal because I was figuring well you have it turned back so far that you know it professor c: But you may be in the wrong range for the P L P features for some reason . And the  the the run time of the recognizer on the PLP features is longer which sort of implies that the networks are bushier , you know , there 's more things it 's considering which goes along with the fact that the matches aren't as good . I guess this was a little bit off topic , I guess , because I was I was thinking in terms of th this as being a a a a core item that once we once we had it going we would use for a number of the front - end things also . professor c:  Wanna grad b: That 's as far as my stuff goes , professor c: What 's what 's on grad b: yeah , well I tried this mean subtraction method . Due to Avendano , I 'm taking s  six seconds of speech ,  I 'm using two second FFT analysis frames , stepped by a half second so it 's a quarter length step and I I take that frame and four f the four I take Sorry , I take the current frame and the four past frames and the four future frames and that adds up to six seconds of speech . And  the I tried that with HDK , the Aurora setup of HDK training on clean TI - digits , and  it it helped  in a phony reverberation case  where I just used the simulated impulse response  the error rate went from something like eighty it was from something like eighteen percent to  four percent . And on meeting rec recorder far mike digits , mike on channel F , it went from  forty - one percent error to eight percent error . phd e: On on the real data , not with artificial reverb ? grad b: Right . grad b: And that that was  trained on clean speech only , which I 'm guessing is the reason why the baseline was so bad . And professor c: That 's ac actually a little side point is I think that 's the first results that we have    of any sort on the far field  on on the far field data  for recorded in in meetings . professor c: Did he ? On the near field , on the ne grad b: On the far field also . What kind of numbers was he getting with that ? grad b: I I 'm not sure , I think it was about five percent error for the PZM channel .  , clean TI - digits is , like , pretty pristine training data , and if they trained the SRI system on this TV broadcast type stuff , I think it 's a much wider range of channels and it professor c: No , but wait a minute . I I I th I think he What am I saying here ? Yeah , so that was the SRI system . So So probably it should be something we should try then is to is to see if is at some point just to take i to transform the data and then and then  use th use it for the SRI system . grad b: b You me you mean  ta professor c: So you 're so you have a system which for one reason or another is relatively poor , grad b: Yeah . professor c: and and  you have something like forty - one percent error  and then you transform it to eight by doing doing this this work . So here 's this other system , which is a lot better , but there 's still this kind of ratio . It 's something like five percent error with the the distant mike , and one percent with the close mike . professor c: So the question is how close to that one can you get if you transform the data using that system . grad b: r Right , so so I guess this SRI system is trained on a lot of s Broadcast News or Switchboard data . grad b: Do you know which one it is ? phd e: It 's trained on a lot of different things . It 's trained on  a lot of Switchboard , Call Home , grad b:  - huh . phd e:  a bunch of different sources , some digits , there 's some digits training in there . grad b: O one thing I 'm wondering about is what this mean subtraction method  will do if it 's faced with additive noise . Cuz I I it 's cuz I don't know what log magnitude spectral subtraction is gonna do to additive noise . professor c: Yeah , grad b: That 's that 's the professor c: well , it 's it 's not exactly the right thing grad b:  - huh . So  grad b: OK , so it 's then then it 's it 's it 's reasonable to expect it would be helpful if we used it with the SRI system and professor c: Yeah , as helpful  , so that 's the question . Yeah , w we 're often asked this when we work with a system that that isn't isn't sort of industry industry standard great , grad b:  - huh . professor c:  and we see some reduction in error using some clever method , then , you know , will it work on a on a on a good system . I think , you know , one one percent word error rate on digits is  digit strings is not  you know stellar , but but given that this is real digits , as opposed to  sort of laboratory grad b:   professor c: you know , if you 're doing a  a sixteen digit  credit card number you 'll basically get it wrong almost all the time . So I don't think I can do the digits , but  , I guess I 'll leave my microphone on ? phd e: yeah