Two items , which was , digits and possibly stuff on on , forced alignment , which Jane said that Liz and Andreas had in information on , professor b:  grad e: but they didn't , phd f:   OK , so there 's digits , alignments , and , I guess the other thing , which I came unprepared for , is , to dis s s see if there 's anything anybody wants to discuss about the Saturday meeting . Yeah , it was grad e: Yeah , it 's forced alignment of people 's schedules . professor b: With with whatever it was , a month and a half or something ahead of time , the only time we could find in common roughly in common , was on a Saturday . postdoc c: Have Have we thought about having a conference call to include him in more of in more of the meeting ? I  , I don't know , if we had the if we had the telephone on the table professor b: No . phd f: No , actually I I have to I have to shuttle kids from various places to various other places . And I don't have and I don't , have a cell phone phd d: A cell phone ? phd f: so I can't be having a conference call while driving . phd f:  professor b: So we have to equip him with a with a with a head - mounted , cell phone grad e: Ye - we and we 'd have to force you to read lots and lots of digits , professor b: and grad e: so it could get real real car noise . phd f: I let , my five - year - old have a try at the digits , eh .  , did everyone get the results or shall I go over them again ?  that it was basically the only thing that was even slightly surprising was that the lapel did so well .  , and in retrospect that 's not as surprising as maybe i it shouldn't have been as surprising as I as as I felt it was . And as Morgan pointed out , that there are actually some advantages to it in terms of breath noises and clothes rustling if no one else is talking . professor b: Well , it 's Yeah , sort of the bre the breath noises and the mouth clicks and so forth like that , the lapel 's gonna be better on . professor b: The lapel is typically worse on the on clothes rustling , but if no one 's rustling their clothes , grad e: Right .  , a lot of people are just sort of leaning over and reading the digits , professor b: it 's it 's grad e: so it 's it 's a very different task than sort of the natural . grad g: Probably the fact that it picks up other people 's speakers other people 's talking is an indication of that it the fact it is a good microphone . phd f: D do the lapel mikes have any directionality to them ? professor b: There typically don't , no . phd f: Because I I suppose you could make some that have sort of that you have to orient towards your mouth , grad e: They have a little bit , phd f: and then it would grad e: but they 're not noise - cancelling . professor b: And th it 's and because you don't know how people are gonna put them on , you know . So , also , Andreas , on that one the the back part of it should be right against your head . professor b: and and it was  , there the point of interest to the group was primarily that , the ,  the system that we had that was based on H T K , that 's used by , you know , all the participants in Aurora , was so much worse than the than the S R grad e: Everybody . professor b: And the interesting thing is that even though , yes , it 's a digits task and that 's a relatively small number of words and there 's a bunch of digits that you train on , it 's just not as good as having a a l very large amount of data and training up a a a nice good big  . phd f: And we know Di - did I send you some results without adaptation ? grad e: No . grad e: Or if you did , I didn't include them , cuz it was professor b: So phd f: Yeah , I think I did , actually . A a a couple percent or some  Well , I don't know it Overall  , I I don't remember , but there was there was a significant , loss or win from adaptation with with adaptation . And then there was a very small like point one percent on the natives  , win from doing , you know , adaptation to the recognition hypotheses . And I tried both means adaptation and means and variances , and the variances added another or subtracted another point one percent . professor b: But I think one thing is that , I would presume Hav - Have you ever t Have you ever tried this exact same recognizer out on the actual TI - digits test set ? phd f: This exact same recognizer ? No . Cuz my my cuz my sense ,  phd f: But but , I have  , people people at SRI are actually working on digits . phd f: I could and they are using a system that 's ,  you know , h is actually trained on digits , but h h otherwise uses the same , you know , decoder , the same , training methods , and so forth , professor b:   professor b: Yeah , bu although I 'd be I think it 'd be interesting to just take this exact actual system so that these numbers were comparable phd f:   Cuz our sense from the other from the Aurora , task is that grad e: And try it with TI - digits ? phd f:   professor b: cuz we were getting sub one percent numbers on TI - digits also with the tandem thing . professor b: One is , yeah , the SRI system is a lot better than the HTK phd f:  . professor b: but the other is that , the digits recorded here in this room with these close mikes , i  , are actually a lot harder than the studio - recording TI - digits . I think , you know , one reason for that , might be that there 's still even though it 's close - talking , there still is some noise and some room acoustics . professor b: And another might be that , I 'd I would presume that in the studio , situation recording read speech that if somebody did something a little funny or n pronounced something a little funny or made a little that they didn't include it , grad e: They didn't include it . grad e: Whereas , I took out the ones that I noticed that were blatant that were correctable . grad e: And then there was another one where Jose couldn't tell whether I couldn't tell whether he was saying zero or six . grad e: You know , so I just e edited out the first , i  , word of the utterance .  , so there 's a little bit of correction but it 's definitely not as clean as TI - digits . grad e: Right ? So it would probably do even a little better still on the SRI system , but we could give it a try . But remember , we 're using a telephone bandwidth front - end here , on this ,  on this SRI system , so , I was I thought that maybe that 's actually a good thing because it it gets rid of some of the  , the noises , you know , in the the below and above the  , the , you know , speech bandwidth professor b:   phd f: and , I suspect that to get sort of the last bit out of these higher - quality recordings you would have to in fact , use models that , were trained on wider - band data . And of course we can't do that or grad e: Wha - what 's TI - digits ? I thought t professor b: It 's wide - band , yeah . It 's in in fact , we looked it up grad e: It is wide - band . grad e: I couldn't remember whether that was TI - digits or one of the other digit tasks . professor b: See w grad e: So , Morgan , you 're getting a little breath noise . phd f: Now , eh , does grad e: You might wanna move the mike down a little bit . phd f: one one issue one issue with with that is that  , the system has this , notion of a speaker to which is used in adaptation , variance norm  , you know , both in , mean and variance normalization and also in the VTL estimation . phd f: Do y ? Is ? So does so th so does does , the TI - digits database have speakers that are known ? grad e: Yep . phd f: And is there is there enough data or a comparable comparable amount of data to to what we have in our recordings here ? grad e: That I don't know . professor b: Well , the other thing would be to do it without the adaptation and compare to these numbers without the adaptation .  , but I 'm not so much worried about the adaptation , actually , than than the ,  the , VTL estimation . phd f: If you have only one utterance per speaker you might actually screw up on estimating the the warping , factor . But it 's not the amount of speakers , it 's the num it 's the amount of data per speaker . phd f: So grad e: So , although I I sort of know how to run it , there are a little a f few details here and there that I 'll have to dig out . phd f: And there 's a there 's a script and that is actually all in one script . So there 's this one script that parses waveform names and extracts things like the , speaker , ID or something that can stand in as a speaker ID . So , we might have to modify that script to recognize the , speakers , in the in the , TI - digits database . And that ,  phd f: Or you can fake you can fake names for these waveforms that resemble the names that we use here for the for the meetings . phd f: That would be the , sort of probably the safest way to do grad e: I might have to do that anyway to to do because we may have to do an extract to get the amount of data per speaker about right . grad e: The other thing is , isn't TI - digits isolated digits ? phd f: Right . grad e: Or is that another one ? I 'm I looked through a bunch of the digits t corp corpora , and now they 're all blurring . phd f: By the way , I think we can improve these numbers if we care to compr improve them by , not starting with the Switchboard models but by taking the Switchboard models and doing supervised adaptation on a small amount of digit data collected in this setting . phd f: Because that would adapt your models to the room acoustics and f for the far - field microphones , you know , to the noise . And then you use those adapted models , which are not speaker adapted but sort of acous you know , channel adapted grad e: Channel adapted . But the thing is , w when you it depends whether you 're ju were just using this as a a starter task for you know , to get things going for conversational or if we 're really interested i in connected digits . And for for connected digits over the telephone you don't actually want to put a whole lot of effort into adaptation phd f: Well , I don't know . You don't don't ,  postdoc c: This is this that one 's better . phd f: but , you know , I  , my impression was that you were actually interested in the far - field microphone , problem ,  . phd f: Right ? Then , eh because you you don't have any postdoc c: Yeah . phd f: That 's where the most m acoustic mismatch is between the currently used models and the the r the set up here . professor b: I I guess I 'm saying I don't know if we 'd want to do that as the as phd d: Other way . postdoc c: If you have a strong fe if you have a strong preference , you could use this . At any rate , I don't know if w postdoc c: I don't know . phd f: It is ? professor b: I don't know if we wanna use that as the postdoc c: Yeah . phd f: I I postdoc c: and then you have to scr phd f: I I already adjusted this a number of times . phd f: I I grad e: Yeah , I think these mikes are not working as well as I would like . phd f: can't quite seem to Yeah , I think this contraption around your head is not working so well . Anyway , what I was saying is that I I think I probably wouldn't want to see that as sort of like the norm , that we compared all things to . The other thing that that ,  of course , what Barry was looking at was was just that , the near versus far . professor b: But , I think even even if there was , only a factor of two or something , like I was saying in the email , I think that 's that 's a big factor . professor b: N grad e: Liz , you could also just use the other mike if you 're having problems with that one . We we we think that this has spikes on it , phd a: It 's this thing 's This is too big for my head . postdoc c: so it 's not as good acoustically , phd f: Yeah , basically your ears are too big . So , it doesn't you know , it 's sit phd f:  postdoc c: Well , if you 'd rather have this one then it 's phd a: OK . grad e: So the To get that , pivoted this way , it pivots like this . grad e: So if it doesn't bounce around too much , that 's actually good placement . I know what I was go w phd f: What k u By the way , wh what factor of two did you ? professor b: Oh , no , no . phd f:  professor b: It 's tha that that we were saying , you know , well is how much worse is far than near , you know . professor b: And  it depends on which one you 're looking at , phd f: That factor of two . I I know what I was thinking was that maybe , i i we could actually t t try at least looking at , some of the the large vocabulary speech from a far microphone , at least from the good one . professor b: before I thought we 'd get , you know , a hundred and fifty percent error or something , but if if ,  if we 're getting thirty - five , forty percent or something , u  phd f:   phd a: Actually if you run , though , on a close - talking mike over the whole meeting , during all those silences , you get , like , four hundred percent word error . But I 'm saying if you do the same kind of limited thing as people have done in Switchboard evaluations or as a phd a: Yeah . Where you know who the speaker is and there 's no overlap ? And you do just the far - field for those regions ? professor b: Yeah . Right ? grad e: Could we do exactly the same thing that we 're doing now , but do it with a far - field mike ? professor b: Yeah , do it with one of on grad e: Cuz we extract the times from the near - field mike , but you use the acoustics from the far - field mike . There 's ,  You can use times where that person is talking only from the transcripts but the segmentations were were synchronized . Or you can do a forced alignment on the close - talking to determine that , the you know , within this segment , these really were the times that this person was talking and elsewhere in the segment other people are overlapping and just front - end those pieces . Or you can run it on the whole data , which is which is , you know , a professor b: But but but how did we get the how did we determine the links , that we 're testing on in the stuff we reported ? phd a: In the H L T paper we took segments that are channel time - aligned , which is now h being changed in the transcription process , which is good , and we took cases where the transcribers said there was only one person talking here , because no one else had time any words in that segment and called that " non - overlap " . professor b: But anyway so I think that we should try it once with the same conditions that were used to create those , and in those same segments just use one of the P Z phd a: Right . professor b: And then , you know , the thing is if we were getting ,  what , thirty - five , forty percent , something like that on on that particular set , does it go to seventy or eighty ? phd a: Right . professor b: Or , does it use up so much memory we can't decode it ? phd a: It might also depend on which speaker th it is and how close they are to the PZM ? professor b:  phd a: I don't know how different they are from each other . So we would then use that one , too , grad e: So phd f: Oh , OK . phd a: or ? professor b: You know , it 's so i but I would I 'd pick that one . It 'll be less good for some people than for other , but I I 'd like to see it on the same exact same data set that that we did the other thing on . grad e: Actually I sh actually should 've picked a different one , professor b: Right ? grad e: because that could be why the PDA is worse . professor b: But the other is , it 's very , even though there 's I 'm sure the f f the the SRI , front - end has some kind of pre - emphasis , it 's it 's ,  still , th it 's picking up lots of low - frequency energy . professor b: So , even discriminating against it , I 'm sure some of it 's getting through . When you listen to it , the PZM and the PDA Yeah , th the PDA has higher sound floor but not by a lot . grad e: Th - we wanted them to be to be typical of what would be in a PDA . professor b: But , the thing is people use those little mikes for everything because they 're really not bad . professor b: if you 're not doing something ridiculous like feeding it to a speech recognizer , they they they you know , you can hear the sou hear the sounds just fine . professor b: You know , it 's They  , i it 's more or less the same principles as these other mikes are built under , it 's just that there 's less quality control . So like I said , the front - end guys are very much interested in in this is as as well and phd f: So so , but where is this now ?  , what 's where do we go from here ? grad e: Yeah . phd f: we so we have a we have a a system that works pretty well but it 's not , you know , the system that people here are used to using to working with . professor b: Well , I think what we wanna do is we want to eh , phd f: So what what do we do now ? professor b: and we 've talked about this in other contexts we want to have the ability to feed it different features . professor b: And then , from the point of view of the front - end research , it would be s  , substituting for HTK . And then if we can feed it different features , then we can try all the different things that we 're trying there . professor b: And then , also Dave is is thinking about using the data in different ways , to  , explicitly work on reverberation phd f:   So so the key thing that 's missing here is basically the ability to feed , you know , other features i into the recognizer professor b: Right . And , es I don't know when Chuck will be back but that 's exactly what he he 's gonna professor b: H h He 's he 's sort of back , but he drove for fourteen hours an and wasn't gonna make it in today . phd f: It 's  , the the front - end is f i tha that 's in the SRI recognizer is very nice in that it does a lot of things on the fly but it unfortunately is not designed and ,  like the , ICSI system is , where you can feed it from a pipeline of of the command . So , the what that means probably for the foreseeable future is that you have to , dump out ,  you know , if you want to use some new features , you have to dump them into individual files and give those files to the recognizer . So , although you you can pipe it as well , we tend to do it that way because that way you can concentrate on one block and not keep re - doing it over and over . So I 've I grad e: So tha that 's exactly what the P - file is for . phd f: Yeah , the the the cumbersome thing is is ,  is that you actually have to dump out little little files . phd a:  phd f: So for each segment that you want to recognize you have to dump out a separate file . phd f: Just like i th like th as if there were these waveform segments , but instead you have sort of feature file segments . So the s the the next thing we had on the agenda was something about alignments ? phd a: Oh . Yes , we have I don't know , did you wanna talk about it , or ? I can give a I was just telling this to Jane and and W we we were able to get some definite improvement on the forced alignments by looking at them first and then realizing the kinds of errors that were occurring and  , some of the errors occurring very frequently are just things like the first word being moved to as early as possible in the recognition , which is a  , I think was both a a pruning problem and possibly a problem with needing constraints on word locations . We tried saying I don't know , I got this whacky idea that just from looking at the data , that when people talk their words are usually chunked together . They 're might say one word and then another word far away if they were doing just backchannels ? But in general , if there 's , like , five or six words and one word 's far away from it , that 's probably wrong on average . And we had lowered that we had used tighter pruning after Liz ran some experiments showing that , you know , it runs slower and there 's no real difference in phd a: Actually it was better with slightly better or about th grad e: No gain . phd a: It 's probably cuz the recognition 's just bad en at a point where it 's bad enough that that you don't lose anything .  , but it turned out for for to get accurate alignments it was really important to open up the pruning significantly . phd f:  because otherwise it would sort of do greedy alignment , in regions where there was no real speech yet from the foreground speaker . phd f: so that was one big factor that helped improve things and then the other thing was that , you know , as Liz said the we f enforce the fact that , the foreground speech has to be continuous .  , yeah , it isn't always true , and I think what we really want is some clever way to do this , where , you know , from the data or from maybe some hand - corrected alignments from transcribers that things like words that do occur just by themselves a alone , like backchannels or something that we did allow to have background speech around it phd d: Yeah . And , we basically also made noise models for the different sort of grouped some of the mouth noises together . And we also There was some neat or , interesting cases , like there 's one meeting where , Jose 's giving a presentation and he 's talking about , the word " mixed signal " and someone didn't understand , that you were saying " mixed " I think , Morgan . phd a: And the next turn was a lot of people saying " mixed " , like " he means mixed signal " or " I think it 's mixed " . phd h: Sh phd a: And Chuck 's on the lapel here , and he also says " mixed " but it 's at the last one , and of course the aligner th aligns it everywhere else to everybody else 's " mixed " , phd h: Yeah . So there 's I think there 's some issues about u We probably want to adapt at least the foreground speaker . But , I guess Andreas tried adapting both the foreground and a background generic speaker , and that 's actually a little bit of a f funky model . Like , it gives you some weird alignments , just because often the background speakers match better to the foreground than the foreground speaker . phd a: especially when you get lots of the same words , occurring in the phd f: Well , the I I think you can do better by  , cloning so we have a reject phone . And you and what we wanted to try with you know , once we have this paper written and have a little more time , t cloning that reject model and then one copy of it would be adapted to the foreground speaker to capture the rejects in the foreground , like fragments and stuff , and the other copy would be adapted to the background speaker .  , in general we actually phd f: And phd a: Right now the words like partial words are reject models and you normally allow those to match to any word . phd a: But then the background speech was also a reject model , and so this constraint of not allowing rejects in between you know , it needs to differentiate between the two . phd a: And another one is turns , like people starting with " well I think " and someone else is " well how about " . So the word " well " is in this in this segment multiple times , and as soon as it occurs usually the aligner will try to align it to the first person who says it . But then that constraint of sort of  , proximity constraint will push it over to the person who really said it in general . grad e: Is the proximity constraint a hard constraint , or did you do some sort of probabilistic weighting distance , or ? phd f: We we didn't phd a: Right now it 's a kluge . We it 's straightforward to actually just have a a penalty that doesn't completely disallows it but discourages it . But , we just didn't have time to play with , you know , tuning yet another yet another parameter . phd f: And really the reason we can't do it is just that we don't have a we don't have ground truth for these . So , we would need a hand - marked , word - level alignments or at least sort of the boundaries of the speech betw you know , between the speakers .  , and then use that as a reference and tune the parameters of the of the model , to op to get the best performance . professor b: G given I  , I wa I wa I was gonna ask you anyway , how you assessed that things were better . phd a: Oh , it was painful because the thing is , you know the alignments share a lot in common , so And you 're yo you 're looking at these segments where there 's a lot of speech . phd a:  that if you look at the individual segments from just one person you don't see a lot of words , phd h: Ju professor b: Yeah . phd a: And so the reject is also mapping and pauses So I looked at them all in Waves and just lined up all the alignments , and , at first it sort of looked like a mess and then the more I looked at it , I thought " OK , well it 's moving these words leftward and " You know , it wasn't that bad . So But , I don't , you know , have time to l to look at all of them and it would be really useful to have , like , a a transcriber who could use Waves , just mark , like , the beginning and end of the foreground speaker 's real words like , the beginning of the first word , the end of the last word and then we could , you know , do some adjustments . I have to ask you something , is i does it have to be Waves ? Because if we could benefit from what you did , incorporate that into the present transcripts , that would help . postdoc c: And then , the other thing is , I believe that I did hand So . One of these transcripts was gone over by a transcriber and then I hand - marked it myself so that we do have , the beginning and ending of individual utterances . And also I went back to the original one that I first transcribed and and did it w  , w  , utterance by utterance for that particular one . So I think you do have if that 's a sufficient unit , I think that you do have hand - marking for that . I used it in Transcriber phd f: U  postdoc c: and it 's it 's in the phd a: well , Jane and I were just in terms of the tool , talking about this . You know , interface - wise if you 're looking at speech , you wanna be able to know really where the words are . And so , we can give you some examples of sort of what this output looks like , postdoc c: Yeah , that 's right . Middle of the word , or phd a: and see if you can in maybe incorporate it into the Transcriber tool some way , or postdoc c: Well , I th I 'm thinking just ch e e incorporating it into the representation . postdoc c: if it 's if it 's phd a: You mean like Yeah , word start insights . postdoc c: if you have start points , if you have , like , time tags , phd a: Right . Isn't that what what you ? Well , see , Adam would be phd f: Yeah , whatever you use . phd f: we convert it to this format that the , NIST scoring tool unders  , CTM . And and then that 's the that 's what the grad e: I think Transcriber , outputs CTM . phd a: So ,  postdoc c: It seems like she if she 's g if she 's moving time marks around , phd f: Right . postdoc c: since our representation in Transcriber uses time marks , it seems like there should be some way of of using that benefitting from that . phd a: Yeah , it wou the advantage would just be that when you brought up a bin you would be able if you were zoomed in enough in Transcriber to see all the words , professor b:   phd a: you would be able to , like , have the words sort of located in time , if you wanted to do that . professor b: So so if we e e even just had a a It sounds like w we we almost do . phd a: You mean on on the hand - marked ,  So we we only r hav I only looked at actually alignments from one meeting that we chose , professor b: Yeah . phd a: Not randomly phd f: We knew we knew that it had these insertion errors from phd a: It had sort of average recognition performance in a bunch of speakers phd f: Yeah . phd a:  That Yeah , actually it wasn't the new new , it was the medium new . grad g: You did you adjust the the utterance times , for each channel ? postdoc c: Yes . And furthermore , I found that there were a certain number where not not a lot , but several times I actually moved an utterance from Adam 's channel to Dan 's or from Dan 's to Adam 's . So there was some speaker identif And the reason was because I transcribed that at a point before  , before we had the multiple audio available f so I couldn't switch between the audio . I I transcribed it off of the mixed channel entirely , which meant in overlaps , I was at a at a terrific disadvantage . And finally I did it using the speakers of my ,  of you know , off the CPU on my on my machine cuz I didn't have a headphone . postdoc c: So it @ @ , like ,  Yeah , I  , i in retrospect it would 've been good to ha have got I should 've gotten a headphone . But in any case , thi this is this was transcribed in a in a , less optimal way than than the ones that came after it , and I was able to you know , an and this meant that there were some speaker identif identifications which were changes . grad g: Is that what you 're referring to ?  , cuz there 's this one instance when , for example , you 're running down the stairs . It 's a Yeah , I 've I 've I 'm very well acquainted with this meeting . grad g: Yeah , I can s phd a: " And then she said , and then he said . grad g: Is that what you 're talking about ? Or were there mislabellings as far as , like , the a Adam was ? postdoc c: That was fixed , before i i i I think I I think I understood that pretty grad g: Yeah . Yeah , no , tha that That I think went away a couple of versions ago , grad g: Yeah . So , with under  , listening to the mixed channel , there were times when , as surprising as that is , I got Adam 's voice confused with Dan 's and vice versa grad g: OK . The other thing that was w interesting to me was that I picked up a lot of , backchannels which were hidden in the mixed signal , phd a: Right . But the other thing that I I hadn't thought about this , but I thou I wanted to raise this when you were  , with respect to also a strategy which might help with the alignments potentially , but that 's When I was looking at these backchannels , they were turning up usually very often in w well , I won't say " usually " but anyway , very often , I picked them up in a channel w which was the person who had asked a question . S so , like , someone says " an and have you done the so - and - so ? " And then there would be backchannels , but it would be the person who asked the question . postdoc c: i it wouldn't be perfect , but but it does seem more natural to give a backchannel when when you 're somehow involved in the topic , phd a: No , that 's really interesting . postdoc c: and the most natural way is for you to have initiated the topic by asking a question . I think it 's actually I think what 's going on is backchannelling is something that happens in two - party conversations . phd f: And if you ask someone a question , you essentially initiating a little two - party conversation . phd a: Well , actu Yeah , when we looked at this postdoc c: Exactly . phd f: So then you 're so and then you 're expected to backchannel because the person is addressing you directly and not everybody . postdoc c: But in addition , you know , if someone has done this analysis himself and isn't involved in the dyad , but they might also give backchannels to verify what what the answer is that this that the the answerer 's given professor b: H phd a: Right . professor b: I tell you , I say I say "  - huh " a lot , phd a: It 's postdoc c: There you go . phd a: Well , but it 's interesting cuz ,  professor b: while people are talking to each other . phd a: just from We were looking at word frequency lists to try to find the cases that we would allow to have the reject words in between in doing the alignment . phd a: And "  - huh " is not as frequent as it sort of would be in Switchboard , if you looked at just a word frequency list of one - word short utterances . And so I was thinking thi it 's not like you 're being encouraged by everybody else to keep talking in the meeting . And  , that 's all , I I 'll stop there , cuz I I think what you say makes a lot of sense . And that would phd a: But it was sort of postdoc c: Well , an And what you say is the is the re  , o other side of this , which is that , you know , so th there are lots of channels where you don't have these backchannels , w when a question has been asked and and these phd a: Right . phd a: even if you consider every other person altogether one person in the meeting , but we 'll find out anyway . We were I guess the other thing we 're we 're I should say is that we 're gonna ,  try compare this type of overlap analysis to Switchboard , where phd f: And phd a: and CallHome , where we have both sides , so that we can try to answer this question of , you know , is there really more overlap in meetings or is it just because we don't have the other channel in Switchboard professor b: y y you folks have probably already told me , but were were you intending to do a Eurospeech submission , or ? phd a: you mean the one due tomorrow ? professor b: Yeah . Well , we 're still , like , writing the scripts for doing the research , and we will Yes , we 're gonna try . phd a: And I was telling Don , do not take this as an example of how people should work . professor b: Do as I say , grad g: That 's r phd a: So , we will try . phd a: It 'll probably be a little late , grad e: Well phd a: but I 'm gonna try it . phd a: Well , I 'm no We may be in the same position , and I figured we 'll try , because that 'll at least get us to the point where we have We have this really nice database format that Andreas and I were working out that It it 's not very fancy . It 's just a ASCII line by line format , but it does give you information phd f: It 's the it 's the spurt format . I was trying to find what 's a word for a continuous region with pauses around it ? postdoc c:  . professor b: And that 's  , I I was using that for a while when I was doing the rate of speech stuff , phd a: I would jus professor b: because I because I looked up in some books and I found OK , I wanna find a spurt in which phd a: Ah , right ! It 's just , like , defined by the acoustics . professor b: and an because cuz it 's another question about how many pauses they put in between them . professor b: But how fast do they do the words within the spurt ? phd a: Right . phd a: Well , that 's what we were calling spurt , grad e: It 's gonna grad g: you know " Burst " also ? grad e: Burst . grad g: Isn't " burst " is used also ? phd a: so grad e: Spurt has the horrible name overloading with other with hardware at ICSI . phd a: Well , well , Chafe had this wor I think it was Chafe , or somebody had a the word " spurt " originally , professor b: But but that just phd h: Here @ @ phd a: and so I But tha that 's good to know . postdoc c: Actually phd a: Was thi it 's Chafe ? postdoc c: Well , see , I know S Sue wrote about spurts of development . phd f: So maybe we should talk phd a: Maybe it was Sue ? Y postdoc c: But , in any case , I think it 's a good term , phd a: So we have spurts and we have spurt - ify dot shell and spurt - ify professor b: Yeah . postdoc c: I know I know Ch - Chafe dealt with phd f: So s grad g: That 's cool . postdoc c: But maybe he speaks about spurts as well phd f: We postdoc c: and I just don't know . phd f: So what we 're doing  , this this is just maybe someone has s some some ideas about how to do it better , grad g: Mmm . We 're from each alignment we 're producing , one of these CTM files , postdoc c: Great . phd f: which essentially has it 's just a linear sequence of words with the begin times for every word and the duration . Right ? phd f: And and and of course phd a: It 's just phd f: Right . Third column is the , start times of the words and the fourth column is the duration of the words . Then we have a messy alignment process where we actually insert into the sequence of words the , tags for , like , where where sentence ends of sentence , question marks , various other things . phd a: So , Don sort of , propagated the punctuation from the original transcriber phd f: Right . phd a: so whether it was , like , question mark or period or , you know , comma and things like that , and we kept the and disfluency dashes  , kept those in because we sort of wanna know where those are relative to the spurt overlaps phd f:   phd a: sp overlaps , phd f: So so those are actually sort of retro - fitted into the time alignment . phd a: or phd f: And then we merge all the alignments from the various channels and we sort them by time . So you you id identify by some criterion , which is pause length you identify the beginnings and ends of these spurts , and you put another set of tags in there to keep those straight . phd f: And then you merge everything in terms of , you know , linearizing the sequence based on the time marks . And then you extract the individual channels again , but this time you know where the other people start and end talking you know , where their spurts start and end . So , you you basically have everything sort of lined up and in a form where you can look at the individual speakers and how their speech relates to the other speakers ' speech . phd a: I think that 's actually really u useful also phd f: And phd a: because even if you weren't studying overlaps , if you wanna get a transcription for the far - field mikes , how are you gonna know which words from which speakers occurred at which times relative to each other ? You have to be able to get a transcript like like this anyway , just for doing far - field recognition . phd a: I thi it 's just an issue we haven't dealt with before , how you time - align things that are overlapping anyway . phd f: So phd a: i I never thought about it before , grad e: Well phd f: And and we phd a: but grad e: Y yes . phd f: In grad e: s when I came up with the original data suggested data format based on the transcription graph , there 's capability of doing that sort of thing in there . Well , this is this is just phd a: Yeah , this is like a poor man 's ver formatting version . It 's like there 're twelve different scripts which you run and then at the end you have what you want . All we care about is whether that there 's a certain word was overlapped by someone else 's word . So you sort of at that point , you discretize things into just having overlap or no overlap . phd f: But if you wanted to do a more fine - grained analysis and say , you know , how far into the word is the overlap , you could do that . phd f: It 's just it 'll just require more phd a: Just sort of huge . phd f: you know , slightly different postdoc c: What 's interesting is it 's exactly what , i in discussing with , Sue about this , phd a: Yeah . postdoc c: she , i i i indicated that that you know , that 's very important for overlap analysis . phd a: and also I think as a human , like , I don't always hear these in the actual order that they occur . So I can have two foreground speakers , you know , Morgan an and  , Adam and Jane could all be talking , and I could align each of them to be starting their utterance at the correct time , and then look where they are relative to each other , and that 's not really what I heard . postdoc c: This is This is Bever 's Bever 's effect , phd a: Y Yeah . postdoc c: when where In psy ps psycho - linguistics you have these experiments where people have perceptual biases a as to what they hear , phd a: It 's sort of Yeah , you sort of move things around until you get to a low information point postdoc c: that that Not the best phd a: and yo then you can bring in the other person . So it 's actually not even possible , I think , for any person to listen to a mixed signal , even equalize , and make sure that they have all the words in the right order .  , and the good thing is that we have It 's sort of a beginning of what Don can use to link the prosodic features from each file to each other . phd f: Plus , mayb phd h:  ? phd a: We - I ju Otherwise we won't get the work done on our deadline . Maybe , you know , you could you could look at this format and see if you find anything interesting . professor b: No , it 's that 's the good thing about these pape paper deadlines and , you know , class projects , and and things like that , postdoc c: Well , what I 'm thinking is phd f: Yeah . postdoc c: Well , my phd f: Well th th the other thing that that that yo that you usually don't tell your graduate students is that these deadlines are actually not that , you know , strictly enforced , professor b: because you you really get g phd a: Forces you to do the work . phd f: because the professor b: Oh , now it 's out in the public , this this this secret information . postdoc c: I think we can ha phd f: bec b Nah phd a: So grad e: No . phd f: Well That 's another issue , professor b: By th by the way , this is totally unfair , you may you may feel , phd f: but professor b: but the the ,  the morning meeting folks actually have an an extra month or so . The Aurora there 's a special Aurora phd a:  phd f: When professor b: There 's a special Aurora session phd a: Oh . professor b: and the Aurora pe people involved in Aurora have till Ma -  , early May or something to turn in their paper . phd a: Oh , well maybe we 'll submit to s Actually phd f: Well , then you can just Maybe you can submit the digits paper on e for the Aurora session . professor b: It 's it 's not the Aurora  , it it 's it 's actually the Aurora task . phd f: But but the people  , a a paper that is not on Aurora would probably be more interesting at that point phd a: Maybe they 'll phd f: because everybody 's so sick and tired of the Aurora task . If you if you have it 's to if you discuss some relation to the Aurora task , like if you use the same professor b: This is not the Aurora task . So they just do a little grep for phd a: Do  , d d Do not do not we are not setting a good example . Well , a relation other than negation , maybe , phd a: This is not a phd f:  . phd a: But the good thing is this does grad e: Well , I I don't know .  , you could you could do a paper on what 's wrong with the Aurora task by comparing it to other ways of doing it . phd f: How well does an Aurora system do on on you know , on digits collected in a in this environment ? phd h:  grad e: Different way . professor b: you know , the people who were involved in the the only people who are allowed to test on that are people who who made it above a certain threshold in the first round , phd f:   professor b:  w in ninety - nine and it 's it 's sort of a it 's not like a phd f: Well , that 's maybe why they don't f know that they have a crummy system . professor b: Oh , you don't like HTK ? phd f: If they phd h: Yeah . phd f: I don't h I don't have any stock in HTK or Entropic or anything .  , this it it 's the HTK that is trained on a very limited amount of data . phd f: But so , if you But maybe you should , you know , consider more using more data , or  professor b: Oh , yeah . And they i i phd f: If yo if you sort of hermetically stay within one task and don't look left and right , then you 're gonna grad e: But they they had professor b: i But grad e: They had something very specific in mind when they designed it . grad e: And so so you can you can argue about maybe that wasn't the right thing to do , but , you know , they they they had something specific . professor b: But , one of the reasons I have Chuck 's messing around with with the back - end that you 're not supposed to touch  , for the evaluations , yes , we 'll run a version that hasn't been touched . professor b: But , one of the reasons I have him messing around with that , because I think it 's sort of an open question that we don't know the answer to . People always say very glibly that i if you s show improvement on a bad system , that doesn't mean anything , cuz it may not be show  , because , you know , it doesn't tell you anything about the good system . You know , that if some peopl If you 're actually are getting at something that has some conceptual substance to it , it will port . professor b: And in fact , most methods that people now use were originally tried with something that was not their absolute best system at some level . If we 're getting three percent error on , u  , English , nati native speakers , using the Aurora system , and we do some improvements and bring it from three to two , do those same improvements bring , th you know , the SRI system from one point three to you know , to point eight ? phd f:  . professor b: You know , that 's that 's pretty solid , on the segmentation stuff . And the Aurora folks here will will definitely get something in on Aurora , phd d: which is not phd f: Actually this this ,  So , there 's another paper . phd f: And he tested it mostly on digits because it 's sort of a you know , it doesn't take weeks to train it . And got some very impressive results , with , you know , discriminative , Gaussian training .  , you know , like , error rates go from I don't know , in very noisy environment , like from ,  I for now I OK , now I have the order of magnit I 'm not sure about the order of magnitude . Was it like from ten percent to eight percent or from e e you know , point you know , from one percent to point eight percent ? professor b: H i it got it got better . I think the only thing we had left was unless somebody else Well , there 's a couple things .  , one is anything that , anybody has to say about Saturday ? Anything we should do in prep for Saturday ?  I guess everybody knows about  , u  , Mari was asking was trying to come up with something like an agenda and we 're sort of fitting around people 's times a bit . But , clearly when we actually get here we 'll move things around this , as we need to , but so you can't absolutely count on it . professor b: I think this is phd f: Are we recording it ? phd a: We won't have enough microphones , professor b:  phd a: but professor b: u No . professor b: We won we wanna  , they 're there 's gonna be , Jeff , Katrin , Mari and two students . professor b: And Brian 's coming , phd f: But you know th professor b: so that 's six . phd a: We don't even have enough channel professor b: Well phd f: Because it would be a different kind of meeting , phd d: Yeah . phd f: that 's what I 'm professor b: Well phd f: But phd h: Yeah . professor b: I hadn't really thought of it , phd f: Maybe just maybe not the whole day professor b: but phd f: but just , you know , maybe some  , professor b: Maybe part of it . phd a: That 's their initiation into our professor b: Any phd a: w grad e: Into our our our cult . phd a: Yeah , our Yeah , our phd f: Maybe the sections that are not right afte you know , after lunch when everybody 's still munching and phd a: So can you send out a schedule once you know it , jus ? professor b: OK . phd a: There 's a res Is it changed now , or ? professor b: But I hadn't heard back from Mari after I I u u  , brought up the point abou about Andreas 's schedule . professor b: So , I 'll make a postdoc c: I 'm looking forward to seeing your representation . That 'd be ,  phd a: And w we should get the two meetings from y postdoc c: I 'd like to see that . phd a: I know about the first meeting , but the other one that you did , the NSA one , which we hadn't done cuz we weren't running recognition on it , because the non - native speaker postdoc c:   phd f: The ,  th the other good thing about the alignments is that , it 's not always the machine 's fault if it doesn't work . phd f: You can find , problems with with the transcripts , you know , grad e: Oh . phd a: Tha - There are some cases like where the the wrong speaker  , these ca Not a lot , but where the the wrong person the the speech is addre attached to the wrong speaker phd f: But phd a: and you can tell that when you run it . phd a: So these are from the early transcriptions that people did on the mixed signals , like what you have . It also raises the possibility of , using that kind of representation  , I don't know , this 'd be something we 'd wanna check , but maybe using that representation for data entry and then displaying it on the channelized , representation , cuz it I think that the  , my my preference in terms of , like , looking at the data is to see it in this kind of musical score format . phd a: Yeah , if you can get it to postdoc c: And and but , this if this is a better interface for making these kinds of , you know , lo clos local changes , then that 'd be fine , too . Th - the other thing I had actually was , I I didn't realize this till today , but , this is , Jose 's last day . phd f: Oh ! grad e: You 're not gonna be here tomorrow ? phd h: My my last meeting about meetings . phd d: The last meeting meeting ? phd h: Because , eh , I leave , eh , the next Sunday . phd h: And I I would like to to to say thank you very much , eh , to all people in the group and at ICSI , phd f:   And I 'm sorry by the result of overlapping , because , eh , I haven't good results , eh , yet but , eh , I I pretend to to continuing out to Spain , eh , during the the following months , professor b:  - huh . phd h: eh , because I have , eh , another ideas but , eh , I haven't enough time to to with six months it 's not enough to to to research , grad e: Yep . phd h: eh , and e i  , if , eh , the topic is , eh , so difficult , in my opinion , there isn't professor b: Yeah . Maybe somebody else will come along and will be , interested in working on it and could start off from where you are also , you know . But , eh , I I will try to recommend , eh , at , eh , the Spanish government but , eh , the following @ @ scholarship , eh , eh , eh , will be here more time , because eh , i in my opinion is is better , eh , for us to to spend more time here and to work more time i i in a topic . You e you have , eh you are lucky , and you you find a solution in in in some few tim  , months , eh ? OK . Eh , I I bring the chocolate , eh , to to tear , with with you , phd a: Oh . I I hope if you need , eh , something , eh , from us in the future , I I will be at Spain , to you help ,  . I guess , unless somebody has something else , we 'll read read our digits grad e: Digits ? professor b: and we 'll get our phd d:  . professor b: get our last bit of , Jose 's Jose Jose 's digit phd d: Oops . grad e: Are we gonna do them simultaneously or ? phd h: You eh professor b: I 'm sorry ? phd h: Ye - ye you prefer , eh , to eat , eh , chocolate , eh , at the coffee break , eh , at the ? Or you prefer now , before after ? postdoc c: Well , we have a time phd f: No , we prefer to keep it for ourselves . grad e: Well , we 've gotta wait until after di after we take the mikes off . grad e: So are we gonna do digits simultaneously phd a: You This is our reward if we do our digi professor b: Well ? Yeah . phd h: I I think , eh , it 's enough , eh , for more peopl for more people after . professor b: We 're gonna we 're gonna do digits at the same phd a: Oh . professor b: Alright , so in the interest of getting to the phd a: We could do digits while other people eat . grad e: It 's just the rest of the digits the rest of the digits are very clean , professor b: She is serious . phd d: Yeah ! grad e: without a lot of background noise , phd a: And it You have to write down , like , while y what you 're what ch chocolate you 're eating grad e: so I 'm just not sure phd a: cuz they might make different sounds , like n nuts chocolate with nuts , chocolate without nuts . professor b: Actually actually kind of careful cuz I have a strong allergy to nuts , so I have to sort of figure out one without th phd a: That w Oh , yeah , they they might .  phd a: This is You know , this is a different kind of speech , professor b: Well phd h: Take take several . professor b: Well well , why don't we ? He he 's worried about a ticket . phd a: You laughed at me , too , the first time I sa said professor b: I did , phd a: You really shouldn't , te professor b: and now I love it so much . grad e: OK , everyone ready ? phd a: You have to sort of ,  Jose , if you haven't done this , you have to plug your ears while you 're t talking professor b: W wait wait a minute wait a minute . Oh , you 've done this one before ? postdoc c: Hey , you 've done this before . phd d: That 's phd a: Together ? postdoc c: You 've read digits together with us , haven't you  , at the same time ? phd a: I 'm not we we Oh , and you haven't done this either . phd a: I the first time is traumatic , professor b: We phd a: but professor b: Y Yeah , bu postdoc c: Oh , and the groupings are important , phd h: Mmm . So ,  phd f: You mean that the the grouping is supposed to be synchronized ? professor b: No , no . phd f: No ? phd a: We - we 'll give everybody the same sheet phd f: It 's like a like a Greek like a Greek choir ? phd a: but they say different phd f: You know ? professor b: Yes . OK , why don't we go ?  , one two three Go ! postdoc c: OK . grad e: Did you read it twice or what ? phd a: He 's try No , he 's trying to get good recognition performance