grad g: So I think maybe what 's causing it to crash is I keep starting it and then stopping it to see if it 's working . postdoc b: And it looks like you 've found a way of  mapping the location to the without having people have to give their names each time ? phd a: Sounds like an initialization thing . postdoc b:  it 's like you have the So you know that grad g: No . postdoc b: are you going to write down that I sat here ? grad g: I 'm gonna collect the digit forms and write it down . u And I should say that  , you just pau you just read each line an and then pause briefly . professor e: So  , you see , Don , the unbridled excitement of the work that we have on this project . grad g: And I 'm surprised I sort of I 'm surprised I forgot that , professor e: Yeah , I I 'd I think it 's some grad g: but  I think that would be a good thing to add . professor e: Yeah , well , that 's  , so I I do have a a an agenda suggestion .  , we I think the things that we talk about in this meeting  tend to be a mixture of  procedural  mundane things and  research points and  I was thinking I think it was a meeting a couple of weeks ago that we we spent much of the time talking about the mundane stuff cuz that 's easier to get out of the way and then we sort of drifted into the research and maybe five minutes into that Andreas had to leave . So  I 'm suggesting we turn it around and and  sort of we have anybody has some mundane points that we could send an email later ,  hold them for a bit , and let 's talk about the the research - y kind of things .  , so  the one th one thing I know that we have on that is  we had talked a a couple weeks before   about the  the stuff you were doing with with    l l attempting to locate events , we had a little go around trying to figure out what you meant by " events " but I think , you know , what we had meant by " events " I guess was  points of overlap between speakers . But I th I gather from our discussion a little earlier today that you also mean  interruptions with something else phd d: Yeah . phd d: To professor e: So at any rate you were you 've you 've done some work on that phd d: right . professor e: and  then the other thing would be it might be nice to have a preliminary discussion of some of the other  research  areas that  we 're thinking about doing .  , I think especially since you you haven't been in in these meetings for a little bit , maybe you have some discussion of some of the p the plausible things to look at now that we 're starting to get data ,  and one of the things I know that also came up  is some discussions that that  that  Jane had with Lokendra  about some some some   work about I I I d I I don't want to try to say cuz I I 'll say it wrong , but anyway some some potential collaboration there about about the about the working with these data . Well , I don't know if we if this is sort of like everybody has something to contribute sort of thing , I think there 's just just a couple a couple people primarily  but   , wh why don't Actually I think that that last one I just said we could do fairly quickly so why don't you you start with that . postdoc b: so , he was interested in the question of you know , relating to his to the research he presented recently ,  of inference structures , and  , the need to build in , this this sort of  mechanism for understanding of language . And he gave the example in his talk about how  , e a I 'm remembering it just off the top of my head right now , but it 's something about how  , i " Joe slipped " you know , " John had washed the floor " or something like that . And I don't have it quite right , but that kind of thing , where you have to draw the inference that , OK , there 's this time sequence , but also the the the causal aspects of the  floor and and how it might have been the cause of the fall and that  it was the other person who fell than the one who cleaned it and it These sorts of things . So , I looked through the transcript that we have so far , and  , fou identified a couple different types of things of that type and  , one of them was something like  , during the course of the transcript , w we had gone through the part where everyone said which channel they were on and which device they were on , and  , the question was raised " Well , should we restart the recording at this point ? " And and Dan Ellis said , " Well , we 're just so far ahead of the game right now we really don't need to " . Now , how would you interpret that without a lot of inference ? So , the inferences that are involved are things like , OK , so , how do you interpret " ahead of the game " ? You know . So it 's the it 's i What you what you int what you draw you know , the conclusions that you need to draw are that space is involved in recording , grad g: metaphorically . postdoc b: that  , i that i we have enough space , and he continues , like " we 're so ahead of the game cuz now we have built - in downsampling " . So you have to sort of get the idea that  , " ahead of the game " is sp speaking with respect to space limitations , that  that in fact downsampling is gaining us enough space , and that therefore we can keep the recording we 've done so far . grad g: So , do you think his interest is in using this as a data source , or training material , or what ? professor e: Well , I I should maybe interject to say this started off with a discussion that I had with him , so  we were trying to think of ways that his interests could interact with ours grad g:   professor e: and   I thought that if we were going to project into the future when we had a lot of data ,  and  such things might be useful for that in or before we invested too much  effort into that he should  , with Jane 's help , look into some of the data that we 're already have and see , is there anything to this at all ? grad g:   professor e: Is there any point which you think that , you know , you could gain some advantage and some potential use for it . Cuz it could be that you 'd look through it and you say " well , this is just the wrong task for for him to pursue his " grad g: Wrong , yeah . professor e: And and  I got the impression from your mail that in fact there was enough things like this just in the little sample that that you looked at that that it 's plausible at least .  , he was he he you know We met and he was gonna go and  you know , y look through them more systematically professor e: Yeah . postdoc b: So it 's , you know , not a matter of a professor e: Yeah . professor e: So anyway , that 's that 's e a quite different thing from anything we 've talked about that , you know , might might might come out from some of this .  , he 's talking about just using text postdoc b: That 's his major I mentioned several that w had to do with implications drawn from intonational contours phd c: pretty much , or ? postdoc b: and that wasn't as directly relevant to what he 's doing . postdoc b: inferences that you draw i from professor e: he certainly could use text , but we were in fact looking to see if there is there is there something in common between our interest in meetings and his interest in in in this stuff . grad g: And I imagine that transcripts of speech  text that is speech probably has more of those than sort of prepared writing . postdoc b: Yeah , I don't think I would make that leap , because i in narratives , you know  , if you spell out everything in a narrative , it can be really tedious , grad g:   grad g: Yeah , I 'm just thinking , you know , when you 're when you 're face to face , you have a lot of backchannel and And postdoc b: Oh . And so I think it 's just easier to do that sort of broad inference jumping if it 's face to face .  , so , if I just read that Dan was saying " we 're ahead of the game " in that in that context , postdoc b: Well Yeah . grad g: I might not realize that he was talking about disk space as opposed to anything else . postdoc b: I you know , I I had several that had to do with backchannels and this wasn't one of them . postdoc b: This this one really does  m make you leap from So he said , you know , " we 're ahead of the game , w we have built - in downsampling " . postdoc b: And the inference , i if you had it written down , would be grad g: I guess it would be the same . But there are others that have backchannelling , it 's just he was less interested in those .  , I f f f I 've @ @ d A minute  , several minutes ago , I , like , briefly was was not listening and So who is " he " in this context ? phd c: Yeah , there 's a lot of pronoun phd f: OK . So I was just realizing we 've You guys have been talking about " he "  for at least  , I don't know , three three four minutes without ever mentioning the person 's name again . Actually to make it worse , Morgan uses " you " and " you " phd f: So this is this is this is gonna be a big , big problem if you want to later do  , you know , indexing , or speech understanding of any sort . phd f: You just wrote this ? phd c: and he never said Li - He looked grad g: Well , I think he 's doing that intentionally , phd c: Right , so it 's great . grad g: aren't you ? phd c: So this is really great phd f: Right . phd c: because the thing is , because he 's looking at the per even for addressees in the conversation , phd d: Yeah . phd c: Yeah , if you have the P Z Ms you should be able to pick up what a person is looking at from their voice . postdoc b: Oh , but you 'd have the phd c: Put Morgan always like this postdoc b: You 'd have fainter phd c: and postdoc b: Wouldn't you get fainter reception out here ? professor e: Well , these grad g: Sure , but I think if I 'm talking like this ? Right now I 'm looking at Jane and talking , now I 'm looking at Chuck and talking , I don't think the microphones would pick up that difference . grad g: So if I 'm talking at you , or I 'm talking at you . professor e: I probably been affect No , I th I think I 've been affected by too many conversations where we were talking about lawyers and talking about and concerns about " oh gee is somebody going to say something bad ? " and so on . professor e: And so I so I 'm I 'm tending to stay away from people 's names even though  postdoc b: I am too . phd c: Even though you could pick up later on , just from the acoustics who you were t who you were looking at . grad g: or or is that just too sensitive ? professor e: No no , it isn't sensitive at all . postdoc b: Well professor e: I was just I was just I was overreacting just because we 've been talking about it . professor e: It 's OK to postdoc b: I I came up with something from the Human Subjects people that I wanted to mention .  , it fits into the m area of the mundane , but they did say You know , I asked her very specifically about this clause of how , you know , it says " no individuals will be identified  , " in any publication using the data . " OK , well , individuals being identified , let 's say you have a a snippet that says , " Joe s  thinks such - and - such about about this field , but I think he 's wrongheaded . " Now  , we 're we 're gonna be careful not to have the " wrongheaded " part in there , but but you know , let 's say we say , you know , " Joe used to think so - and - so about this area , in his publication he says that but I think he 's changed his mind . Then the issue of of being able to trace Joe , because we know he 's well - known in this field , and all this and and tie it to the speaker , whose name was just mentioned a moment ago , can be sensitive . professor e: b But I postdoc b: So I think it 's really really kind of adaptive and wise to not mention names any more than we have to because if there 's a slanderous aspect to it , then how much to we wanna be able to have to remove ? professor e: Yeah , well , there 's that . But I  I think also to some extent it 's just educating the Human Subjects people , in a way , because there 's If  You know , there 's court transcripts , there 's there 's transcripts of radio shows  people say people 's names all the time . It 's just that i  you 're right that there 's more poten If we never say anybody 's name , then there 's no chance of of of slandering anybody , phd c: But , then it won't  , if we if we professor e: but grad g: It 's not a meeting . postdoc b: Well , my feeling on it was that it wasn't really important who said it , you know . phd f: Well , if you ha since you have to  go over the transcripts later anyway , you could make it one of the jobs of the people who do that to mark grad g: Well , we t we t we talked about this during the anon anonymization . grad g: If we wanna go through and extract from the audio and the written every time someone says a name . professor e: So it 's postdoc b: So , we need to talk about this later . I sorta knew I was doing it but it was phd f: Well , I still don't know who " he " is . phd c: No , you have to say , you still don't know who " he " is , with that prosody .  , I I would like to move it into into  what Jose  has been doing postdoc b: Yeah . I I remind that me my first objective eh , in the project is to to study difference parameters to to find a a good solution to detect eh , the overlapping zone in eh speech recorded . But eh , tsk , ehhh In that way I I I begin to to study and to analyze the ehn the recorded speech eh the different session to to find and to locate and to mark eh the the different overlapping zone . And eh so eh I was eh I am transcribing the the first session and I I have found eh , eh one thousand acoustic events , eh besides the overlapping zones , eh I I  the eh breaths eh aspiration eh , eh , talk eh , eh , clap , eh I don't know what is the different names eh you use to to name the the n speech phd a: Nonspeech sounds ? phd d: Yeah . grad g: Oh , I don't think we 've been doing it at that level of detail . Eh , I I I do I don't need to to to mmm to m to label the the different acoustic , but I prefer because eh I would like to to study if eh , I I will find eh , eh , a good eh parameters eh to detect overlapping I would like to to to test these parameters eh with the another eh , eh acoustic events , to nnn to eh to find what is the ehm the false eh , the false eh hypothesis eh , nnn , which eh are produced when we use the the ehm this eh parameter eh  pitch eh , eh , difference eh , feature grad g:   phd a: You know I think some of these  that are the nonspeech overlapping events may be difficult even for humans to tell that there 's two there . phd a: if it 's a tapping sound , you wouldn't necessarily or , you know , something like that , it 'd be it might be hard to know that it was two separate events . grad g: Well You weren't talking about just overlaps phd d: Ye grad g: were you ? You were just talking about acoustic events . phd d: I I I I t I t I talk eh about eh acoustic events in general , grad g: Someone starts , someone stops Yeah . professor e: How many overlaps were there  in it ? No no , how many of them were the overlaps of speech , though ? phd d: How many ? Eh almost eh three hundred eh in one session grad g: Oh , God ! phd d: in five eh in forty - five minutes . postdoc b: Does this ? So if you had an overlap involving three people , how many times was that counted ? phd d: Yeah , three people , two people . Eh ,  I would like to consider eh one people with difference noise eh in the background , be professor e: No no , but I think what she 's asking is if at some particular for some particular stretch you had three people talking , instead of two , did you call that one event ? phd d: Oh . This th I I I con I consider I consider eh an acoustic event , the overlapping zone , the period where three speaker or eh are talking together . grad g: Well So let 's postdoc b: For grad g: So let 's say me and Jane are talking at the same time , and then Liz starts talking also over all of us . phd d: Yeah ? grad g: Is there an event right here ? phd d: Eh no . For me is the overlapping zone , because because you you have s you have more one eh , more one voice eh , eh produced in a in in a moment . If professor e: So then , in the region between since there there is some continuous region , in between regions where there is only one person speaking . professor e: Is it Are you calling the beginning or the end of it the event , phd d: Yeah . professor e: or are you calling the entire length of it the event ? phd d: I consider the the , nnn the nnn , nnn eh , the entirety eh , eh , all all the time there were the voice has overlapped .  , I 'm not considering eh the the ehm eh , the fact of eh , eh , for example , what did you say ? Eh at first eh , eh two talkers are  , eh speaking , and eh , eh a third person eh join to to that . For me , it 's eh it 's eh , all overlap zone , with eh several numbers of speakers is eh , eh the same acoustic event . Wi - but  , without any mark between the zone of the overlapping zone with two speakers eh speaking together , and the zone with the three speakers . grad g: Well , but But you could imagine that three people talking has a different spectral characteristic than two . phd d: I I don't know what eh will will happen with the grad g: Yep . grad g: That 's a lot of overlap , phd d: Yeah ? professor e: So again , that 's that 's three three hundred in forty - five minutes that are that are speakers , just speakers . postdoc b: But a a a th professor e: So that 's about eight per minute . postdoc b: But a thousand events in twelve minutes , that 's phd d: Yeah , but Yeah . postdoc b: Well , but a thousand taps in eight minutes is a l in twelve minutes is a lot . phd c: Actually phd d: I I con I consider I consider acoustic events eh , the silent too . grad g: Silence starting or silence ending phd d: Yeah , silent , ground to bec to detect eh because I consider acoustic event all the things are not eh speech .  silent , I I I I don't I I haven't the eh I I would like to to do a stylistic study professor e: Yeah . phd d: and give you eh with the report eh from eh the the study from the the the session one session . When eh eh I w I I was eh look at eh nnn , the difference speech file , for example , eh if eh we use the ehm the mixed file , to to transcribe , the the events and the words , I I saw that eh the eh speech signal , collected by the eh this kind of mike eh of this kind of mike , eh are different from the eh mixed signal eh , we eh collected by headphone . The the the I I I knew that eh the signal eh , eh would be different , but eh the the problem is eh , eh we eh detected eh difference events in the speech file eh collected by by that mike  qui compared with the mixed file . And so if when you transcribe eh only eh using the nnn the mixed file , it 's possible eh if you use the transcription to evaluate a different system , it 's possible you eh in the eh i and you use the eh speech file collected by the eh fet mike , to eh to nnn to do the experiments with the the system , professor e:   phd d: its possible to evaluate eh , eh or to consider eh acoustic events that which you marked eh in the mixed file , but eh they don't appear in the eh speech signal eh collected by the by the mike . The the reason that I generated the mixed file was for IBM to do word level transcription , not speech event transcription . grad g: So I agree that if someone wants to do speech event transcription , that the mixed signals here phd d: Yeah . grad g: if I 'm tapping on the table , you it 's not gonna show up on any of the mikes , but it 's gonna show up rather loudly in the PZM . So and I I I say eh that eh , eh , or this eh only because eh I c I I in my opinion , it 's necessary to eh to eh to put the transcription on the speech file , collected by the objective signal . phd d:  the the the signal collected by the eh , the real mike in the future , in the prototype to to eh correct the initial eh segmentation eh with the eh real speech professor e:   professor e: Yeah , well , just  , just in that that one s ten second , or whatever it was , example that Adam had that that we we passed on to others a few months ago , there was that business where I g I guess it was Adam and Jane were talking at the same time and and  , in the close - talking mikes you couldn't hear the overlap , and in the distant mike you could . So yeah , it 's clear that if you wanna study if you wanna find all the places where there were overlap , it 's probably better to use a distant mike . professor e: On the other hand , there 's other phenomena that are going on at the same time for which it might be useful to look at the close - talking mikes , phd d: Yeah . phd c: But why can't you use the combination of the close - talking mikes , time aligned ? professor e: so it 's grad g: If you use the combination of the close - talking mikes , you would hear Jane interrupting me , but you wouldn't hear the paper rustling . And so if you 're interested in phd c: I  if you 're interested in speakers overlapping other speakers and not the other kinds of nonspeech , that 's not a problem , professor e: Some of it 's masking masked . grad g: Although the other issue is that the mixed close - talking mikes  , I 'm doing weird normalizations and things like that . phd d: I I I think eh I saw the nnn the eh but eh I eh I have eh any results . I I I saw the the speech file collected by eh the fet mike , and eh eh signal eh to eh to noise eh relation is eh low . phd d: And I I found that nnn that eh , ehm , pr probably , grad g: Did Did you phd d: I 'm not sure eh by the moment , but it 's it 's probably that eh a lot of eh , eh for example , in the overlapping zone , on eh in in several eh parts of the files where you you can find eh , eh eh , smooth eh eh speech eh from eh one eh eh talker in the in the meeting , professor e:   phd d: it 's probably in in that eh in in those files you you can not find you can not process because eh it 's confused with with noise . But eh my idea is to to process only nnn , this eh nnn , this kind of s of eh speech . I 'm not sure it 's a good idea , but eh professor e: No i grad g: Well , it 's more realistic but it 'll it 'll be a lot harder . professor e: Well , it 'd be hard , but on the other hand as you point out , if your if i if if your concern is to get  the overlapping people people 's speech , you will you will get that somewhat better . professor e: Are you making any use  you were you were working with th the data that had already been transcribed . professor e: Now  did you make any use of that ? See I was wondering cuz we st we have these ten hours of other stuff that is not yet transcribed . professor e: Do you phd d: The the transcription by Jane , t eh i eh , I I I want to use to to nnn , eh to put i i it 's a reference for me . But eh the transcription eh for example , I I don't I I 'm not interested in the in the in the words , transcription words , eh transcribed eh eh in eh follow in the in the in the speech file , but eh eh Jane eh for example eh put a mark eh at the beginning eh of each eh talker , in the in the meeting ,  eh she she nnn includes information about the zone where eh there are eh there is an overlapping zone . But eh there isn't any any mark , time temporal mark , to to c eh to mmm e - heh , to label the beginning and the end of the of the professor e:   Right , so she is phd d: ta I 'm I I I think eh we need this information to professor e: Right . So the twelve you you it took you twelve hours of course this included maybe some some time where you were learning about what what you wanted to do , but but  , it took you something like twelve hours to mark the forty - five minutes , your grad g: Twelve minutes . professor e: I thought you did forty - five minutes of phd d: No , forty - five minutes is the is the session , all the session . phd d: Tw - twelve hours of work to to segment eh and label eh twelve minutes from a session of part of f professor e: Oh . I I consider all the all the session because eh I I count the nnn the nnn the overlappings marked by by Jane , professor e: Oh , OK . So it 's three hundred in forty - five minutes , but you have you have time  ,  marked twelve minute the the the  overlaps in twelve minutes of it . phd f: So , can I ask can I ask whether you found  , you know , how accurate  Jane 's   labels were as far as grad g: Well , not just the overlaps , everything . phd f: you know , did she miss some overlaps ? or did she n ? phd d: But , by by the moment , I I don't compare , my my temporal mark with eh Jane , but eh I I want to do it . Because eh eh i per perhaps I have eh errors in the in the marks , I and if I I compare with eh Jane , it 's probably I I I can correct and and and to get eh eh a more accurately eh eh transcription in the file . phd c: Well , not only a word level , but actually phd d: Well phd f: I 'm expect I 'm not expecting phd d: No , it 's phd c: you didn't need to show the exact point of interruption , you just were showing at the level of the phrase or the level of the speech spurt , or grad g: Right . postdoc b: And and sometimes , you know , it was like you could have an overlap where someone said something in the middle , phd d: Yeah . postdoc b: but , yeah , w it just wasn't important for our purposes to have it that i disrupt that unit in order to have , you know , a the words in the order in which they were spoken , it would have it would have been hard with the interface that we have . postdoc b: Now , my a Adam 's working on a of course , on a revised overlapping interface , phd d:  - huh . phd d: I I I think It 's it 's a good eh work , postdoc b: but phd d: but eh I think we need eh eh more information . phd f: I expect you to find more overlaps than than Jane grad g: Always need more for postdoc b: Yeah . I I have to go to phd f: because you 're looking at it at a much more detailed level . professor e: I have grad g: But if it takes sixty to one professor e: Well , I but I have a suggestion about that .  , obviously this is very , very time - consuming , and you 're finding lots of things which I 'm sure are gonna be very interesting , but in the interests of making progress ,  might I s how how would it affect your time if you only marked speaker overlaps ? phd d: Only . professor e: but only mark speaker Do you think that would speed it up quite a bit ? phd d: OK . I I I I w I I wanted to professor e: Do y do you think that would speed it up ?  , speed up your your your marking ? phd d: nnn , I don't understand very . professor e: Now , my suggestion was for the other thirty - three phd d: On - only to mark only to mark overlapping zone , but professor e: Yeah , and my question is , if you did that , if you followed my suggestion , would it take much less time ? phd d: Oh , yeah . professor e: Then I think it 's a good idea , because it phd d: Sure , because I I need a lot of time to to put the label or to do that . professor e: There 's there 's  continual noise  from fans and so forth , and there is  more impulsive noise from  taps and so forth phd d: Yeah . We know that all that 's there and it 's a g worthwhile thing to study , but obviously it takes a lot of time to mark all of these things . professor e: Whereas th i I would think that  you we can study more or less as a distinct phenomenon the overlapping of people talking . Then you can get the Cuz you need If it 's three hundred  i i it sounds like you probably only have fifty or sixty or seventy events right now that are really phd d: Yeah . professor e: And and you need to have a lot more than that to have any kind of  even visual sense of of what 's going on , much less any kind of reasonable statistics . phd c: Now , why do you need to mark speaker overlap by hand if you can infer it from the relative energy in the grad g: Well , that 's That 's what I was gonna bring up . phd c: you shouldn't need to do this p completely by hand , professor e: OK , yeah . professor e: So the idea was that what he was going to be doing was experimenting with different measures such as the increase in energy , such as the energy in the LPC residuals , such as  there 's a bunch of things  , increased energy is - is sort of an obvious one . professor e: and  , it 's not obvious , you could you could do the dumbest thing and get get it ninety percent of the time . But when you start going past that and trying to do better , it 's not obvious what combination of features is gonna give you the you know , the right detector . And so the i the idea of the manual marking was to say " OK this , i you know , it 's it 's really here " . phd a: But I think Liz is saying why not get it out of the transcripts ? phd c: What  is get it from the close - talking mikes . phd c: A or ge get a first pass from those , professor e: We t we t w we t we talked about that . phd c: and then go through sort of It 'd be a lot faster probably to phd f: And you can grad g: Yeah , that 's his ,  professor e: We we we talked about that . s But so it 's a bootstrapping thing and the thing is , phd c: Yeah , I just professor e: the idea was , i we i i we thought it would be useful for him to look at the data anyway , and and then whatever he could mark would be helpful , phd c: Right . You know , do you bootstrap from a simple measurement which is right most of the time and then you g do better , or do you bootstrap from some human being looking at it and then then do your simple measurements ,  from the close - talking mike . phd c: Well , that 's what I wonder , because  or how bad it is , professor e: Well phd c: be  , because that would be interesting grad g: I 'm working on a program to do that , and phd c: especially because the bottleneck is the transcription . phd c: so  it seems like one kind of project that 's not perfect , but  , that you can get the training data for pretty quickly is , you know , if you infer form the close - talking mikes where the on - off points are of speech , professor e: Right , we discussed that . phd c: you know , how can we detect that from a far - field ? grad g: And postdoc b: Oh . grad g: I 've I 've written a program to do that , phd c: OK , I 'm sorry I missed the grad g: and it ,  professor e: It 's OK . And , it seems to work , I 've I 'm sort of fiddling with the parameters , to get it to actually generate something , and I haven't I don't what I 'm working on was working on was getting it to a form where we can import it into the user interface that we have , into Transcriber . I 've worked on it for about half a day , grad h: I have to go . grad g: so give me another half day and I we 'll have something we can play with . professor e: See , this is where we really need the Meeting Recorder query stuff to be working , because we 've had these meetings and we 've had this discussion about this , and I 'm sort of remembering a little bit about what we decided , phd c: Right . phd c: It professor e: So , I think it was partly that , you know , give somebody a chance to actually look at the data and see what these are like , partly that we have e some ground truth to compare against , you know , when when he he gets his thing going , grad g: But professor e: and phd c: Well , it 's definitely good to have somebody look at it . I was just thinking as a way to speed up you know , the amount of postdoc b:   postdoc b: Was that  there m there was this already a script I believe  that Dan had written , that  handle bleedthrough ,  cuz you have this this close you have contamination from other people who speak loudly . So I I haven't tried that , but that If It it might be something it might be a good way of cleaning it up a little . postdoc b: So , some thought of maybe having Yeah , having that be a preprocessor and then run it through yours . professor e: But but that 's a refinement postdoc b: That 's what we were discussing . professor e: and I think we wanna see try the simple thing first , cuz you add this complex thing up  afterwards that does something good y y yo you sort of wanna see what the simple thing does first . professor e: But  , having having somebody have some experience , again , with with  with marking it from a human standpoint , we 're  , I don't expect Jose to to do it for  f fifty hours of of speech , but  we if  if he could speed up what he was doing by just getting the speaker overlaps so that we had it , say , for forty - five minutes , then at least we 'd have three hundred examples of it . professor e: And when when  Adam was doing his automatic thing he could then compare to that and see what it was different . phd a: You know , I did I did  something almost identical to this at one of my previous jobs , and it works pretty well .  , i almost exactly what you described , an energy detector with a median filter , you look for runs . phd a: And so I think doing that to generate these possibilities and then going through and saying yes or no on them would be a quick way to to do it . grad g: Yeah , do you have a patent on it ? phd a: It was when I was working for the government . postdoc b: Well , is this something that we could just co - opt , or is it ? phd a: Nah . I think I think it 's phd a: Yeah , he 's it it doesn't take a long time . I just thought if it was tried and true , then and he 's gone through additional levels of of development . Although if you if you have some parameters like what 's a good window size for the median filter phd a: Oh ! I have to remember . phd f: They grad g: I was doing pretty short , you know , tenth of a second , sorts of numbers . professor e: I don't know , it if if we want to  So , maybe we should move on to other other things in limited time . postdoc b: Can I ask one question about his statistics ? So so in the tw twelve minutes , if we took three hundred and divided it by four , which is about the length of twelve minutes , i  , I 'd expect like there should be seventy - five overlaps . postdoc b: Did you find  more than seventy - five overlaps in that period , or ? phd d: More than ? postdoc b: More than How many overlaps in your twelve minutes ? phd d: How many ? Eh , not @ @ I Onl - only I I transcribe eh only twelve minutes from the professor e: Yeah . phd d: but eh I I don't co eh I don't count eh the the overlap . phd d: I consider I I The the nnn The the three hundred is eh considered only you your transcription . grad g: I b I bet they 're more , because the beginning of the meeting had a lot more overlaps than than sort of the middle . grad g: Because i we 're we 're dealing with the  , in the early meetings , we 're recording while we 're saying who 's talking on what microphone , and things like that , phd d: Yeah . professor e: So so I was gonna ask , I guess about any any other things that that that either of you wanted to talk about , especially since Andreas is leaving in five minutes , that that you wanna go with . phd c: Can I just ask about the data , like very straightforward question is where we are on the amount of data and the amount of transcribed data , just cuz I 'm I wanted to get a feel for that to sort of be able to know what what can be done first and like how many meetings are we recording professor e: Right so there 's this this There 's this forty - five minute piece that Jane transcribed . phd c: and professor e: That piece was then  sent to IBM so they could transcribe so we have some comparison point . Then there 's s a larger piece that 's been recorded and  put on CD - ROM and sent  to IBM . phd c: How many meetings is that ? Like how many grad g: What 's that ? professor e: That was about ten hours , and there was about phd c: t ten It 's like ten meetings or something ?  - huh . And then then we phd a: Ten meetings that have been sent to IBM ? phd c: And professor e: Yeah . grad g: Well , I haven't sent them yet because I was having this problem with the missing files . phd a: H how many total have we recorded now , altogether ? professor e: We 're saying about twelve hours . And we 're recording only this meeting , like continuously we 're only recording this one now ? or ? professor e: No . No , so the the that 's the that 's the biggest one  , chunk so far , grad g: Nope . phd c: Do they meet every week , professor e: And then there phd c: or every professor e: they do . w w And we talked to them about recording some more and we 're going to , we 've started having a morning meeting , today  i starting a w a week or two ago , on the  front - end issues , and we 're recording those ,  there 's a network services and applications group here who 's agreed to have their meetings recorded , phd c: Great . So actually , we 're gonna h start having a a pretty significant chunk and so , you know , Adam 's sort of struggling with trying to get things to be less buggy , and come up quicker when they do crash and stuff things like that , now that  the things are starting to happen . So right now , yeah , I th I 'd say the data is predominantly meeting meetings , but there are scattered other meetings in it and that that amount is gonna grow  so that the meeting meetings will probably ultimately i if we 're if we collect fifty or sixty hours , the meeting meetings it will probably be , you know , twenty or thirty percent of it , not not not eighty or ninety . phd c: So there 's probably there 's three to four a week , grad g: That 's what we 're aiming for . professor e: Yeah and th the the other thing is I 'm not pos I 'm sort of thinking as we 've been through this a few times , that I really don't know maybe you wanna do it once for the novelty , but I don't know if in general we wanna have meetings that we record from outside this group do the digits .  in fact , the morning group is really motivated cuz they 're working on connected digits , so it 's grad g: Actually that 's something I wanted to ask , is I have a bunch of scripts to help with the transcription of the digits . grad g: We don't have to hand - transcribe the digits because we 're reading them and I have those . grad g: And so I have some scripts that let you very quickly extract the sections of each utterance .  , if I did that , is someone gonna be working on it ? professor e: yeah , I I think definitely s so Absolutely . grad g: is it something of interest ? professor e: Yeah , whoever we have working on the acoustics for the Meeting Recorder are gonna start with that .  , I I 'm I 'm interested in it , I just don't have time to do it now . phd f: I was these meetings I 'm sure someone thought of this , but these this  reading of the numbers would be extremely helpful to do  adaptation . phd c: Actually I have o grad g: I I would really like someone to do adaptation . grad g: So if we got someone interested in that , I think it would be great for Meeting Recorder . professor e: Well  , one of the things I wanted to do , that I I talked to to Don about , is one of the possible things he could do or m also , we could have someone else do it , is to do block echo cancellation , grad g: Since it 's the same people over and over . professor e: to try to get rid of some of the effects of the the the far - field effects .  ,  we have the party line has been that echo cancellation is not the right way to handle the situation phd f:   professor e: because people move around , and  , if if it 's if it 's  not a simple echo , like a cross - talk kind of echo , but it 's actually room acoustics , it 's it 's it 's you can't really do inversion , phd f:   professor e: and even echo cancellation is going to  be something It may you Someone may be moving enough that you are not able to adapt quickly and so the tack that we 've taken is more " lets come up with feature approaches and multi - stream approaches and so forth , that will be robust to it for the recognizer and not try to create a clean signal " . But it occurred to me a few months ago that  party lines are always , you know , sort of dangerous . And so we haven't had anybody try to do a good serious job on echo cancellation and we should know how well that can do . So that 's something I 'd like somebody to do at some point , just take these digits , take the far - field mike signal , and the close  mike signal , and apply really good echo cancellation .  , there was a have been some nice talks recently by by Lucent on on their b phd f:  . professor e: the block echo cancellation particularly appealed to me ,  you know , trying and change it sample by sample , but you have some reasonable sized blocks . And  , you know , th phd a: W what is the  the artifact you try to you 're trying to get rid of when you do that ? phd f: Ciao . professor e:  so it 's it you have a a direct   , what 's the difference in If you were trying to construct a linear filter , that would  phd f: I 'm signing off . that would subtract off the   parts of the signal that were the aspects of the signal that were different between the close - talk and the distant . You know , so so   I guess in most echo cancellation Yeah , so you Given that  Yeah , so you 're trying to So you 'd There 's a a distance between the close and the distant mikes so there 's a time delay there , and after the time delay , there 's these various reflections . And if you figure out well what 's the there 's a a least squares algorithm that adjusts itself adjusts the weight so that you try to subtract essentially to subtract off  different  different reflections . Right ? So let 's take the simple case where you just had you had some  some delay in a satellite connection or something and then there 's a there 's an echo . phd a: So that would mean like if you were listening to the data that was recorded on one of those .  , just the raw data , you would you might hear kind of an echo ? And and then this noise cancellation would get professor e: Well , I 'm I 'm I 'm saying That 's a simplified version of what 's really happening . What 's really happening is Well , when I 'm talking to you right now , you 're getting the direct sound from my speech , but you 're also getting , the indirect sound that 's bounced around the room a number of times . OK ? So now , if you  try to r you To completely remove the effect of that is sort of impractical for a number of technical reasons , but I but not to try to completely remove it , that is , invert the the room response , but just to try to   eliminate some of the the effect of some of the echos .  , a number of people have done this so that , say , if you 're talking to a speakerphone ,  it makes it more like it would be , if you were talking right up to it . You say I I I want to use this  this item but I want to subtract off various kinds of echos . So you construct a filter , and you have this this filtered version  of the speech  gets   gets subtracted off from the original speech . phd a: Kind of a clean up thing , that professor e: It 's a clean up thing . professor e: So , echo cancelling is is , you know , commonly done in telephony , and and and it 's sort of the obvious thing to do in this situation if you if , you know , you 're gonna be talking some distance from a mike . phd a: When  , I would have meetings with the folks in Cambridge when I was at BBN over the phone , they had a  some kind of a special speaker phone and when they would first connect me , it would come on and we 'd hear all this noise . And then it was  And then it would come on and it was very clear , professor e: Yeah . So it 's taking samples , it 's doing adaptation , it 's adjusting weights , and then it 's getting the sum . So  ,  anyway that 's that 's kind of a reasonable thing that I 'd like to have somebody try somebody look And and the digits would be a reasonable thing to do that with . I think that 'd be enough data plenty of data to do that with , and i for that sort of task you wouldn't care whether it was  large vocabulary speech or anything .  postdoc b: Is Brian Kingsbury 's work related to that , or is it a different type of reverberation ? professor e: Brian 's Kingsbury 's work is an example of what we did f f from the opposite dogma . Right ? Which is what I was calling the " party line " , which is that  doing that sort of thing is not really what we want . We want something more flexible ,  i i where people might change their position , and there might be , you know There 's also  oh yeah , noise . It 's if you have a clean situation but you just have some delays , Then we 'll figure out the right the right set of weights for your taps for your filter in order to produce the effect of those those echos . But  if there 's noise , then the very signal that it 's looking at is corrupted so that it 's decision about what the right you know , right right  delays are is ,  is right delayed signal is is is  is incorrect . And so , in a noisy situation , also in a in a situation that 's very reverberant with long reverberation times and really long delays , it 's it 's sort of typically impractical . So for those kind of reasons , and also a a c a complete inversion , if you actually I mentioned that it 's kind of hard to really do the inversion of the room acoustics .  , that 's difficult because  often times the the  the system transfer function is such that when it 's inverted you get something that 's unstable , and so , if you you do your estimate of what the system is , and then you try to invert it , you get a filter that actually  , you know , rings , and and  goes to infinity . So it 's so there 's there 's there 's that sort of technical reason , and the fact that things move , and there 's air currents  there 's all sorts of all sorts of reasons why it 's not really practical . So for all those kinds of reasons ,  we we we sort of  , concluded we didn't want to in do inversion , and we 're even pretty skeptical of echo cancellation , which isn't really inversion , and  we decided to do this approach of taking  , just picking  features , which were  will give you more something that was more stable , in the presence of , or absence of , room reverberation , and that 's what Brian was trying to do . I guess you you actually already said this thing about the  about the consent forms , which was that we now don't have to So this was the human subjects folks who said this , or that that ? postdoc b: The a apparently  , we 're gonna do a revised form , of course .  but once a person has signed it once , then that 's valid for a certain number of meetings . So I think from a s practical standpoint , maybe we could have them do it once every ten meetings , or something . It won't be that many people who do it that often , but  just , you know , so long as they don't forget that they 've done it , I guess .  , back on the data thing , so there 's this sort of one hour , ten hour , a hundred hour sort of thing that that we have . We have we have an hour  that that is transcribed , we have we have twelve hours that 's recorded but not transcribed , and at the rate we 're going ,  by the end of the semester we 'll have , I don't know , forty or fifty or something , if we if this really  Well , do we have that much ? phd c: Not really . professor e: Let 's see , we have phd c: So that 's what You know , that professor e:  eight weeks ,  is phd c: So that 's not a lot of hours . professor e: Eight weeks times three hours is twenty - four , so that 's Yeah , so like thirty thirty hours ? phd a: Three Three hours .  I was starting to think of some projects where you would use well , similar to what we talked about with  energy detection on the close - talking mikes . There are a number of interesting questions that you can ask about how interactions happen in a meeting , that don't require any transcription . So what are the patterns , the energy patterns over the meeting ? And I 'm really interested in this but we don't have a whole lot of data . So I was thinking , you know , we 've got the room set up and you can always think of , also for political reasons , if ICSI collected you know , two hundred hours , that looks different than forty hours , even if we don't transcribe it ourselves , professor e: But I don't think we 're gonna stop at the end of this semester . phd c: so professor e: Right ? So , I th I think that if we are able to keep that up for a few months , we are gonna have more like a hundred hours . phd c: is there Are there any other meetings here that we can record , especially meetings that have some kind of conflict in them or some kind of deci  , that are less well I don't  , that have some more emotional aspects to them , or strong grad g: We had some good ones earlier . phd c: There 's laughter ,  I 'm talking more about strong differences of opinion meetings , maybe with manager types , or grad g: I think it 's hard to record those . phd c: To be allowed to record them ? postdoc b: It 's also likely that people will cancel out afterwards . professor e: Yeah , people will get postdoc b: But I but I wanted to raise the KPFA idea . So I I  , I I 'd mentioned to Adam , and that was another thing I was gonna talk  , mention to them before that  there 's  It it oc it occurred to me that we might be able to get some additional data by talking to  acquaintances in local broadcast media . Because , you know , we had talked before about the problem about using found data , that that  it 's just set up however they have it set up and we don't have any say about it and it 's typically one microphone , in a ,  or and and so it doesn't really give us the the the  characteristics we want . But  , it did occur to me that we could go to friends in broadcast media and say " hey you have this panel show , or this you know , this discussion show , and  can you record multi - channel ? " And  they may be willing to record it  with phd c: With lapel mikes or something ? professor e: Well , they probably already use lapel , but they might be able to have it it wouldn't be that weird for them to have another mike that was somewhat distant . professor e: It wouldn't be exactly this setup , but it would be that sort of thing , and what we were gonna get from UW , you know , assuming they they they start recording , isn't als also is not going to be this exact setup . the other thing that occurred to me after we had that discussion , in fact , is that it 's even possible , since of course , many radio shows are not live ,  that we could invite them to have like some of their record some of their shows here . postdoc b: Wow ! phd c: Well Or The thing is , they 're not as averse to wearing one of these head - mount  , they 're on the radio , grad g: Right , as we are . phd c: Th - that 's an a side of style a style that we 're not collecting here , so it 'd be great . professor e: And and the  , the other side to it was the what which is where we were coming from I 'll I 'll talk to you more about it later is that is that there 's there 's  the radio stations and television stations already have stuff worked out presumably ,  related to , you know , legal issues and and permissions and all that . So I think it 's something we should look into , you know , we 'll collect what we collect here hopefully they will collect more at UW also and  and maybe we have this other source . But yeah I think that it 's not unreasonable to aim at getting , you know , significantly in excess of a hundred hours . The thing was , I was hoping that we could @ @ in the under this controlled situation we could at least collect , you know , thirty to fifty hours . And at the rate we 're going we 'll get pretty close to that I think this semester . Yeah I was mostly trying to think , " OK , if you start a project , within say a month , you know , how much data do you have to work with . And you you wanna s you wanna sort of fr freeze your your data for awhile so  right now and we don't have the transcripts back yet from IBM right ? Do Oh , do we now ? professor e: Well , we don't even have it for this f you know , forty - five minutes , that was phd c: So  , not complaining , I was just trying to think , you know , what kinds of projects can you do now versus  six months from now professor e: Yeah . So I was thinking right now it 's sort of this exploratory stuff where you you look at the data , you use some primitive measures and get a feeling for what the scatter plots look like , grad g: Right . professor e: and and and  and meanwhile we collect , and it 's more like yeah , three months from now , or six months from now you can you can do a lot of other things . phd c: Cuz I 'm not actually sure , just logistically that I can spend you know , I don't wanna charge the time that I have on the project too early , before there 's enough data to make good use of the time . phd c:  anyway , I shouldn't say too much , but  if someone came that was great and wanted to do some real work and they have to end by the end of this school year in the spring , how much data will I have to work with , with that person . And so it 's professor e: i Yeah , so I would think , exploratory things now .  , three months from now  ,  the transcriptions I think are a bit of an unknown cuz we haven't gotten those back yet as far as the timing , but I think as far as the collection , it doesn't seem to me l like , unreasonable to say that  in January , you know , ro roughly  which is roughly three months from now , we should have at least something like , you know , twenty - five , thirty hours . phd c: And we just don't know about the transcription part of that , professor e: So that 's postdoc b: Yeah , we need to I think that there 's a possibility that the transcript will need to be adjusted afterwards , phd c: so .  , it postdoc b: and  es especially since these people won't be  used to dealing with multi - channel  transcriptions . postdoc b: So I think that we 'll need to adjust some And also if we wanna add things like  , well , more refined coding of overlaps , then definitely I think we should count on having an extra pass through . There 'd be no reason why a person couldn't get together several  , you know , friends , and come and argue about a topic if they wanted to , right ? professor e: If they really have something they wanna talk about as opposed to something @ @  , what we 're trying to stay away from was artificial constructions , but I think if it 's a real Why not ? Yeah . postdoc b: You could do this , phd c: Well yeah , postdoc b: you know . phd c: or just if you 're if you ha If there are meetings here that happen that we can record even if we don't  have them do the digits , or maybe have them do a shorter digit thing like if it was , you know , one string of digits , or something , they 'd probably be willing to do . grad g: We don't have to do the digits at all if we don't want to . phd c: Then , having the data is very valuable , cuz I think it 's  politically better for us to say we have this many hours of audio data , especially with the ITR , if we put in a proposal on it .  , whether it 's transcribed or not  , is another issue , but there 's there are research questions you can answer without the transcriptions , or at least that you can start to answer . phd a: Would it help at all  , we 're already talking about sort of two levels of detail in meetings . One is   without doing the digits Or , I guess the full - blown one is where you do the digits , and everything , and then talk about doing it without digits , what if we had another level , just to collect data , which is without the headsets and we just did the table - mounted stuff . I 'm really scared grad g: It seems like it 's a big part of this corpus is to have the close - talking mikes . phd c:  or at least , like , me personally ? I would I couldn't use that data . phd c: So it 's a great idea , professor e: Yeah , I I b By the by the way , I don't think the transcriptions are actually , in the long run , such a big bottleneck . phd c: and if it were true than I would just do that , but it 's not that bad like the room is not the bottleneck , and we have enough time in the room , it 's getting the people to come in and put on the and get the setup going . Right ? And and  d Do you have any idea when when  the you 'll be able to send  the ten hours to them ? grad g: Well , I 've been burning two C Ds a day , which is about all I can do with the time I have . So early next week we send it to them , and then then we check with them to see if they 've got it and we we start , you know asking about the timing for it . professor e: So I think once they get it sorted out about how they 're gonna do it , which I think they 're pretty well along on , cuz they were able to read the files and so on . professor e: Right ? grad g: Yeah , but professor e: Well grad g: Yeah , who knows where they are . professor e: Yeah , but You know , so they they they have you know , they 're volunteering their time and they have a lot of other things to do , phd c: What if grad g: Yeah , you we can't complain . professor e: right ? But they But at any rate , they 'll I I think once they get that sorted out , they 're they 're making cassettes there , then they 're handing it to someone who they who 's who is doing it , and  I think it 's not going to be I don't think it 's going to be that much more of a deal for them to do thirty hours then to do one hour , I think . phd c: Really ? So it 's the amount of professor e: It 's it 's just getting it going . phd c: I don't know , if there 's any way without too much more overhead , even if we don't ship it right away to IBM even if we just collect it here for awhile , to record you know , two or three more meeting a week , just to have the data , even if they 're  not doing the digits , but they do wear the headphones ? professor e: But the lunch meetings are pretty much one person getting up and phd c: No , I meant , sorry , the meetings where people eat their lunch downstairs , maybe they don't wanna be recorded , but grad g: Oh , and we 're just chatting ? phd c: Just the ch the chatting . phd c: I actually I actually think that 's useful data ,  the chatting , grad g: Yeah , the problem with that is I would I think I would feel a little constrained to You know ?  , some of the meetings phd c: but OK . phd c: Alright , so I 'll just throw it out there , if anyone knows of one more m or two more wee meetings per week that happen at ICSI ,  that we could record , I think it would be worth it . Well , we should also check with Mari again , because they because they were really intending , you know , maybe just didn't happen , but they were really intending to be duplicating this in some level . And there 's a lot of different meetings at UW   really m a lot more than we have here right cuz we 're not right on campus , grad g: Right . phd a: Is the  , notion of recording any of Chuck 's meetings dead in the water , or is that still a possibility ? professor e: they seem to have some problems with it .  , but , again , Jerry is Jerry 's open So  , we have two speech meetings , one  network meeting ,  Jerry was open to it but I I s One of the things that I think is a little a little bit of a limitation , there is a think when the people are not involved  in our work , we probably can't do it every week . You know ? I I I I think that that people are gonna feel  are gonna feel a little bit constrained . Now , it might get a little better if we don't have them do the digits all the time . And the then so then they can just really sort of try to put the mikes on and then just charge in and grad g: Yep . phd c: What if we give people you know , we cater a lunch in exchange for them having their meeting here or something ? postdoc b: Well , you know , I I do think eating while you 're doing a meeting is going to be increasing the noise . postdoc b: But I had another question , which is  , you know , in principle , w  , I know that you don't want artificial topics , phd c: Alright , alright , alright . postdoc b: but  it does seem to me that we might be able to get subjects from campus to come down and do something that wouldn't be too artificial . postdoc b: and i you know , people who are Because , you know , there 's also this constraint . We d it 's like , you know , the the  goldibears goldi goldilocks , it 's like you don't want meetings that are too large , but you don't want meetings that are too small . And  a and it just seems like maybe we could exploit the subj human subject p p pool , in the positive sense of the word . phd a: Well , even  , coming down from campus is sort of a big thing , but what about postdoc b: We could pay subjects . phd a: or what about people in the in the building ? phd c: Yeah , I was thinking , there 's all these other peo phd a: there 's the State of California downstairs , and phd c: Yeah .  grad g: I just really doubt that  any of the State of California meetings would be recordable and then releasable to the general public . grad g: So I  I talked with some people at the Haas Business School who are i who are interested in speech recognition phd c: Alright , well . grad g: and , they sort of hummed and hawed and said " well maybe we could have meetings down here " , but then I got email from them that said " no , we decided we 're not really interested and we don't wanna come down and hold meetings . phd a: What about Joachim , maybe he can professor e: But but we c But I think , you know , we get some scattered things from this and that . professor e:  i I have better contacts in radio than in television , but phd a: You could get a lot of lively discussions from those radio ones . phd c: Well , and they 're already they 're these things are already recorded , grad g: Yep . phd c: we don't have to ask them to even and I 'm not sure wh how they record it , but they must record from individual professor e: n Well No , I 'm not talking about ones that are already recorded . I 'm talking about new ones phd c: Why why not ? professor e: because because because we would be asking them to do something different . I know for instance Mark Liberman was interested  in in LDC getting data , and professor e: Right , that 's the found data idea . professor e: But what I 'm saying is  if I talk to people that I know who do these th who produce these things we could ask them if they could record an extra channel , let 's say , of a distant mike . So , since I 'm interested in the distant mike stuff , I wanna make sure that there is at least that somewhere phd c: Right .   professor e: and  But if we ask them to do that they might be intrigued enough by the idea that they  might be e e willing to the I might be able to talk them into it . We 're getting towards the end of our disk space , so we should think about trying to wrap up here . Well I don't why don't we why d u why don't we   turn them turn grad g: OK , leave leave them on for a moment until I turn this off , cuz that 's when it crashed last time