You know how to toggle the display width function grad a: Well maybe she hadn't just started transcribing me yet .  , so we haven't had a meeting for a while , and and probably won't have one next week , I think a number of people are gone .  , so Robert , why don't you bring us up to date on where we are with EDU ? grad b:  in a in a smaller group we had  , talked and decided about continuation of the data collection . So Fey 's time with us is almost officially over , and she brought us some thirty subjects and , t collected the data , and ten dialogues have been transcribed and can be looked at . She 's actually suggested to look  , at the psychology department students , because they have to partake in two experiments in order to fulfill some requirements . And  , we want to design it so that they really have to think about having some time , two days , for example , to plan certain things and figure out which can be done at what time , and , sort of package the whole thing in a in a re in a few more complicated  , structure . As for SmartKom , I 'm the last SmartKom meeting I mentioned that we have some problems with the synthesis , which as of this morning should be resolved . grad b: " should be " means they aren't yet , but but I think I have the info now that I need . Plus , Johno and I are meeting tomorrow , so maybe   , when tomorrow is over , we 're done . And ha n hav we 'll never have to look at it again Maybe it 'll take some more time , to be realistic , but at least we 're we 're seeing the end of the tunnel there .  , the  ,  I don't think we need to discuss the formalism that 'll be done officially s once we 're done .  , something happened , in on Eva 's side with the PRM that we 're gonna look at today , and  , we have a visitor from Bruchsal from the International University . grad a: So when you said " Andreas " I thought you were talking about Stolcke . grad b: And , grad a: Now I know that we aren't , OK . grad b: Eh grad c: Cuz there is another Andreas around , grad a:  . grad b: So my scientific director of the EML is also the dean of the International University , one of his many occupations that just contributes to the fact that he is very occupied . And , the  , he @ @ might tell us a little bit about what he 's actually doing , and why it is s somewhat related , and by  using maybe some of the same technologies that we are using .  , so , I 've be just been looking at , Ack ! What are you doing ? Yeah .  , so , this is , sort of like the latest thing I have on it , and I sorta constructed a couple of classes . Like , a user class , a site class , and and you know , a time , a route , and then and a query class . And I tried to simplify it down a little bit , so that I can actually  , look at it more .  , so basically I took out a lot of stuff , a lot of the decision nodes , and then tried to The red lines on the , graph are the  , relations between the different  , classes . Like , a user has like , a query , and then , also has , you know  , reference slots to its preferences , the special needs and , you know , money , and the user interest . And so this is more or less similar to the flat Bayes - net that I have , you know , with the input nodes and all that . And So I tried to construct the dependency models , and a lot of these stuff I got from the flat Bayes - net , and what they depend on , and it turns out , you know , the CPT 's are really big , if I do that , so I tried to see how I can do ,  put in the computational nodes in between . And so I ended up making several classes Actually , you know , a class of with different attributes that are the intermediate nodes , and one of them is like , time affordability money affordability , site availability , and the travel compatibility . And so some of these classes are s some of these attributes only depend on stuff from , say , the user , or s f just from , I don't know , like the site . S like , these here , it 's only like , user , but , if you look at travel compatibility for each of these factors , you need to look at a pair of , you know , what the  , preference of the user is versus , you know , what type of an event it is , or you know , which form of transportation the user has and whether , you know , the onsite parking matters to the user , in that case . And that makes the scenario a little different in a PRM , because , then you have one - user objects and potentially you can have many different sites in in mind . And , they all depend on the same users , but different sites , and that makes a I 'm tr I w I wa have been trying to see whether the PRM would make it more efficient if we do inferencing like that . And so , I guess you end up having fewer number of nodes than in a flat Bayes - net , cuz otherwise you would c well , it 's probably the same . But  , No , you would definitely have be able to re - use , like , all the user stuff , and not not having to recompute a lot of the stuff , because it 's all from the user side . But , you know , in the case where , it depends on both the user and the site , then I 'm still having a hard time trying to see how  , using the PRM will help .  , so anyhow , using those intermediate nodes then , this this would be the class that represent the intermediate nodes . And that would basically it 's just another class in the model , with , you know , references to the user and the site and the time . And then , after you group them together this no the dependencies would of the queries would be reduced to this .  , so I think that 's about as far as I 've gone on the PRM stuff . And one is basically how desirable a site is meaning , how good it matches the needs of a user .  , so , instead of  , doing a lot of , you know , computation about , you know , which one site it wants of the user wants to visit , I 'll come well , try to come up with like , sort of a list of sites . And for each site , you know , where h how how well it fits , and basically a rating of how well it fits and what to do with it . She 's ac   Eva 's got a little write - up on it that  , probably gives the the details to anybody who needs them .  , so the You you didn't look at all yet to see if there 's anybody has a implementation . So one so one of the questions , you know , about these P R Ms is grad d:   professor f: we aren't gonna build our own interpreter , so if if we can't find one , then we  , go off and do something else and wait until s one appears .  , so one of the things that Eva 's gonna do over the next few weeks is see if we can track that down . If there is an interpreter , it looks like you know , what Eva 's got should run and we should be able to actually  , try to solve , you know , the problems , to actually take the data , and do it .  , I actually think it is cleaner , and the ability to instantiate , you know , instance of people and sites and stuff , will help in the expression . I think there are things that you can express this way which you can't express in a normal belief - net , without going to some incredible hacking of sort of rebuilding it on the fly .  , the notion of instantiating your el elements from the ontology and stuff fits this very nicely and doesn't fit very well into the extended belief - net . So , people who have thought about the problem , like Robert i it looked to me like if Eva were able to come up with a you know , value for each of a number of  , sites plus its EVA thing , that a travel planner should be able to take it from there . And you know , with some other information about how much time the person has and whatever , and then plan a route . grad b:  -  , well , first of all  , great looks , mu much cleaner , nnn , nnn , Certain certain beauty in it , so , if beauty is truth , then ,  we 're in good shape . So if you have a write - up then  , I 'd love to read it grad d:   grad b: and  because , i Can you go all the way back to the the very top ? grad d: Yeah . grad b:  these @ @ these w w when these are instantiated they take on the same values ? that we had before ? grad d: I can't really see the whole thing . grad b: or are they have they changed , in a sense ? grad d: Well I think I basically leave them to similar things . grad d: Some of the things might that might be different , maybe like are that the hours for the site . grad d: And , eventually I meant that to mean whether they 're open at this hour or not . grad d: And status would be , you know , more or less like , whether they 're under construction , and and or stuff like that . grad b: And the , other question I would have is that presumably , from the way the Stanford people talk about it , you can put the probabilities also on the relations . I don't remember whether they carried that over to this or not , grad a: Mmm . grad b: It 's sort of in the definition or in the in Daphne 's definition of a PRM is that classes and relations , professor f: OK . grad b: and you 're gonna have CPT 's over the classes and their relations . grad d: I remember them learning when , you know , you don't know the structure for sure , professor f: Yeah . grad d: but I don't remember reading how you specify grad b: Yeah , that would be exactly my question . So , the the plan is is when Daphne gets back , we 'll get in touch and supposedly , we 'll actually get s deep seriously connected to to their work and grad b: Yep . professor f: somebody 'll  , you know If it 's a group meeting once a week probably someone 'll go down and , whatever . We get rocking and rolling on this again , once we get a package , if , when , and how , then this becomes foregrounded grad d:   grad b: And  , until then we 'll come up with a something that 's @ @ that 's way more complicated for you . grad b: Because this was laughingly easy , right ? grad d: Actually I had to take out a lot of the complicated stuff , cuz I I made it really complicated in the beginning , and Jerry was like , " this is just too much " . So , you could , from this , go on and say suppose there 's a group of people traveling together and you wanted to plan something that somehow , with some Pareto optimal  , thing for grad a: That 's good . professor f: or grad a: Except for humans can't really solve it either , so . Well that 's the that would that would be a  , you could sell it , as a grad a: Yeah . professor f: OK , eh you don't have to fight about this , just give your preferences to the grad a: And then you can blame the computer . But what does it  Would a pote potential result be to to split up and never talk to each other again ? You know . So there i there are some some u  , you know , elaborations of this that you could try to put in to this structure , but I don't think it 's worth it now . But  , it 's good , yeah and and there were a couple other ideas of of  , things for Eva to look at in in the interim . So , I 'm not really involved in any project , that 's  that 's relevant to you  , a at the moment , the the reason is really for me  , to have an opportunity to talk to some other researchers in the field . And and so I 'll just n sort of give you a real quick introduction to what I 'm working on , and  , I just hope that you have some comments or , maybe you 're interested in it to find out more , and and so I 'll be  , happy to talk to you and and  , I 'd also like to find out some more and and maybe I 'll just walk around the office and and then and ask some some questions , in a couple days . OK , so , what I started looking at , to begin with is just  , content management systems  , i i in general . So  ,  what 's  Sort of the state of the art there is to   you have a bunch of of  documents or learning units or learning objects , and you store meta - data  , associate to them . So there 's some international standards like the I - triple - E ,  There 's an I - triple - E , LON standard , and  , these fields are pretty straightforward , you have  author information , you have  , size information , format information and so on . One is  you store keywords associated with the  with the document , and one is  , you have sort of a , well , what is the document about ? So it 's some sort of taxonomic  , ordering of of the of the units . Now , if you sort of put on your semantic glasses ,  you say , well that 's not all that easy , because there 's an implicit  , assumption behind that is that  , all the users of this system share the same interpretation of the keyword and the same interpretation of  , whichever taxonomy is used , and  , I think that 's a that 's a very that 's a key point of these systems and they sort of always brush over this real quickly without really elaborating much of that and  As a matter of fact , the only thing that m apparently really works out so far are library ordering codes , which are very , very coarse grain , so you have some like , science , biology , and then But that 's really all that we have at the moment . Now , what this  a standard like this would give us is we could  , sort of  with a search engine just query  , different repositories all over the world . But we can't really  , so what I 'm what I try to do is  , to have  ,  So . So the scenario is the following , you you 're working on some sort of project and you encounter a certain problem . Now , what what we have at our university quite a bit is that  , students  , try to u program a certain assignment , for example , they always run into the same problems , and they always come running to us , and they 'll say why 's it not it 's not working , and we always give out the same answer , so we thought , well , it 'd be nice to have a system that could sort of take care of this , and so , what I want to build is basically a a smart F A Q system . Now , what you  need to do here is you need to provide some context information which is more elaborate than " I 'm looking for this and this and this keyword . I 'm I 'm sure you have the same when when somebody utters a sentence in a certain , context it , and and the same sentence in another context makes a huge difference . So , I want to be able to model information like , so in the in the context of in the context of developing distributed systems , of a at a computer science school , what kind of software is the person using , which homework assignment is he or she working on at the moment , maybe what 's the background of that student 's  , which  , which error message was encountered . Now , basically giving this   so we somehow need to have a formalized  , way of writing this down basically , and that 's where the shared interpretation of of certain terms and keywords comes in again . And , using this and some some  , knowledge about the domain I think you can do some some simple inferences . Like you know that when somebody 's working about  , working on on servlets for example , he 's using Java , cuz servlets are used are written in Java . So some some inferences like that , now , u using this you can infer more information , and you could then match this to the meta - data of  off the documents you 're you 're searching against . So ,  what I wanna do is basically have some sort of  given these inputs , and then I can compute how many documents match , and use this as a metric in the search . Now , what I plan to do is I want to  sort of do a   try to improve the quality of the search results , and I want to do this by having a depth  ,  steepest descent approach . So if I knew which operating system the person was working on , would this improve my search result ? And and having  ,  a symbolic formalized model of this I could simply compute that , and find out which  which questions are worth  , asking . And that 's what I then propagate back to the user , and and sort of try to optimize the search in this way . Now , the big problem that I 'm facing right now is  , it 's fairly easy to hack up a system  quickly , that that works in the small domain , but the problem is obviously the scalability . And   , so Robert was mentioning  , earlier today is that  , Microsoft for example with their printer set up program has a Bayesian network , which does exactly this , but there you face a problem that these are very hard to extend . And so ,  what I 'm What I try to do is basically try to model this  , in a way that you could really combine  , knowledge from very different sources , and and  , sort of looking into some of the ideas that the semantic web community  , came up with . Trying to to have  , an approach how to integrate s  certain  representation of certain concepts and also some computational rules , what you can do with those . What I 'm also looking into is a probabilistic approach into this because document retrievals is a very fuzzy procedure , so it 's probably not that easy to simply have a symbolic  , computational model . And then  ,  sort of as an add - on to this whole idea ,  that would be now , depending on what the search engine or the content repository depending on which  , which  , rules and which ontologies it it uses , or basically its view of the world ,  you can get very different results . And there you could have an idea where you actually have sort of a a peer to peer approach , where we 're all sort of carrying around our individual bookshelves , and  , if you have a question about a homework , it 's probably makes sense to ask somebody who 's in your class with you , sort of the guru in the certain area , rather than going to some Yahoo - like  , search engine . And I think a lot of the even though it 's a it 's a very different domain , but I think a lot of the , issues are are fairly similar . grad a: And so some of the I don't know how much you know about the larger Heidelberg project , I Are you grad c:  I know , yeah I know abou about it . It 's like , you know , the c context - based factors that influence how you interpret , grad c:   We - we 've kind of talked about that , but we haven't worried too much about that end of the discourse . grad b: Well , in a in one t one s mmm , small difference in a in a way , is that he doesn't have to come up with an answer , but he wants to point to the places w w grad a: Documents that have the answers . So I 'm I 'm not I 'm not building an expert grad a: you have to s still m understand what the content says about itself , and then match it to what you think the informational needs grad a:   You 're just taking the keywords as a topic text , as grad c: I I assume that that the there will be learning systems that that tag their their content . grad c: And  , m @ @ and basically what I what I envision is that you rather than just supplying a bunch of keywords you could basically for for an FAQ for example you could state sort of like a logic condition , when this document applies . So " this document explains how to set up your  , mail account on Linux " or something like this . So something something very specific that you can then But the I think that the key point with these  , learning systems is that  , a learning system is only as good as  the amount of content it it carries . grad c: You can have the best learning system with the best search interface , if there 's no content inside of it , it 's not very useful . So I think ultimately because  ,  developing these these rules and these inference  inferences I think is very costly , so  ,  I think you must be able to reuse some some existing  , domain domain information , or or or ontologies that that  other people wrote and then try to integrate them , and then also search the entire web basically , rather than just the small  , content management system . grad c: So I think that 's that 's crucial for for the success of or @ @ grad a: So , you 're not I guess I 'm trying to figure out how how it maps to the kinds of things that we 've talked about in this group , and , actually associated groups , grad c:   grad a: cuz some of us do pretty detailed linguistic analyses , and I 'm guessing that you you won't be doing that ? OK . grad a: So , you take the query , and and professor f: On the other hand , FrameNet could well be useful . professor f: Because , you know , the standard story is that keyworks keywords evoke frames , and the frames may well give you additional keywords or  , if you know that that that a a bunch of keywords  , indicate a frame , then you can find documents that actually have the whole frame , rather th than just  , individual grad c: Mmm . There 's linguists and stuff and there 's a lot of it and they 're they 're busily working away . And this looks t it seems to be that this is a place where you might be able to do that . I 'm sure I could learn a lot about  , yeah , just how to how to come up with these structures , grad a: Mmm . grad c: cuz it 's it 's very easy to whip up something quickly , but it maybe then makes sense to to me , but not to anybody else , and and if we want to share and integrate things , they must well , they must be well designed really . grad b: And Andreas and I tried to come up wi or we had come up actually with a eh with him working on an interface for FrameNet , as it was back then , that would p do some of the work for this machine , professor f: Right . grad b: which  , never got done because Prashant found a happy occupation professor f: W yeah , I know ,  it it he w he did w what what he did was much more s sensible for him . professor f: I think  , grad b: But so I 'm just saying , the  , we had that idea professor f: you know Yeah . professor f: Yeah , actually you guys never grad b: And Srini 's doing information extraction also , right ? professor f: Right . grad c: professor f: this was supposedly an exchange program , and I we you know , it 's fine . We don't care , but it just I 'm a little surprised that  , Andreas didn't come up with anyone else he wanted to send .  I had forgotten a I To be honest with you , I 'd totally forgotten we had a program . grad b:  it 's in the program ? grad c:  I I think it 's it 's really the lack of students  , at IU at the moment . I think it 's ju it 's more the lack of of students , really , and w we have all these sponsors that are always sort of eager to get some teams . grad c: Well  if if I were a student , I 'd love to come here , rather than work for some German company , or professor f: Yeah . professor f: Oh , right ! grad c: Well , I didn't say anybody to anything to offend well , except for the sponsors maybe , but professor f: Right . So I thi tha that 's that 's one of the things that might be worth looking into while you 're here . professor f: unfortunately , Srini , who is heavily involved in DAML and all this sort of stuff is himself out of town . professor f: I don't know why he @ @ oh , I , who knows ? grad a: The  professor f: Anyway , s yeah , you 'll see you 'll certainly see a lot of the people there . grad a: The other person I thought of is Dan Gildea ? because he did some work on topic spotting professor f: Yeah . I don't Depending on how well you wanna integrate with that end , grad c:   grad a: you know , like , taking the data and fig you said the learning systems that figure out We There 's someone in ICSI who actually has been working on has worked on that kinda stuff , and he 's worked with frame net , so you could talk to him about , you know , both of those things at once . grad c: So , who is that again ? grad a: And , he 's in one of the rooms on the fifth floor and stuff , grad b: Who ? I can take you to his office . professor f: Alright , was there anything else for this ? One of these times soon we 're gonna hear about construal . professor f: Good thinking ! grad b:  well , maybe I can I can bribe my way out of this . professor f: Well , no , but he 's he 's he 's he 's as you said , he 's , like the state legislature , he 's trying to offer us bribes . grad b: This  Oh , they refused the budget again ? Is it so about CITRIS ? Yeah , still nothing . professor f: this this this t the s we 're , involved in a literally three hundred million dollar  , program . And , the State of California is now a month and a half behind its legis its legally required date to approve a budget . professor f: and , yesterdayday there was this   I thought that the other branch would just approve it , but now there 's actually a little back sliding to people who who approved it got flak from there , eh anyway . So ,  Oh ! I have to tell you a wonderful story about this , OK ? And then we 'll go . In fact he 's hired to run a lot of CITRIS , even though we don't have the money they So they 've been hiring people right and left , so , they think the money 's coming . And , is now doing all the politics for CITRIS , but also , has a  , a lot of interest in  , actually doing things for society , so digital divide and stuff like that . But the really interesting thing was , he st he s he s said something about , you know I 'm interested in things that have high social multiplier , something that is of great social value . He said , " for example " , this was his only example , " if you had a adult literacy program that was as good as an individual tutor , and as compelling as a video game , then that would have a huge social impact " . So it was nice that  , he 's got this view , of A , that 's what you should try to do , and B , language would be a good way to do it . I didn't push him on the ch on the child thing , grad a:  - huh . professor f: but , you know , a again , if if you if you grad a: Oh . So this is reading , rather than teaching Another project we started on , and and didn't get funded for was , to try to build an automatic tutoring program , for kids whose first language wasn't English . So , enormous problem in California , and the idea was if we 're so smart about language understanding and speech understanding , couldn't we build  , programs that would be tutors for the kids . professor f: and  , I know none of us have the spare time to look at it right now , but it i it 's it 's interesting and I may  , talk to him some more about is em somebody already doing this , and stuff like that . So I I did manage to get pull my head out of the sling by sidetracking into CITRIS , professor f: No , no . grad b: but , I I 'll volunteer to put it right back in by stating that I am n  among some other things in the process of writing up stuff that we have been discussing at our daily meetings , professor f: Yeah . grad b: and also revising , thanks for all the comments , the c the original construal proposal . And , if I put one and one together , I may end up with a number that 's greater than one and that I I can potentially present once you get back . Yeah , so OK , so that 'd be great , but I 'd I think it 's it 's time again , right ? grad b: Absolutely . grad b: But  , and hopefully all sidetracking  , other things will have disappeared , soon


Can you  , say your name and talk into your mike one at a time ? phd c: This is Eric on channel three , I believe . postdoc e: Oh , darn , what am I doing wrong ? undergrad d: Can you see me on channel four ? Really ? grad a: Yeah , I s undergrad d: My lucky day . postdoc e: screen no , it is , oh , maybe it just warmed up ? grad a: No . postdoc e: Oh , darn , can you can't see channel five yet ? grad a: well , the mike isn't close enough to your mouth , so . postdoc e: Oh , this would be k OK , is that better ? grad a: S  , try speaking loudly , undergrad d: I like the high quality labelling . grad a: so , postdoc e: Hello , grad a: OK , good . undergrad d: David , can we borrow your labelling machine to improve the quality of the labelling a little bit here ? postdoc e: hello . phd b: One t undergrad d: How how many are there , one to five ? phd b: One five , yeah . postdoc e: Would you like to join the meeting ? grad a: Well , we don't wanna renumber them , postdoc e: I bet grad a: cuz we 've already have like , forms filled out with the numbers on them . grad a: OK , Dan , are you on ? phd b: I 'm on I 'm on two and I should be on . undergrad d: Want to join the meeting , Dave ? Do we do do we have a spare ,  grad a: And I 'm getting lots of responses on different ones , so I assume the various and assorted P Z Ms are on . undergrad d: We ' r we 're we ' r This is this this is a meeting meeting . postdoc e: This is abou we 're we 're mainly being taped but we 're gonna talk about , transcription for the m future meeting meetings . grad a: Right , Dan ? undergrad d: So , I don't understand if it 's neck mounted you don't get very good performance . grad a: What are you doing ? undergrad d: Cuz when you do this , you can Rouww - Rouww . postdoc e: Why didn't I you were saying that but I could hear you really well on the on the transcription on the , tape . grad a: Well , I m I would prefer that people wore it on their head phd b: I I don't know . grad a: It 's very badly designed so it 's phd b: It 's very badly designed ? undergrad d: What do you mean it doesn't go over the ears ? phd b: Why ? It 's not s It 's not supposed to cover up your ears . grad a: Yeah but , there 's nowhere to put the pad so it 's comfortable . phd b: it 's only badly postdoc e: So that 's what you 're d He 's got it on his temples so it cuts off his circulation . grad a: Well , you know , I 'm just that sort of digit - y g sorta guy . phd b: We didn't postdoc e: Now , just to be sure , the numbers on the back , this is the channel ? phd b: That 's the microphone number . phd b: Transcript number phd c: OK , this is Eric on microphone number three , undergrad d: This is Beck on mike four . Should I turn off the VU meter Dan ? Do you think that makes any difference ? phd b: Oh , God . grad a: Why ? Are you gonna do something other than hit " quit " ? phd b: No , but I 'm gonna look at the  , logs as well . postdoc e: you said turn off the what ? grad a: The VU meter which tells you what the levels on the various mikes are and there was one hypothesis that perhaps that the act of recording the VU meter was one of the things that contributed to the errors . undergrad d: Yeah , but Eric , you didn't think that was a reasonable hypothesis , right ? postdoc e: I See . grad a: That was me , undergrad d: Oh , I 'm sorry y grad a: I thought that was undergrad d: That was malarkey . undergrad d: Are - are yo are you recording where the table mikes are by the way ? phd b: No . undergrad d: Why not ? phd b: I think , I s I spoke to somebody , Morgan , about that . undergrad d: Why don't you just do this ? grad a: that 's what we 've done before . phd b: I know what they they 're they 're four , three , two , one . grad a: Which means we need to move this thing , and sorta decide how we 're actually going to do things . phd b: It 'll be a lot easier if we have a if we have them permanently in place or something like that . undergrad d: Is this b is this being recorded ? grad a: That 's right . undergrad d: I think Lila actually is almost getting r pretty close to even getting ready to put out the purchase order . grad a: OK , so , topic of this meeting is I wanna talk a little bit about transcription .  , I 've looked a little bit into commercial transcription services and Jane has been working on doing transcription .  , and so we wan wanna decide what we 're gonna do with that and then get an update on the electronics , and then , maybe also talk a little bit about some infrastructure and tools , and so on .  , you know , eventually we 're probably gonna wanna distribute this thing and we should decide how we 're gonna how we 're gonna handle some of these factors . phd b: Distribute what ? grad a:  ? phd b: The data ? grad a: Right .  , so we 're we 're collecting a corpus and I think it 's gonna be generally useful . grad a: and so we will undergrad d: u Using , like , audio D V Ds or something like that ? grad a: Excuse me ? phd b: Yes . Yeah , audio D V C Ds , undergrad d: Or t grad a: you know . grad a: And and so how we do we distribute the transcripts , how do we distribute the audio files , how do we how do we just do all that infrastructure ? phd c: Well , I think  , for that particular issue ther there are known sources where people go to to find these kind of things like the LDC for instance . postdoc e: Yeah , grad a: Right , but but so should we do it in the same format as LDC postdoc e: that 's right . grad a: and what does that mean to what we 've done already ? phd b: Right . The It 's not so much the actu The logistics of distribution are secondary to preparing the data in a suitable form for distribution . So , as it is , it 's sort of a ad - hoc combination of stuff Dan set and stuff I set up , which we may wanna make a little more formal . phd b: And the other thing is that , University of Washington may want to start recording meetings as well , grad a: Right . phd b: in which case w w we 'll have to decide what we 've actually got so that we can give them a copy . So I have a bunch of scripts with X Waves , and some Perl scripts , and other things that make it really easy to extract out and align where the digits are . And if U d UW 's going to do the same thing I think it 's worth while for them to do these digits tasks as well . grad a: And what I 've done is pretty ad - hoc , so we might wanna change it over to something a little more standard . undergrad d: An - and there 's interest up there ? grad a: What 's that ? undergrad d: There 's interest up there ? grad a: Well they they certainly wanna collect more data . grad a: And I would like that just to get a disjoint speaker set and a disjoint room .  , one of the things Morgan and I were talking about is we 're gonna get to know this room really well , phd c:   undergrad d: Did you notice the fan difference ? phd b: Oh , now you 've touched the fan control , now all our data 's gonna be undergrad d: Hear the difference ? grad a: Oh , it 's enormous . undergrad d: Do you wanna leave it off or not ? postdoc e: That 's better . phd b: That 's undergrad d: Yeah , the You sure ? phd b: Oh , yeah . phd b: So it could be that it 's not actually wired backwards undergrad d: That 's right . phd c: It would you know , it would real really mean that we should do short meetings when you turn off the turn off the air conditioning , grad a: Carbon monoxide poisoning ? undergrad d: Short meetings , that 's right . undergrad d: Actually , the a th air the air conditioning 's still working , that 's just an auxiliary fan . grad a: So phd c: So , in addition to this issue about the UW stuff there was announced today , via the LDC , a corpus from I believe Santa Barbara . phd c: And I don't know exactly how they recorded it but apparently there 's a lot of different styles of speech and what not . phd c: And postdoc e: They had people come in to a certain degree and they and they have DAT recorders . Right ? postdoc e: I I assume so , actually , I hadn't thought about that . Unless they added close field later on but , I 've listened to some of those data and I , I 've been I I was actually on the advisory board for when they set the project up . phd b: What 's it sound like ? postdoc e: I 'm glad to see that it got released . grad a: Yeah , I I wish postdoc e: So it it 's a very nice thing . grad a: I wish we had someone here working on adaptation phd c: S grad a: because it would nice to be able to take that stuff and adapt it to a meeting setting . You know phd c: But it may be it may be useful in postdoc e: How do you mean do you mean mechanical adaptation or grad a: No , software , to adapt the speech recognition . phd c: Well , what I was thinking is it may be useful in transcribing , if it 's far field stuff , grad a:   phd c: right ? In doing , some of our first automatic speech recognition models , it may be useful to have that kind of data postdoc e: Great idea . grad a: Well that 's not good , right ? phd c: That 's that 's not great . postdoc e: It sounds undergrad d: Tr postdoc e: well but what  is that ,  undergrad d: But far field means great distance ?  grad a: Just these . undergrad d: And so that 's why they 're getting away with just two channels or something , or are they using multiple DATs ? postdoc e: oh , good question and I can't ans answer it . grad a: So we can get that just with , media costs , undergrad d: Still a point . grad a: is that right ? phd c: in fact we get it for free grad a: Oh . phd c: So , I can I can actually arrange for it to arrive in short order if we 're postdoc e: The other thing too is from from a grad a: Well , it 's silly to do unless we 're gonna have someone to work on it , so maybe we need to think about it a little bit . postdoc e: The other thing too is that their their jus their transcription format is really nice and simple in in the discourse domain . Maybe we should maybe we should get a copy of it just to see what they did phd b: Yeah , absolutely . grad a: OK , why don't you go ahead and do that then Eric ? phd b: Absolutely . postdoc e: but , phd c: Well they 're postdoc e: this is clever . phd c: Apparently this was like phase one postdoc e: Got it through the LDC . phd c: and the there 's still more that they 're gonna do apparently or something like that unless of course they have funding issues postdoc e: Great . phd c: and then then it ma they may not do phase two but from all the web documentation it looked like , " oh , this is phase one " , whatever that means . Yeah , that  , they 're really well respected in the linguistics d side too and the discourse area , phd c: OK . phd c: But , it  it would also maybe help be helpful for Liz , if she wanted to start working on some discourse issues , you know , looking at some of this data and then , grad a: Right . grad a: Actually , that 's another thing I was thinking about is that maybe Jane should talk to Liz , to see if there are any transcription issues related to discourse that she needs to get marked . undergrad d: That would be a meeting meeting meeting ? grad a: A meeting meeting meeting . But maybe we should ,  find some day that Liz  , Liz and Andreas seem to be around more often . phd c: So maybe we should find a day when they 're gonna be here and and Morgan 's gonna be here , and we can meet , at least this subgroup . grad a: Well , I was even thinking that maybe we need to at least ping the U - dub to see phd c: We need we need to talk to them some more . grad a: you know , say " this is what we 're thinking about for our transcription " , if nothing else . So , well w shall we move on and talk a little bit about transcription then ? phd b:   What we 're using right now is a tool , from this French group , called " Transcriber " that seems to work very well .  , so it has a , nice useful Tcl - TK user interface and , undergrad d: Thi - this is the process of converting audio to text ? grad a: Right . So all all so what you have to do is just identify a segment of speech in time , and then write down what was said within it , and identify the speaker . And so the things we that we know that I know I want are the text , the start and end , and the speaker .  , and then things like repairs , and false starts , and , filled pauses , and all that other sort of stuff , we have to decide how much of that we wanna do . My my view on it was when you have a repair then ,  it seems  , we saw , there was this presentation in the one of the speech group meetings about how grad a:   postdoc e: and I think Liz has done some stuff too on that , that it ,  that you get it bracketed in terms of like well , if it 's parenthetical , which I know that Liz has worked on , then  y y you 'll have different prosodic aspects . postdoc e: And then also if it 's a r if it 's a repair where they 're like what I just did , then it 's nice to have sort of a sense of the continuity of the utterance , the start to be to the finish . And , it 's a little bit deceptive if you include the repai the pre - repair part and sometimes or of it 's in the middle . Anyway , so what I was doing was bracketing them to indicate that they were repairs which isn't  , very time - consuming . undergrad d: I is there already some sort of plan in place for how this gonna be staffed or done ? Or is it real is that what we 're talking about here ? grad a: Well , that 's part of the thing we 're talking about . So what we wanted to do was have Jane do basically one meeting 's worth , you know , forty minutes to an hour , postdoc e:   undergrad d: It this is this is like five times real time or ten times real time postdoc e: Yeah , as a pilot study . grad a: Ten times about , is and so one of the things was to get an estimate of how long it would take , and then also what tools we would use . And so the next decision which has to be made actually pretty soon is how are we gonna do it ? So . undergrad d: And so you make Jane do the first one so then she can decide , oh , we don't need all this stuff , just the words are fine . postdoc e: I wanna hear about these  , we have a g you were s continuing with the transcription conventions for s grad a: R right , so so one one option is to get linguistics grad students and undergrads to do it .  , it will require a post pass ,  people will have to look at it more than once to make sure that it 's been done correctly , but I just can't imagine that we 're gonna get anything that much better from a commercial one . undergrad d: Can't we get Joy to do it all ? grad a: Yeah right . postdoc e: No , that 's grad a: We will just get Joy and Jane to do everything . undergrad d: Is tha wasn't that what she was doing before ? Yeah , that 's right . grad a: But , you know , that 's what we 're talking about is getting some slaves who who need money undergrad d: Right . grad a: and , duh , again o postdoc e: I object to that characterization ! phd b: Oh , really . And so again , I have to say " are we recording " postdoc e: Oh , thank you . grad a: and then say , Morgan has has consistently resisted telling me how much money we have . Right ? undergrad d: Well , we already we already We already have a plan in place for the first meeting . undergrad d: Right ? That 's postdoc e: Well th there is als Yeah , really . There is also the o other possibility which is if you can provide not money but instructional experience or some other perks , you can you could get people to to  , to do it in exchange . undergrad d: Well , i b but seriously , I  , Morgan 's obviously in a bind over this and thing to do is just the field of dreams theory , which is we we go ahead as though there will be money at the time that we need the money . grad a: So at any rate , Jane was looking into the possibility of getting students , at is that right ? Talking to people about that ? postdoc e: I 'm afraid I haven't made any progress in that front yet . So ,  undergrad d: I d do So until you actually have a little experience with what this this French thing does we don't even have postdoc e: And I do have grad a: She 's already done quite a bit . So that 's where you came up with the f the ten X number ? postdoc e: yeah . undergrad d: Or is that really just a guess ? postdoc e: Actually that 's the the one people usually use , ten X . phd c: How fast are you ? postdoc e: And I haven't really calculated How fast am I ? undergrad d: Yeah i postdoc e: I haven't done a s see , I 've been at the same time doing kind of a boot strapping in deciding on the transcription conventions that that are you know , and and stuff like , you know , how much phd b: Mmm . postdoc e: There 's some interesting human factors problems like , yeah , what span of of time is it useful to segment the thing into in order to  , transcribe it the most quickly . postdoc e: Cuz then , you know , you get like if you get a span of five words , that 's easy . And then there 's the issue of it 's easier to hear it th right the first time if you 've marked it at a boundary instead of somewhere in the middle , phd b:   postdoc e: cuz then the word 's bisected or whatever and And so  , I 've been sort of playing with , different ways of mar cuz I 'm thinking , you know , if you could get optimal instructions you could cut back on the number of hours it would take . undergrad d: D does  this tool you 're using is strictly it doesn't do any speech recognition does it ? grad a: No . undergrad d: But but is there anyway to to wire a speech recognizer up to it and actually run it through postdoc e: That 's an interesting idea . grad a: We 've we 've thought about doing that postdoc e: Hey ! grad a: but the recognition quality is gonna be horrendous . undergrad d: First of all the time marking you 'd get you could get by a tool . undergrad d: And so if the if if the issue really postdoc e: That 's interesting . undergrad d: I 'm think about the close caption that you see running by on on live news casts . And in a lot of them you see typos and things like that , grad a:   undergrad d: but it but it occurs to me that it may be a lot easier to correct things than it is to do things from scratch , no matter how wonderful the tool is . Yeah , we undergrad d: But if if there was a way to merge the two phd c: Well , but sometimes it 's easier to type out something instead of going through and figuring out which is the right grad a: we 've talked about it postdoc e: That 'd be fun . grad a: but phd c: it depends on the error rate , right ? undergrad d: Well s but but again the timing is for fr should be for free . grad a: We don't care about the timing of the words , just of the utterances . postdoc e: No ,  the the boundary phd c: We cut it s s phd b: We don't we don't know , actually . phd b: We haven't decided which which time we care about , and that 's kind of one of the things that you 're saying , is like you have the option to put in more or less timing data and , be in the absence of more specific instructions , we 're trying to figure out what the most convenient thing to do is . grad a: Yeah , so so what what she 's done so far , is sort of more or less breath g not breath groups , sort of phrases , continuous phrases . grad a: And so , that 's nice because you you separate when you do an extract , you get a little silence on either end . postdoc e: Although I was I you know , the alternative , which I was sort of experimenting with before I ran out of time , recently was ,  that , you know , ev if it were like an arbitrary segment of time i t pre - marked cuz it does take time to put those markings in . postdoc e: It 's really the i the interface is wonderful because , you know , the time it takes is you listen to it , and then you press the return key . But then , you know , it 's like , you press the tab key to stop the flow and and , the return key to p to put in a marking of the boundary . But , you know , obviously there 's a lag between when you hear it and when you can press the return key phd b: Yeah . postdoc e: so it 's slightly delayed , so then you you listen to it a second time and move it over to here . undergrad d: i a postdoc e: Now if it could all be pre - marked at some , l you know , good undergrad d: ar but grad a:  . undergrad d: Are are those d delays adjustable ? Those delays adjustable ? See a lot of people who actually build stuff with human computer interfaces understand that delay , phd b: Yeah . undergrad d: and and so when you by the time you click it it 'll be right on because it 'll go back in time to put the postdoc e: Yeah . postdoc e: It has other grad a: couldn't we Dan ? Yeah , mis Mister TCL ? phd b: Yeah . grad a: But ,  postdoc e: it 's not bad grad a: But , if we tried to do automatic speaker ID . phd b: Yeah , yeah , but grad a: But that would be phd b: But we 've got we 've got the most channel data . postdoc e: Oh , good point ! Ah ! grad a: Yeah , I guess the question is how much time will it really save us versus the time to write all the tools to do it . but the chances are if we if we 're talking about collecting ten or a hundred hours , which is going to take a hundred or a thousand hours to transcribe undergrad d: If grad a: But undergrad d: if we can go from ten X to five X we 're doing a big grad a: We 're gonna need we 're gonna need ten to a hundred hours to train the tools , and validate the tools the do the d to to do all this anyway . phd b: If we 're just doing silence detection postdoc e: But but it op grad a: I knew you were gonna do that .  , it it 's it 's maybe like a week 's work to get to do something like this . postdoc e: Could you get it so that with so it would it would detect volume on a channel and insert a marker ? And the the format 's really transparent . postdoc e: It 's just a matter of a very c clear it 's XML , isn't it ? grad a:   postdoc e: It 's very  , I looked at the the file format and it 's just it has a t a time a time indication and then something or other , and then an end time or something or other . phd c: and undergrad d: Is this already in the past or already in the future ? phd c: Already in the past . undergrad d: You 've already you 've already done some ? grad a: She 's she 's done about half a meeting . phd c: She she 's done one she 's one postdoc e: Yes I have . grad a: About half ? phd c: I 'm go postdoc e: S I 'm not sure if it 's that 's much but anyway , enough to work with . And then go through  , and go through and and try and re - transcribe it , given that we had perfect boundary detection . undergrad d: And forgetting all the words because you 've been thr postdoc e: Yeah , that 's what I was thinking . phd c: Yeah , that 's part of the problem is , is that what we really need is somebody else to come along . phd b: Well , no , you should do it you should do it Do it again from scratch and then do it again at the boundaries . undergrad d: And so you 'll get you 'll get down to one point two X by the time you get done . undergrad d: No , but the thing is the fact that she 's she 's did it before just might give a lower bound . undergrad d: It 's And if the lower bound is nine X then w it 's a waste of time . postdoc e: Well ,  but there 's an extra problem which is that I didn't really keep accurate phd b: Oh ! postdoc e: it wasn't a pure task the first time , phd b: Yeah . So I think though it 's a good proposal to be used on a new a new batch of text that I haven't yet done yet in the same meeting . phd b: The point we where do we get the the the oracle boundaries from ? phd c: Right . grad a: Yeah , one person would have to assign the boundaries and the and the other person would have to postdoc e: Well , but couldn't I do it for the next phd b: We we we could get fake grad a:  that 's easy enough . postdoc e: Well , but the oracle boundaries would come from volume on a partic specific channel wouldn't they ? grad a: No , no . You wanna know given Given a perfect human segmentation , you wanna know how well postdoc e: Yeah . phd c: the the question is , is it worth giving you the segmentation ? postdoc e: Oh , I see what you mean . grad a: I could generate the segmentation and and you could do the words , and time yourself on it . grad a: That would at least tell us whether it 's worth spending a week or two trying to get a tool , that will compute the segmentations . undergrad d: And the thing to keep in mind too about this tool , guys is that sure , you can do the computation for what we 're gonna do in the future but if if UW 's talking about doing two , or three , or five times as much stuff and they can use the same tool , then obviously there 's a real multiplier there . postdoc e: And the other thing too is with with speaker identification , if if that could handle speaker identification that 's a big deal . postdoc e: That 's a major that 's like , one of the two things that phd c: there 's gonna there 's gonna be in the meeting , like the reading group meeting that we had the other day , that 's it 's gonna be a bit of a problem phd b: OK . grad a: That undergrad d: But you didn't say anything worth while anyway , right ? grad a: That 'll s phd b: Right . phd b: So I I need to we need to look at what what the final output is but it seems like we it doesn't it seems like it 's not really not that hard to have an automatic tool to generate the phrase marks , and the speaker , and speaker identity without putting in the words . I 've already become pretty familiar with the format , postdoc e: That 'd be so great . grad a: If you 'd tell me where it is , huh ? postdoc e: We didn't finish the the part of work already completed on this , did we ?  , you you talked a little bit about the transcription conventions , grad a:   postdoc e: and , I guess you 've mentioned in your progress report , or status report , that you had written a script to convert it into So , I when I i the it 's quickest for me in terms of the transcription part to say something like , you know , if if Adam spoke to ,  to just say , " A colon " , Like who could be , you know ,  at the beginning of the line . postdoc e: and E colon instead of entering the interface for speaker identification and clicking on the thing , indicating the speaker ID . So , and then he has a script that will convert it into the the thing that , would indicate speaker ID .  , I looked at Cyber Transcriber postdoc e: Yeah , well that 's true , but grad a: which is a service that you send an audio file , they do a first - pass speech recognition . postdoc e: What i just approximately , what did you find out in terms of price or or whatever ? grad a: Well , for Cyber Transcriber they don't quote a price . postdoc e: Or of action ? grad a: For thirty dollars an hour for of their work . Oh , of their grad a: So so if it 's ten times it 's three hundred dollars an hour . postdoc e: Oh ! phd c: So that 's three that 's three hours . undergrad d: D did you talk to anybody that does closed captioning for for  , TV ? phd c: Right . undergrad d: Cuz they a usually at the end of the show they 'll tell what the name of the company is , the captioning company that 's doing it . undergrad d: Well , you know , the the thing the thing about this is thinking kind of , maybe a little more globally than I should here but that really this could be a big contribution we could make .  , we 've been through the STP thing , we know what it what it 's like to to manage the manage the process , and admittedly they might have been looking for more detail than what we 're looking for here but it was a it was a big hassle , right ? phd b: Yeah . And it 's it 's only our time , where " our " of course includes Dan , Dan and you guys . j Just seems like phd b: Yeah ,  I don't know if we 'd be able to do any thing f to help STP type problems . But certainly for this problem we can do a lot better than undergrad d: Bec Why ? Because they wanted a lot more detail ? grad a: Right . Because they had because they only had two speakers , right ?  , the the segmentation problem is grad a: Trivial . So what took them so long ? grad a: mostly because they were doing much lower level time . grad a: And so we 're w we decided early on that we were not gonna do that . But there 's still the same issue of managing the process , of of reviewing and keeping the files straight , and all this stuff , that which is clearly a hassle . And so so what I 'm saying is that if we hire an external service I think we can expect three hundred dollars an hour . phd c: And who who knows if they 're gonna be able to m manage multal multiple channel data ? phd b: Yeah , they won't . phd b: No , but  , they they they won't they won't they will refuse to transcribe this kind of material . postdoc e: And then there 's the problem also that phd b: That 's not what they 're d quoting for , right ? grad a: Yes , it is . undergrad d: Well , they might they might quote it phd b: For quoting meetings ? grad a: Sev - several of them say that they 'll do meetings , and conferences , and s and so on . undergrad d: Th - th the th there may be just multiplier for five people costs twice as much and for ten people co Something like that . So what they had is , if it 's an easy task it costs twenty - four dollars an hour and it will take maybe five or six times real time . And what they said is for the hardest tasks , bad acoustics , meeting settings , it 's thirty - two dollars an hour and it takes about ten times real time . undergrad d: A lot of companies I 've worked for y the ,  the person leading the meeting , the executive or whatever , would sort of go around the room and and mentally calculate h how many dollars per hour this meeting was costing , grad a: So . But you know , it 's a lot like , " he 's worth fifty an hour , he 's worth " And so he so here we 're thinking , " well let 's see , if the meeting goes another hour it 's going to be another thousand dollars . " You know ? It 's grad a: Yep , we have to have a short meeting . But at any rate , so we we have a ballpark on how much it would cost if we send it out . undergrad d: And we 're talking about do doing how many hours worth of meetings ? grad a: Thirty or forty . phd c: So it 's thirty dollars an hour , essentially , right ? undergrad d: Yeah . And the question is what 's the difference phd b: How how much lower are they ? phd c: or ei eight dollars . What do you know what the going rate is ? It 's it 's on the order of eight to ten . postdoc e: yeah , I was gonna say eight you 'd say ten ? phd c: Let 's say ten . undergrad d: The - these are not for engineering graduate students , right ? grad a: Right , these are linguistics grad students . phd c: Yeah , I I I don't I don't know what the I don't know what the standard undergrad d: That 's right . phd c: so that means that even if it takes them thirty times real time it 's cheaper to to do graduate students . grad a: that 's why I said originally , that I couldn't imagine sending it out 's gonna be cheaper . postdoc e: The other thing too is that , if they were linguistics they 'd be you know , in terms of like the post editing , i  tu  content wise they might be easier to handle cuz they might get it more right the first time . grad a: we can't tell them , you know , " for this meeting we really wanna mark stress postdoc e: Good point . grad a: And and they 're not gonna provide they 're not gonna provide stress , they 're not gonna re provide repairs , they 're not gonna provide they they may or may not provide speaker ID . undergrad d: Just hypoth hypothetically assuming that that we go ahead and ended up using graduate students . I who who 's the person in charge ? Who 's gonna be the Steve here ? grad a: I hope it 's Jane . undergrad d: You ? grad a: Is that alright ? postdoc e: Oh , interesting .  , now would this involve some manner of  , monetary compensation or would I be the voluntary , coordinator of multiple transcribers for checking ? grad a: I would imagine there would be some monetary involved but we 'd have to talk to Morgan about it . See ? postdoc e: Oh , undergrad d: That 's why Dave should have been here . grad a: Well , I would like you to do it because you have a lot more experience than I do , postdoc e: Oh , cool . grad a: but if if that 's not feasible , I will do it with you as an advisor . undergrad d: W we 'd like you to do it and we 'd like to pay you . phd c: That 's a undergrad d: And and then postdoc e: OK . undergrad d: an an an and be and be sure and say , would you like fries with that when you 're thinking about your pay scale . Yeah , no , that I I would be interested in that in becoming involved in the project in some aspect like that grad a: OK . grad a: any more on transcript we wanna talk about ? phd b: What s so what are you so you 've done some portion of the first meeting . phd b: To carry on doing it ? postdoc e: What Well , you know what I thought was right now we have p So I gave him the proposal for the transcription conventions . And so my next plan would be phd b: What what do they what do they cover ? postdoc e: They 're very minimal . postdoc e: and this is a way which meshes well with with , making it so that , you know , on the At the phd b: Yeah . When you when you get the ,  you you get the speech signal you also get down beneath it , an indication of , if you have two speakers overlapping in a s in a single segment , you see them one displayed one above each other . And then at the same time the top s part of the screen is the actual verbatim thing . You can clip click on individual utterances and it 'll take you immediately to that part of the speech signal , and play it for you . undergrad d: Is there a limit to the number of speakers ? grad a: the user interface only allows two . And so , using this the convention that Jane and I have discussed , you can have as many overlapping speakers as you want . undergrad d: Th - this is the French software , right ? grad a: Yeah , French . phd b: Oh , really ? undergrad d: did you ask them to change the interface for more speakers ? postdoc e: Oh . grad a: Yes , and they said that 's on in in the works for the next version . grad a: Multichannels was also Well , they said they wanted to do it but that the code is really very organized around single channels . undergrad d: Do - do you know what they 're using it for ? Why 'd they develop it ? grad a: For this exact task ? phd c: For transcription . undergrad d: Are they linguists ? phd c: It 's undergrad d: But  , are they are they linguists or are they speech recognition people ? grad a: I think they 're linguists . phd c: They 're they have some connection to the LDC cuz the LDC has been advising them on this process , the Linguistic Data Consortium . grad a: And they have they 've actually asked if we are willing to do any development and I said , well , maybe . grad a: So if we want if we did if we did something like programmed in a delay , which actually I think is a great idea , I 'm sure they would want that incorporated back in . You know , they they do have So you have when you when you play it back , it 's it is useful to have , a a break mark to se segment it . But it wouldn't be strictly necessary cuz you can use the  , the tabbed key to toggle the sound on and off . And then also once you 've put a break in then you have the option of cycling through the unit . undergrad d: Loop it ? Yo - you n you know , there 's al also the the user interface that 's missing . postdoc e: Or or or undergrad d: It 's missing from all of our offices , and that is some sort of analog input for something like this . It 's something that wh when you move your hand further , the sound goes faster past it , like fast forward . You know , like a joy stick or a  , you could wire a mouse or trackball to do something like that . postdoc e: Why , that 's That 's not something I wanted to have happen . undergrad d: No , but I 'm saying if this is what professionals who actually do this kind of thing for for for m for video or for audio where you you need to do this , postdoc e: I see . undergrad d: and so you get very good at sort of jostling back and forth , rather than hitting tab , and backspace , and carriage return , and enter , and things like that . grad a: Yeah , we talked about things like foot pedals and other analog phd c: Yeah . grad a: So  , tho those are things we could do but I I just don't know how much it 's worth doing . postdoc e: They thought about and also it 'll go around the c the , I wanna say cursor but I 'm not sure if that 's the right thing . postdoc e: Anyway , you can so they thought about different ways of having windows that you c  work within , phd b:   postdoc e: and But so in terms of the con the conventions , then , basically , it 's strictly orthographic which means with some w provisions for , w  , colloquial forms . So if a person said , " cuz " instead of " because " then I put a an apostrophe at the beginning of the word and then in in double ang angle brackets what the full lexical item would be . postdoc e: And this could be something that was handled by a table or something but I think to have a convention marking it as a non - standard or wha I don't mean standard but a a a non  , ortho orthographic , whatever . And and there would be limits to how much refinement you want in indicating something as non - standard pres pronunciation . phd c: How are you handling backchannels ? postdoc e: Backchannels ? grad a: Comments . postdoc e: you know oh , yes , there was some in my view , when i when you 've got it densely overlapping , I didn't worry about I didn't worry about s specific start times . phd c: What do you mean by du postdoc e: I sort of thought that this is not gonna be easily processed anyway and maybe I shouldn't spend too much time getting exactly when the person said " no " , or , you know , i " immediate " . postdoc e: And instead just sort of rendered " within this time slot , there were two people speaking during part of it phd b: Yeah . postdoc e: and if you want more detail , figure it out for yourself " , phd b:   grad a: Well , I think what w what Eric was talking about was channels other than the direct speech , phd c: I see . postdoc e: was sort of the way I felt @ @ grad a: right ? phd c: Well , yeah , what  is wh  , when somebody says "  - huh " in the middle of , a @ @ grad a: Yep . phd c: Oh , cuz I was I was listening to Dan was agreeing a lot to things that you were saying as you were talking . Well , if it if there was a word like " right " , you know , then I wou I would indicate that it happened within the same tem time frame grad a: Yeah , there 's an overlapping mark . grad a: when when no one i when we 're not actually in the meeting , and we 're all sort of separated , and and doing things . But even during the meeting there 's a lot of overlap but it it 's marked pretty clearly .  , some of the backchannel stuff Jane had some comments and but I think a lot of them were because you were at the meeting . grad a: Yeah , but someone who , was just the transcriber wouldn't have known that . grad a: Or when Dan said , " I wa I wasn't talking to you " . undergrad d: So you take a bathroom break in the middle and and keep your head mount grad a: You have to turn off your mike . postdoc e: Well he was so so he was checking the meter levels and and we were handling things while he was labeling the the whatever it was , the PDA ? grad a:   postdoc e:  And and so he was in sort of you were sort of talking you know , so I was saying , like " and I could label this one left . grad a: But postdoc e: But really , no ,  w you know in the context if you know he can't hear what he 's saying grad a: but when you w when you listen to it undergrad d: he he It was a lot funnier if you were there though . postdoc e: yeah , grad a: Well what what it what happens is if you 're a transcriber listening to it it sounds like Dan is just being a total totally impolite . grad a:  But but if you knew that that I wasn't actually in the room , and that Dan wasn't talking to me , it it became OK . undergrad d: So th postdoc e: And that 's w that 's where I added comments . postdoc e: The rest of the time I didn't bother with who was talking to who but but this was unusual circum circumstance . undergrad d: So this is this is gonna go on the meeting meeting transcriber bloopers tape , right ? grad a: Yes . postdoc e: Well and part of it was funny ,  reason was because it was a mixed signal so you couldn't get any clues from volume that , you know , he was really far away from this conversation . I should rewrite the mix tool to put half the people in one channel and half in the other . I have a auto - gain - mixer tool that mixes all the head mounted microphones into one signal postdoc e: That 's a good idea . postdoc e: But I thought it would be you know , I I didn't wanna add more contextual comments than were needed but that , it seemed to me , clarified that the con what was going on . And ,  OK , phd c: So , s postdoc e: so normalization phd c: I was just gonna ask , so I just wanted to c sort of finish off the question I had about backchannels , phd b: Mmm . phd c: which which was , so say somebody 's talking for a while postdoc e: Yeah . phd c: and somebody goes "  -  " in the middle of it , and and and what not , does the conversation come out from the or the person who 's speaking for the long time as one segment and then there 's this little tiny segment of this other speaker or does it does the fact that there 's a backchannel split the the the it in two . postdoc e: OK , my my focus was to try and maintain conten con content continuity and , to keep it within what he was saying . Like I wouldn't say breath groups but prosodic or intonational groups as much as possible . So if someone said "  -  " in the middle of a of someone 's , intonational contour , I I indicated it as , like what you just did . postdoc e: then I indicated it as a segment which contained @ @ this utterance plus an overlap . phd b: But that 's but there 's only one there 's only one time boundary for both speakers , phd c: OK . And you know , it could be made more precise than that phd c: I see , postdoc e: but I just thought phd c: I see , OK . undergrad d: I think whenever we use these speech words we should always do the thing like you 're talking about , accent , postdoc e: Oh , I see what you mean . OK , and so then , in terms of like words like "  " and "  " I just wrote them because I figured there 's a limited number , and I keep them to a  , limited set because it didn't matter if it was " mmm " or "  " , you know , versus "  " . It 'd be good to have that in the in the conventions , what 's to be used . grad a: I I did notice that there were some segments that had pauses on the beginning and end . undergrad d: Well , what 's that mean ? postdoc e: Yeah , OK . phd b: We have to mark those ? postdoc e: So I had phd b: Don't they d can't we just leave them unmarked ? postdoc e: I d Well , you see , that 's possible too . grad a: Well , I wanna leave the marked I don't want them to be part of another utterance . Now that 's refinement that , maybe it could be handled by part of the part of the script or something more phd b: yeah , it seems like it seems like the , tran the transcription problem would be very different if we had these automatic speaker detection turn placing things . Because suddenly  , I don't know , actually it sounds like there might be a problem putting it into the software if the software only handles two parallel channels . Well you were saying , I think it can read grad a: It can read and write as many as you want , it 's just that it postdoc e:  - huh . phd b: But what if you wanna edit it ? Right ?  , the point is we 're gonna generate this transcript with five five tracks in it , but with no words .  , and if there are five five people speaking at once , grad a: Right , i it 's I didn't explain it well . If we use the the little the conventions that Jane has established , I have a script that will convert from that convention to their saved convention . postdoc e: And it can be m edited after the fact , grad a: Yes . postdoc e: can't it also ? But their but their format , if you wanted to in indicate the speakers right there instead of doing it through this indirect route , then i they a c window comes up and it only allows you to enter two speakers . undergrad d: But you 're saying that by the time you call it back in to from their saved format it opens up a window with window with five speakers ? postdoc e: So . undergrad d: Oh ! That is sort of f grad a: It 's just user interface . undergrad d: They didn't quite go the whole grad a: So i it 's undergrad d: Yeah , they didn't go the whole route , grad a: the the the whole saved form the saved format and the internal format , all that stuff , handles multiple speakers . undergrad d: did they ? They just grad a: It 's just there 's no user interface for specifying multiple any more than two . So your your script solves Doesn't it solve all our problems , postdoc e: And that grad a: Yep . undergrad d: cuz we 're always gonna wanna go through this preprocessing grad a: Yep . Oh , yes , I I wanted to have So sometimes a pers I  in terms of like the continuity of thought for transcriptions , it 's i it isn't just words coming out , it 's like there 's some purpose for an utterance . And sometimes someone will do a backchannel in the middle of it but you wanna show that it 's continued at a later point . So I have I have a convention of putting like a dash arrow just to indicate that this person 's utterance continues . And then when it  , catches back up again then there 's an arrow dash , and then you have the opposite direction to indicate continuation of ones own utterance versus , sometimes we had the situation which is you know , which you which you get in conversations , of someone continuing someone else 's utterance , phd b: Mmm . postdoc e: and in that case I did a tilde arrow versus a arrow tilde , to indicate that it was continuation but it wasn't Oh , I guess I did equal arrow for the for the own for yourself things phd b:   postdoc e: And then you could track whether it was the same speaker or not by knowing you know , at the end of this unit you 'd know what happened later . grad a: So phd b: But the only time that becomes ambiguous is if you have two speakers . Like , if you If you only have one person , if you only have one thought that 's continuing across a particular time boundary , you just need one arrow at each end , and if it 's picked up by a different speaker , it 's picked up by a different speaker . I guess if you have more than one thread going , then you then you need to know whether they were swapped or not . You know , undergrad d: It l ou grad a: if you were trying to do a remembrance agent . grad a: But , a lot of these issues , I think that for  , from my point of view , where I just wanna do speech recognition and information retrieval , it doesn't really matter . I you know I did this I did this transcription and I marked that , I marked it with ellipsis because it seemed like there was a difference . It 's something you wanted to indicate that it that I this was the end of the phrase , this was the end of that particular transcript , but it was continued later . Well that 's you know ,  I that 's why I didn't I didn't do it n  , that 's why I thought about it , and and re - ev phd b: Yeah , yeah . postdoc e: and it didn't do I didn't do it in ten times the the time . grad a: Well , so anyway , are we interested then in writing tools to try to generate any of this stuff automatically ? postdoc e: Yeah . postdoc e: I also wanted to ask you if you have a time estimate on the part that you transcribed . Do you have a sense of how long phd b: Yeah , it took me half an hour to transcribe a minute , but I didn't have any I didn't even have a postdoc e: OK . So I was doing it by typ typing into a text file and trying to fit It was horrible . postdoc e: Well , that 's that 's because you didn't have the segmentation help and all the other grad a: But I think for a first try that 's about right . phd b: Is it phd c: So so if we hired a who if we hired a whole bunch of Dan 's undergrad d: That 's right . phd b: It was actually it was quite it was a t undergrad d: a grad a: If we hire an infinite number of Dan 's phd b: it w undergrad d: It 'd b a a postdoc e: And there 's always a warm up thing of grad a: Are we gonna run out of disk space by the way ? phd b: Yeah . undergrad d: d Doesn't it beep in the other room when you 're out of disk space ? phd c: So Is there grad a: No . phd c: Maybe we should s consider also , starting to build up a web site around all of these things . phd b: We could have like business - to - business E - commerce as well ! phd c: That 's right . No , but I 'm it would be interesting it would be interesting to see grad a: Can we sell banner ads ? undergrad d: Get get paid for click - throughs ? grad a: What a good idea , phd b: Yeah . phd c: I want to introduce I I want to introduce the word " snot - head " into the conversation at this point . phd b: We can have undergrad d: You wanna word that won't be recognized ? phd c: You see , cuz  , cuz Exactly . Alright , see here 's here 's here 's my thought behind it which is that , the the stuff that you 've been describing , Jane , I gu one has to , of course indicate , i is very interesting , postdoc e: Alright . phd c: and I I 'd like to be able to to pore through , you know , the the types of tr conventions that you 've come up with and stuff like that . postdoc e: OK , now , w the alternative to a web site would be to put it in Doctor speech . postdoc e: Cuz cuz what I have is a soft link to my transcription that I have on my account phd c: Either 's fine . undergrad d: Web site 's what ? phd b: We could actually maybe we could use the TCL plug - in . See he said the word TCL and and that 's undergrad d: But he does such a good job of it . But I should be allowed to but undergrad d: If you just did a crappy job , no nobody would want you to do it . phd b: I sh I shouldn't be allowed to by m by my own by my according to my own priorities . grad a: And we have we have quite a disparate number of web and other sorts of documents on this project sort of spread around . grad a: and phd c: Right , so we can add in links and stuff like that to other things . phd c: The phd b: Well so then th grad a: Try try to s consolidate . phd c: Well , we could put we could put sort of a disorganized sort of group gestalt undergrad d: Why ? What what 's what 's the issue ? phd b: No one owns the project . undergrad d: W well Do But grad a: It 's mine ! All mine ! phd b: Well then you have to do the web site . undergrad d: But grad a: " Wah - hah - hah - hah - hah - hah . undergrad d: b but but but what are you what are you talking about for web site hacking ? phd b: No undergrad d: You 're talking about writing HTML , right ? grad a: Yeah , I I 'm talking about putting together all the data in a form that that is legible , and pleasant to read , and up to date , and et cetera , et cetera , et cetera . undergrad d: But , is it against the law to actually use a tool to help your job go easier ? grad a: Absolutely . undergrad d: You y grad a: It 's just as easy to use to edit the raw HTML as anything else . undergrad d: No kidding ? phd b: That 's obviously not true , grad a: It 's obviously not true . phd b: but you have undergrad d: No , it it it 's obviously true that he hasn't found any he likes . postdoc e: Which one do you use Jim ? undergrad d: I use something called Trellix . grad a: Now , I guess if I were if I were doing more powerful excuse me more complex web sites I might want to . postdoc e: Well , would this be to document it also for outside people or mainly for in house use ? grad a: But . phd b: I I I I grad a: OK , well , send me links and I wi send me pointers , rather , and I 'll put it together . I don't think we should say , " oh , it 's internal therefore we don't have to make it very good " . phd b: therefore we can put data in it that we don't we don't have to worry about releasing " . postdoc e: Yeah , it is true , that is it benefits to undergrad d: Cuz you 're gonna have to wor do the work sooner or later . undergrad d: d we we out of tape out of disk ? phd b: No , we 're doing we 're doing great . undergrad d: I I was looking for the actual box I plan to use , but I c all I could I couldn't find it at the local store . And it 's two by two , by one , and it would fit right under the right under th the the the the lip , grad a: Yeah , does everyone know about the lip on the table ? It 's great . undergrad d: And , it oc I p especially brought the bottom along to try and generate some frequencies that you may not already have recorded . grad a: Clink ! Clink ! undergrad d: Let 's see see what it does to the But this was the  just just to review , and I also brought this along rather than the projector so we can put these on the table , and sort of w push them around . grad a: And and crinkle them and phd b: What ? postdoc e: And th " that " being a diagram . phd b: What ? undergrad d: That that 's the six tables that we 're looking at . phd b:  th undergrad d: Well why not ?  , cuz this is what 's gonna happen . And and and you So this is the box 's grad a: Get your paper off my PDA ! phd b: Yeah . And so the the two head set jacks would be in the front and then the little LED to indicate that that box is live . The the important issue about the LED is the fact that we 're talking about eight of these total , which would be sixteen channels . And , even though we have sixteen channels back at the capture , they 're not all gonna be used for this . undergrad d: So there 'd be a subset of them used for obviously j just use the ones at this end for for this many . you 'd like a a way to tell whether your box is live , so the LED wouldn't be on . undergrad d: So if you 're plugged in it doesn't work and the LED is off that 's that 's a tip off . And then the ,  would wire the all of the cables in a in a bundle come through here and o obviously collect these cables at the same time . undergrad d: so this this notion of putting down the P Z Ms and taking them away would somehow have to be turned into leaving them on the table grad a: Right . undergrad d: and even though there 's eight cables they 're not really very big around so my model is to get a a a p piece of grad a: Sleeve . undergrad d: yeah , that that stuff that people put with the little you slip the wires into that 's sort of shaped like that cross section . I 'm I 'm r a I 'm going up and then I 'm going down . grad a: Whoo ! phd b: Speed bump ! postdoc e: Sleeping pol phd b: Speed bump . grad a: Speed A " sleeping policeman " ! undergrad d: Yeah , it 's like a speed bum An postdoc e: Speed bump . undergrad d: And they 're ac they 're actually ext extruded from plastic . phd c: What does that mean ? phd b: That 's the s that 's British for speed bump , phd c: Is it a speed bump ? undergrad d: So that the wires go through here . postdoc e: Ah ! grad a: OK , so that undergrad d: s So it would c basically go on the diagonal here . grad a: So why do we have sixteen channels instead of like some fewer number ? phd b: Yeah . undergrad d: because the phd b: How else are you gonna distribute them around the tables ? undergrad d: Because they 're there . Why two each ? phd b: Oh , because then you don't have to just have one each . So that if t if you have two people sitting next to each other they can actually go into the same box . undergrad d: And to See , thi this is really the way people sit on this table . phd b: Only if you had Well it depends on this box , right ? undergrad d: Oh , true enough . grad a: Oh , I didn't understand undergrad d: This this box Thi - thi thi this box is a one off deal . And , it 's function is to s to , essentially a wire converter to go from these little blue wires to these black wires , plus supply power to the microphones cuz the the he the , cheap head mounteds all require low voltage . grad a: So so you 'd imagine some sort of in some sort of patch panel on top to figure out what the mapping was between each d of these two and each of those one or what ? phd b:  ! undergrad d: Well I w I I the simplest thing I could imagine , i which is really , really simple is to quite literally that these things plug in . And there 's a there 's a plug on the end of each of these these , ei eight cables . undergrad d: An - and there 's only there 's only four slots that are you know , in in the first version or the version we 're planning to to build . undergrad d: So that that was the whole issue with the LED , that you plug it in , the LED comes on , and and and you 're live . undergrad d: Now the the the subtle issue here is that tha I I haven't really figured out a solution for this . What happens if somebody unplugs this because they plug in more of something else ? grad a:   undergrad d: Well the there 's no clever way to let the up stream guys know that you 're really not being powered . So th there will be a certain amount of looking at cables has to be done if people , rewire things . phd b: But  there are actually that you know , there 's an extra there 's a mix out on the radio receiver ? undergrad d:   phd b: So there are actually six XLR outs on the back of the radio receiver and only five cables going in , I had the wrong five , so I ended up not recording one of the channels and recording the mix . undergrad d: Wonder whether it works any phd b: But I subtracted the four that I did have from the mix and got a pretty good approximation of the @ @ . undergrad d: Is it is phd b: It 's not bad , grad a: Wow . grad a: So what 's the schedule on these things ? undergrad d: So phd b: But , you always postdoc e: Wow . undergrad d: well I was wrestling with th with literally the w number of connectors in the cable and the the , powering system . And I I was gonna do this very clever phantom power and I decided a couple days ago not to do it . So I think the other thing I 'd like to do is , do something about the set up phd b: See grad a: so that it 's a little more presentable and organized . Dan - Dan and I have sort of gone around on this , and we were thinking about something that opened up in the top to allow access to the mixer for example . undergrad d: But there 's these things sticking out of the mixer which are kind of a pain , so you end up with this thing that if if you stuck the mixer up here and the top opened , it 'd be it 'd be fine . You wouldn't necessarily Well , you s understand what I 'm grad a: Yeah , I understand . undergrad d: the the you can you can start start s sketching it out , grad a: So . undergrad d: and I can certainly build it out of oak no problem , would it you know , arb you know , arbitrarily amount of grad a: I need a desk at home too , alright ? Is that gonna be a better solution than just going out and buy one ? undergrad d: Well , the as we found out with the the thing that , Jeff bought a long time ago to hold our stereo system the stuff you buy is total crap . Works fine for holding a Kleenex , grad a: Right , Kleenex and telephones . grad a: so yeah , I g I guess it 's just a question , is that something you wanna spend your time on ? undergrad d: Oh , I I 'm paid for . phd b: Oh , yeah ! grad a: I 'm telling you , I 'm just gonna cart one of them away if they stay there much longer . undergrad d: let the record show at  at f four thirty - five Adam Janin says postdoc e: Wow . grad a: So , j  , then the other question is do we wanna try to do a user interface that 's available out here ? phd b: Sorry ? undergrad d: Slipped almost slipped it by Dan . grad a: And how do we want to do that ? postdoc e: You mean like see see meter readings , from while sitting here . undergrad d: What 's wrong with yours ? If we bought you a a phd b: Oh , a Applecard . phd c: Well undergrad d: What ? grad a: I have an IRAM machine I 've borrowed and we can use it . phd b: I or the undergrad d: N no , I 'm I 'm I 'm serious . Does does the wireless thing work on your grad a: Wait , isn't that an ethernet connection or is that a phone ? phd b: that 's an ethernet connection . grad a: We jus undergrad d: I 'm serious , that that it it phd b: Yeah . T to have it  , just undergrad d: It 's very convenient especially if Dan happens to be sitting at that end of the table to not have to run down here and and look in the thing every so often , phd b: Yeah . And given given that we 've got a wireless that we 've got a we got the field . undergrad d: Y  , we need obviously need to clear this with ACIRI but , how tough can that be ? There it you 'd all you need 's web access , isn't it ? phd b: W we don't need X access undergrad d: In in theory . That 's that 's what it does , undergrad d: OK , phd b: yeah . grad a: phd b: So grad a: right , so it 's just a question of getting a laptop and a wireless modem . phd b: With a with a with a w undergrad d: No , and he he had , reque @ @ my my proposal is you have a laptop . undergrad d: If if we bought you the thing would you mind using it with i the the phd b: No , I would love to but I 'm not sure if my laptop is compatible with the wave LAN thing they 're using . phd b: Well Apple has their own thing , right ? phd c: He 's undergrad d: Your new one ? grad a: Airport . phd b: Yeah , I think what I think you I think it just plug plugs in a PC card , so you could probably make it run with that , but . grad a: The question is , is there an Apple driver ? undergrad d: I e phd b: Yeah , I 'm sure . But  anyway there are there are abs there are a bunch of machines at ICSI that have those cards undergrad d: But the two t phd b: and so I think if w if it doesn't we should be able to find a machine that does that . I  I know that doesn't don't don't the important people have those little blue VAIOs that undergrad d: Well , b that to me that 's a whole nother . undergrad d: The the idea of con convincing them that we should use their network i is fairly straight forward . undergrad d: The idea of being able to walk into their office and say , " oh , can I borrow your machine for a while " , is is is a non - starter . So , either either we figure out how to use a machine somebody already in the group already owns , a a and the idea is that if it 's it perk , you know , it 's an advantage not not a disadvan or else we we literally buy a machine e exactly for that purpose . undergrad d: Certainly it solves a lot of the problems with leaving a monitor out here all the time . undergrad d: I I I I 'm I 'm not a big fan of doing things to the room that make the room less attractive for other people , grad a: Right . undergrad d: right ? Which is part of the reason for getting all this stuff out of the way grad a: Yeah . undergrad d: and and , so a monitor sitting here all the time you know people are gonna walk up to it and go , " how come I can't get , you know , Pong on this " or , whatev grad a:   I 've I 've borrowed the IRAM VAIO Sony thingy , phd c: Well grad a: and I don't think they 're ever gonna want it back . undergrad d: Well , the certainly , u you should give it a shot first See whether you you can get compatible stuff . Who knows , they might have an extra hardware s phd b: I 'd trade them a flat panel display for it . phd c: What is the , projector supposed to be hooked up to ? undergrad d: Good . There 's gonna be actually a a plug at the front that 'll connect to people 's laptops so you can walk in and plug it in . undergrad d: So people sitting here are going " testing , one , two , three " ! phd c: But  , that 's another that 's another possibility that , you know , solves undergrad d: It a phd b: Yeah . phd b: That 's an end undergrad d: But but but I think the idea of having a control panel it 's that 's there in front of you is really cool . phd b: I think and  , having having it on wireless is is the neatest way neatest way to do it . undergrad d: R grad a: I had undergrad d: As long as you d as l as long as you 're not tempted to sit there and f keep fiddling with the volume controls going , " can you talk a bit louder ? " phd b: Yeah . grad a: I had actually earlier asked if I could borrow one of the cards to do wireless stuff phd b: Yeah . undergrad d: And and it 's a a PCMCIA card , right ? grad a: Yep . undergrad d: right ? In your new machine ? phd c: It 's it really come down to the driver . phd c:  grad a: Right , and if and if his doesn't work , as I said , we can use the PC . undergrad d: Right , i it 'll it 'll work It 'll work the first time . phd b: grad a: So phd b: well , that sounds like a d good solution one way or the other . grad a: So Jim is gonna be doing wiring and you 're gonna give some thought to cabinets ? undergrad d: y yeah .  phd b: We 'd I think undergrad d: Hey , what are those green lights doing ? grad a: They 're flashing ! phd b:  - oh !  - oh ! Does that it means it means it 's gonna explode . undergrad d: Cut the red wire , the red wire ! phd b:  grad a: When people talk , it they go on and off . phd b: using using these undergrad d: And it and the f the five thousand for the wires , so if I 'm gonna do No . phd b: that 's true undergrad d: I have to do phd b: but we haven't spent that , right ? But once we once we 've done the intellectual part of these , we can just knock them out , right ? grad a: Cheap . phd b: And then we could Washington could have a system that didn't have any wireless but would had what 's based on these undergrad d:   phd b: And that 's about cuz you wouldn't even need the mixer if you didn't have the Oh th the P Z undergrad d: Right . But anyway you 'd save , on the seven seven or eight thousand for the for the wireless system . undergrad d: What ? postdoc e: It 's nice it 's nice to be thinking toward that . grad a: There 's a speech compression program that works great on things like this , cuz if the dynamic range is low it encodes it with fewer bits . phd c: Now , shorter words wouldn't would induce more dynamics , right ? You want to have phd b: Yeah , but if the words are more predictable . postdoc e: That 's a long word ! grad a: How do you spell that ? postdoc e: I don't know . grad a: OK , can you do one more round of digits ? Are we done talking ? undergrad d: Well it 's a choice if we get a choice , let 's keep talking . grad a: Are you done ? postdoc e: I 'm done , grad a: I 'm done . Like , you know , candy bars or something , grad a: I 'll I 'll remember to bring M and M 's next time . undergrad d: or or or or or a little ,  you know , toothbrushes like they give you at the d dentist . postdoc e: We don't know that for sure , do we ? grad a: I should have mentioned that s  , to pause between lines but undergrad d: No , I know . grad a: It 's it 's only a hard time for the transcriber not for the speech recognizer . phd b: Tha - tha postdoc e: But I also think you said channel four grad a: Me . phd b: But they 're in order ! undergrad d: But real phd b: They start , six , seven , eight , nine , zero , one , two , three , four , five , six , eight , nine . undergrad d: Where 'd they come from ? phd b: And they 're in order because they 're sorted lexically by the file names , which are have the numbers in digits . phd b: And so they 're actually this is like all the all utterances that were generated by speaker MPJ or something . phd b: It doesn't matter ! It 's like Cuz you said " six , seven , eight " . If I if not I undergrad d: But the real question I have is that , why bother with these ? postdoc e: Oh , interesting . undergrad d: Why don't you just ask people to repeat numbers they already know ? Like phone numbers , you know , social security numbers . phd b: Cuz we have these writt written down , right ? grad a: Because Right . phd b: That 's why grad a: If we have it ,  undergrad d: I know . undergrad d: Credit card numbers , grad a: We don't have to tran undergrad d: yeah . undergrad d: Yeah , so you just say say your credit card numbers , say your phone numbers , say your mother 's maiden name . undergrad d: This grad a: Actually , this I got this directly from another training set , from Aurora . postdoc e: I was I the reason I made my mistake was grad a: What ? postdoc e: Wa - was this ? phd b: There were no there were no direct driver errors , by the look of it , which is good