So , before we get started with the , technical part , I just want to review what I think is happening with the our data collection . Th - this this is s should be im it isn't There 's another thing going on of gathering data , and that 's pretty much independent of this . What we think is gonna happen is that , in parallel starting about now we 're gonna get Fey to , where you 're working with me and Robert , draft a note that we 're gonna send out to various CogSci c and other classes saying , " here 's an opportunity to be a subject . " And then there 'll be a certain number of  , hours during the week which she will be available and we 'll bring in people .  , roughly how many , Robert ? We d Do we know ? grad c: fifty was our sort of our first professor b: OK . So , we 're looking for a total of fifty people , not necessarily by any means all students but we 'll s we 'll start with with that . And , so , I guess there 's a plan to have a meeting Friday afternoon  , with  , Jane , and maybe Liz and whoever , on actually getting the script worked out . But what I 'd like to do , if it 's O K , is to s to , as I say , start the recruiting in parallel and possibly start running subjects next week . The week after that 's Spring Break , and maybe we 'll look for them some subjects next door grad c: Yeah . Also , F both Fey and I will , do something of which I may , eh kindly ask you to to do the same thing , which is we gonna check out our social infrastructures for possible subjects . So , if you happen to be sort of in a non - student social setting , and you know people who may be interested in being subjects We also considered using the Berkeley High School and their teachers , maybe , and get them interested in stuff . But I I will just make a first draft of the , note , the " write - up " note , send it to you and Fey and then professor b: And why don't you also copy Jane on it ? grad c: And , Are we Have we concurred that , these these forms are sufficient for us , and necessary ? professor b: th I think they 're necessary . professor b: and I think we 're just gonna use it as it is , and  grad c: N . There 's one tricky part about , they have the right  I The last paragraph " if you agree to participate you have the opportunity to have anything excised which you would prefer not to have included in the data set . " OK ? Now that , we had to be included for this other one which might have , meetings , you know , about something .  , so what I 'd like to do is also have our subjects sign a waiver saying " I don't want to see the final transcript " . professor b: And if they don't If they say " no , I 'm not willing to sign that " , then we 'll show them the final transcript . professor b: That ,  yeah , so we might actually ,  S i Jane may say that , " you know , you can't do this " , " on the same form , we need a separate form . I 'd I 'd I 'd like to , e e  , add an a little thi eh a thing for them to initial , saying " nah , do I don't want to see the final transcript . professor b: But other than that , that 's one 's been approved , this really is the same project , rec you know . grad c: And ,  and now it 's We have a complete English parser that does everything the German parser does . grad e: What did you end up having to do ?  , wha Was there anything interesting about it at all ? grad c: Well , if you , eh grad d: We 'll show you . professor b: Yeah , we can show us , grad e: or are we gonna see that ? professor b: right ? grad c: Well , w w We d The first we did is we we tried to to do change the the " laufen " into " run " , or " running " , or " runs " . grad c: So this sentence sentence is parses the p the same output , grad e:  . grad c: If if you 'd add add Today and Evening , it 'll add Time or not . grad e: And the t and the time , right ? grad c: So it i it does look at that . grad c: But all the rest is p simply frosting on the cake , and it 's optional for that parser . professor b: So , you can sho You you Are are you gonna show us the little templates ? grad c: And grad e: S grad c: Yeah . I I also have it running here , grad e: The former end g " Oh , I see . grad c: so if I do this now , you can see that it parsed the wonderful English sentence , " Which films are on the cinema today evening ? " But ,  . professor b: No i grad c: It could be " this evening , which which films are on the cinema " , or " running in the cinema , which "  , " today evening " ,  i " Is anything happening in the cinema this evening ? " grad e: OK . professor b: Well grad c: Ge - elaborate , or , more or less ,  professor b: Actually , it 's a little tricky , in that there 's some allowable German orders which aren't allowable English orders and so forth . So it it doe I it These u these optional elements , grad c: It is not professor b: it 's it 's actually a set , not a sequence ? grad c: Yeah . We were I was afraid that ,  professor b: Oh ! grad e: So it really is key word matching , basically . What 's an idiomatic of phrasing this ? Which films are showing ? grad d: Are pl playing at the cinema ? grad c: playing ? grad d: Yeah . grad e: Tonight ? grad d: I changed that file , actually , where it 's on my account . grad e: This this evening ? phd f: Actually , you would say , " which films are on tonight ? " grad d: You want to get it ? Or is di was it easy to get it ? grad c:  . grad c: Except that we d w we don't have this , time information here now , which is ,  Oh . And ,  you have already to make it a little bit more elaborate , right ? grad d: Yeah ,  I changed those sentences to make it , more , idiomatic . And , of course , you can have i many variations in those sentences , they will still parse fine . So , if you want to look at the templates , they 're conveniently located in a file , " template " . I had to change , @ @ " Spielfilm " to " film " , " Film " to " movie " , cinem " Kino " to " cinema " to " today " heu " heute " to " today " , grad e: Huh . grad c: evening " Abend " to " evening " professor b: Capitalized as well grad a:  . professor b: Y i grad d: One thing I was wondering , was , those functions there , are those things that modify the M - three - L basically ? grad c: Yep . grad c: And that 's that 's the next step , professor b: p grad c: but we 'll get to that in a second . professor b: So so , the point is , if it says " this " and " see " , it also will work in " see " and " this " ? grad e: S professor b: In the other order ? grad c: Yeah . professor b: with those two key words ? grad c: Should we try it ? professor b: " This is the one I want to see " or whatever . grad c: It 's correct , but I don't know where it gets it from . grad d:  it 's sort of grad c: And " see this " is exactly the same thing . grad d: One thing I was wondering was , those percentage signs , right ? So , why do we even have them ? grad c: Yep . grad d: Because if you didn't have them grad c: I 'll tell you why . grad c: And the value of the score is , v I assume , I guess , the more of these optional things that are actually in there , the higher the r score it is .  , let 's hope that the generation will not be more difficult , even though the generator is a little bit more complex .  but we 'll Mmm , that means we may need two hours and twenty minutes rather than an hour ten minutes , professor b: Alright . grad c: And the next thing I would like to be able to do , and it seems like this would not be too difficult either , is to say , " OK let 's now pretend we actually wanted to not only change the the mapping of of , words to the M - three - L but we also wanted to change add a new sentence type and and make up some some new M - three - L s " professor b: Yep . And , grad d: So , that 's grad c: that 's shouldn't be too tough . Yeah , so where are those those functions " Action " , " Goodbye " , and so on , right ? Are they actually , Are they going to be called ?  , are they present in the code for the parser ? grad c: Yeah . So what it probably does , is it takes the ,  Is this where it is ? This is already the XML stuff ? This is where it takes its own , syntax , and converts it somehow . Where is the  grad d: What are you looking for ? grad c: where it actually produces the the XML out of the , parsed stuff . You mean , where the where the act how the action " Goodbye " maps into something grad d: Yeah . So when it encounts encounters " Day " , it will , activate those h classes in the in the XML stuff But ,  I saw those actions  , the " Goodbye " stuff somewhere . So whatever n this does  this is , basically , looks l to me like a function call , right ? professor b:  ? Oh , yeah . grad c: And ,  So , whenever it it encounters " Goodbye " , which we can make it do in a second , here grad a: That function automatically generates an initialized XML structure ? grad c: I grad d: I think each of those functions act on the current XML structure , and change it in some way , for example , by adding a a l a field to it , or something . professor b: cause some of them there were other actions  , that that s seemed to step state variables somewhere , grad d: Right . professor b: and confirm that it 's grad c: W we  -  grad d: Oh , you mean that 's not going to actually modify the tree , professor b: I think that 's right . grad c: e mmm  , well i There is a feature called " Discourse - Status " , grad d: When there 's a feature . grad c: And so whenever I just say , " Write " , it will it will put this in here . professor b: Oh , so it always just Is it So it Well , go back , then , cuz it may be that all those th things , while they look like function calls , are just a way of adding exactly that to the XML . grad c: So , this professor b: e I 'm not sure e that grad c:  well , we we 'll see , when we say , let 's test something , " Goodbye " , causes it to c to create basically an " Action Goodbye - End - Action " . grad c: Now , if we know that " Write " produces a " Feature Discourse - Status Confirm Discourse - Status " . But there is some kind of function call , because how does it know to put Goodbye in Content , but , Confirm in Features ? grad c: Oh . It d it n That 's because grad d: So So , it 's not just that it 's adding that field . professor b: It 's it 's the It 's under what sub - type you 're doing it . grad c: Well , sometimes it m Sometimes , i grad d: Well , they 're defined somewhere , presumably . grad c: it grad a: Well , it just automatically initializes things that are common , right ? professor b:  grad a: So it 's just a shorthand . And there is , here , a reference So , this tells us that whatever is has the ID " zero " is referenced here by @ @ the restriction seed and this is exa " I want " What was the sentence ? professor b: " I want two seats here . Here , we change and so , we Here we add something to the Discourse - Status , that the user wants to change something that was sort of done before And ,  and that , whatever is being changed has something to do with the cinema . grad a: So then , whatever takes this M - three - L is what actually changes the state , not the Yeah , OK . I definitely think it 's It 's worth the exercise of trying to actually add something that isn't there . grad c:  ? professor b:  Disc grad c: Sort of get a complete understanding of the whole thing . Then the next thing we talked about is actually , figuring out how to add our own tags , and stuff like that . So , this is  , what I got today is the the new  M - three - L for  , the Maps , professor b: Yep . grad c: and with some examples So , this is the XML and this is sort of what it will look like later on , even though it you can't see it on on this resolution . And this is what it sort of is the the structure of Map requests ,  also not very interesting , and here is the more interesting stuff for us , is the routes , route elements , and , again , as we thought it 's really simple . We have @ @ simple " from objects " and " to objects " and so forth , points of interest along the way And , I asked them whether or not we could ,  First of all , I was little bit It seemed to me that this m way of doing it is sort of a stack a step backwards from the way we 've done it before . professor b: S grad c: So these are these are professor b: So these are these are your friends back at EML . There 's there 's not seven committees , or anything , right ? grad c: No . And every every element of that e r r f of that Every segment we call a " route element " . And so , from A to B we cut up in three different steps , and every step has a " from object " where you start , a " to object " where y where you sort of end , and some points of interest along the way . What w I was sort of missing here , and  , maybe it was just me being too stupid , is , I didn't sort of get the the notion of the global goal of the whole route . And I suggested that they should n be k  , kind enough to do s two things for us , is one , Also allocating , some tags for our Action Schema Enter - Vista - Approach , and And also , since you had suggested that that , we figure out if we ever , for a demo reason , wanted to shortcut directly to the g GIS and the Planner , of how we can do it . Now , what 's the state of the art of getting to entrances , what 's the syntax for that , how get getting to vista points and calculating those on the spot . Wherever you 'll find a route planner it n does nothing but get to the closest point where the street network is at minimal distance to the geometric center . Let ,  I want a a Again , outside of m almost managerial point ,  You 're in the midst of this , so you know better . But it seems to me it 's probably a good idea to li  minimize the number of  , change requests we make of them . Does this does this make sense to you guys ? It  you 're you 're doing the the interaction but it seemed to me that what we ought to do is come up with a  , something where you ,  And I I don't know who 's mok working most closely on it .  , take what they have , send it to everybody saying " this is what they have , this is what we think we should add " , OK ? and then have a d a an iteration within our group saying "  , well " OK ? And get our best idea of what we should add . Is i or , I don't know does this make sense to you ? Or grad c: Yeah . Especially if we want Sort of , what I my feeling was eh we we sort of reserved something that has a r eh an OK label . grad c: I w No matter how we want to call it , this is sort of our playground . grad c: And if we get something in there that is a structure elaborate and and and and and complex enough to to to maybe enable a whole simulation , one of these days , that would be u the the perfect goal . What are the thl class of things we think we might try to do in a year or two ? How how would we try to characterize those and what do we want to request now that 's leave enough space to do all that stuff ? grad c:   professor b: And so that sounds like a great thing to do as the priority item  , as soon as we can do it . professor b: So y so you guys will send to the rest of us  a version of  , this , and the  , description grad a: With sugge yeah , suggested improvements and professor b: Well b Yeah . So , the the  Not everyone  , reads German , so if you 'd  grad a: Mmm . Then , with some sug s suggestions about where where do we go from here ? grad a: OK . But for the moment just  , t for this class of of things , we might want to try to encompass . This is this is everything that that , you know ,  we might want to do in the next couple years . So I just this XML stuff here just has to do with Source - Path - Goal type stuff , in terms of traveling through Heidelberg . grad a: So , but this O Is the domain greater than that ? professor b: No . It 's beyond Source - Path - Goal , but I think we don't need to get beyond it @ @ tourists in Heidelberg . professor b: It seems to me we can get all the complexity we want in actions and in language without going outside of tourists in Heidelberg . OK ? But you know , i depending on what people are interested in , one could have , tours , one could have  , explanations of why something is is , you know , why why was this done , or  , no there 's no end to the complexity you can build into the  , what a tourist in Heidelberg might ask . professor b: So , at least unless somebody else wants t to suggest otherwise I think the general domain we don't have t to  , broaden . And if there 's something somebody comes up with that can't be done that way , then , sure . W we 'll we 'll look at that , but  I 'd be s I I 'd be surprised at if there 's any important issue that that And ,   if if you want to  , push us into reference problems , that would be great . professor b: and you know , what what are these things referring to ? Not only anaphora , but , more generally the ,  this whole issue of , referring expressions , and , what is it that they 're actually dealing with in the world ? grad c:   professor b: And , again , this is li in the databa this is also pretty well formed because there is an ontology , and the database , and stuff . professor b: There aren't really deep mysteries about  , what w what things the system knows about . professor b: And a l and a lot and and anaphora , and pronouns , grad c: Nuh . grad c: Now , we hav the the whole Unfortunately , the whole database is , in German . We have just commissioned someone to translate some bits of it , IE the e the shortest k the the more general descriptions of all the objects and , persons and events . But did y I  I think there will be great because the reference problem really is not trivial , even if you have such a g well - defined world . grad a: Could you give me an example of a reference problem ? so so l I can make it more concrete ? grad c: Well How do I get to the Powder - Tower ? We sort of t think that our bit in this problem is interesting , but , just to get from Powder - Tower to an object I ID in a database is also not really trivial . phd f: Or or if you take something even more scary , " how do I get to the third building after the Tower ? the Ple - Powder - Tower ? " grad a: Mmm . Or , you know , the church across from City Hall , or grad a: Or the re the restaurant where they wear lederhosen ? grad c: Or the phd f: Right . grad c: O or or tower , or this tower , or that building , or phd f: Right . Trying to professor b: Or you can say " how " you know , " how do I get back ? " grad a: Yeah , yeah . And , again , it 's just a question of which of these things , people want to dive into . What , I think I 'm gonna try to do , and I guess , pwww ! let 's say that by the end of spring break , I 'll try to come up with some general story about , construction grammar , and what constructions we 'd use and how all this might fit together . grad c: u u u u That 's what strikes me , that we sort of the de g  , small Something , maybe we should address one of these days , is to That most of the work people actually always do is look at some statements , and and analyze those . grad c: But the whole i is it is it really relevant that we are dealing mostly with , sort of , questions ? professor b: Oh , yeah ? grad c: you know professor b: Well ,  yeah , I d grad c: And this is It seems to me that we should maybe at least spend a session or or brainstorm a little bit about whether that l this is special case in that sense . You know Did we ever find m metaphorical use in in questions in in that sense , really ? professor b: Yeah . grad c: " Who got kicked out of France ? " phd f: Muh professor b: Yeah , or , you know . " Wh - why is he why is he pushing for promotion ? " grad c: Nuh . professor b: er , just pick pick any of them and just do the eh grad c:   professor b: So I don't I don't think , it 's at all difficult  , to convert them to question forms that really exist and people say all the time ,  And sort of we don't know how to handle them , too . Right ?  , it 's I d It We don't know how to handle the declarative forms , @ @ really , and , then , the interrogative forms , ah - oh . grad d: Ooo ! professor b: Nancy , it looked like you were s grad e: Oh . it 's just that that the goals are g very different to cases So we had this problem last year when we first thought about this domain , actually , was that most of the things we talked about are our story understanding . grad e: we 're gonna have a short discourse and the person talking is trying to , I don't know , give you a statement and tell you something . And here , it 's th grad c: Help you create a mental model , blah - blah - blah . grad e: And then here , y you are j  , the person is getting information and they or may not be following some larger plan , you know , that we have to recognize or , you know , infer . And th th the their discourse patterns probably don't follo follow quite as many logical connec professor b: Right . No , I think that 's one of things that 's interesting , is is in this sort of over - arching story we we worked it out for th as you say , this the storytelling scenario . Cuz for a while we were thinking , " well , how can we change the , data to sort of illicit tha illicit , actions that are more like what we are used to ? " But obviously we would rather , you know , try to figure out what 's what 's , you know professor b: Well , I don't know .  , maybe maybe that 's what we 'll do is is s u e We can do anything we want with it . professor b: OK , and the one for next  , summer is just half done and then the other half is this , " generation thing " which we think isn't much different . professor b: So once that 's done , then all the rest of it is , sort of , you know , what we want to do for the research . And we can w we can do all sorts of things that don't fit into their framework at all . professor b: If we can use all the , execution engines , then we can , you know , really try things that would be too too much pain to do ourselves . So , if we want to turn it into u understan standing stories about Heidelberg , we can do that .  , that would just be a t a  grad c: Or , as a matter of fact , we need and if we if we ' r eh take a ten year perspective , we need to do that , because w e w a Assuming we have this , we we ta in that case we actually do have these wonderful stories , and historical anecdotes , professor b: Yeah . So , th the database is huge , and if we want to answer a question on that , we actually have to go one step before that , and understand that . grad c: And so , this has been a a a Deep Map research issue that was is is part of the unresolved , and to - do 's , and something for the future , is how can we sort of run our our text , our content , through a machine that will enable us , later , to retrieve or answer e questions more sensibly ? phd f: Mwa   S Who 's going ? phd f: So ,  So , I was just going to ask , so , what is the the basic thing that that you are , obligated to do , by the summer before w  y c we can move professor b: Ah ! OK . So , what happened is , there 's this , eh ,  Robert was describing the There 's two packages there 's a , quote parser , there 's a particular piece of this big system , which , in German , takes these t sentence templates and produces XML structures . professor b: The other thing is , at the other end , roughly at the same level , there 's something that takes , X M L structures , produces an output XML structure which is instructions for the generator . professor b: OK ? And then there 's a language generator , and then after that a s a synthesizer that goes from an XML structure to , language generation , to actual specifications for a synthesizer . Eh , but again , there 's one module in which there 's one piece that we have to convert to English . And that But as I say , this is all along was viewed as a kind of a m a minor thing , necessary , but but not phd f: Right . professor b: And much more interesting is the fact that , as part of doing this , we we are , you know , inheriting this system that does all sort of these other things . professor b: Not precisely what we want , and that 's that 's wh where it it gets difficult . So , e enough of that , but I , mmm , the e sort of , Johno and I will take up that responsibility , and , get a first draft of that . grad c: y you guys sort of started fighting , on the Bayes - net " Noisy - OR " front ? grad d:  . Yeah , I thought I should , talk a little bit about that , because that might be a good , sort of architecture to have , in general for , problems with , you know , multiple inputs to a node . And what 's the other one ? so that just we know what the d agenda is ? grad c: the Wu paper , I think maybe professor b: Oh , yeah .  , so I I 've been in contact with Wu , so , probably let 's put that off till I I till I understand better , what he 's doing . It 's just a little embarrassing cause all this was in his thesis and I was on his thesis committee , and , so , I r really knew this at one time . professor b: But , I I It 's not only  Is Part of what I haven't figured out yet is is how all this goes together . professor b: squealing sound ? grad d: Or shall I just use this ? professor b: It 's probably just as easy . grad d: Hey ! grad a: I was envying you and your pocket cause I don't have one . grad e: It was a quick one , huh ? professor b: That 's why they invented " pocket T 's " . grad d: So ,  Recall that , we want to have this kind of structure in our Bayes - nets . Namely , that ,  You have these nodes that have several bands , right ? So Does  , they sort of the typical example is that , these are all a bunch of cues for something , and this is a certain effect that we 'd like to conclude . So ,  Like , let 's just look at the case when , this is actually the the final action , right ? So this is like , you know , touch , grad c: Y grad d: or grad c: E - EVA grad d: Sorry .  grad c:  grad d: Yeah , E - EVA , right ? grad c: Yeah . grad d: Enter , V View , Approach , right ? phd f: W what was this ? It i i i ehhh , i ehhh . professor b: Wri - write it out for for grad d: So , this is Yeah . So , we 'd like to take all these various cues , right ? phd f: Like the army . grad d: So this one might be , say ,  grad e: New terminology ? phd f: Yeah . grad c:  ? grad d: Well , let me pick a random one grad e: I haven't heard that before . grad d: and say ,  I don't know , it could be , like This isn't the way it really is , but let me say that , suppose someone mentioned , admission fees Ah , it takes too long . If the thing is a landmark , you know ,  then there 's another thing that says if  if it 's closed or not , at the moment . Right ? And the , problem that we were having was that , you know , given N - nodes , there 's " two to the N " Given N - nodes , and furthermore , the fact that there 's three things here , we need to specify " three times " , " two to the N " probabilities . For example , they could be " time of day " , in which case we could , say , you know , " Morning , afternoon , evening , night " . And , that 's a lot of probabilities to put here , which is kind of a pain .  Where should I put this ? So , the idea is that , Let 's call these , C - one , C - two , C - three , and C - four , and E , for Cause and Effect , I guess . Well , actually , the idea , first of all , is that each of these things has a quote - unquote distinguished state , which means that this is the state in which we don't really know anything about it . So right ? So , for example , if we don't really know if the thing is a landmark or not , Or , i if that just doesn't seem relevant , then that would be th sort of the Disting - the Distinguish state . It 's a really , you know , if there is something for the person talking about the admission fee , you know , if they didn't talk about it , that would be the Distinguish state . grad c: S so , this is a fanciful way of saying " default " ? grad d: So Yeah , yeah . grad d: So , the idea is that , you have these intermediate nodes , right ? E - one , E - two , E - three and E - four ? professor b: So , this is the Heckerman paper you 're working with ? Good . So The idea is that , each of these EI is represents what this would be if all the other ones were in the distinguish state . Right ? So , for example , suppose that the person  , suppose the thing that they talked about is a landmark . Then , this would be W The this would just represent the probability distribution of this , assuming that this cue is turned on and the other ones just didn't apply ? So , you know , if it is a landmark , and no none of the other things really ap applicable , then this would represent the probability distribution . So maybe in this case Maybe we just t k Maybe we decide that , if the thing 's a landmark and we don't know anything else , then we 're gonna conclude that ,  They want to view it with probability , you know , point four . They want to enter it with probability ,  with probability point five and they want to approach it probability point one , say Right ? So we come up with these l little tables for each of those OK . And the final thing is that ,  this is a deterministic function of these , so we don't need to specify any probabilities . We just have to , say what function this is , right ? So we can let this be ,  G of E - one comma E - two . The important point is that  There is a a a general kind of idea of shortcutting the full CPT . And So I gave eh Bhaskara a copy of this , eh sort of " ninety - two " paper . professor b: So , I u w  , yo  you Have you read it yet ? grad d: you can Yeah , you should take a look at it , I guess . professor b: And  One of the questions just come at Bhaskara is , " How much of this does JavaBayes support ? " grad d: Yeah , it 's a good question . grad d: And ,  In a sense it sup we can make it supported by , manually , entering , you know , probabilities that are one and zeros , right ? professor b: Right . So the little handout that The little thing that I sent I sent a message saying , here is a way to take One thing you could do , which is kind of s in a way , stupid , is take this deterministic function , and use it to build the CPT . So , if Ba - JavaBayes won't do it for you , grad c: Mmm .  and , what I sent out about a week ago , was an idea of how to do that , for , evidence combination . So one of one function that you could use as your " G function " is an e e Evidence - Combining . So you just take the  , if each of th if each of the ones has its own little table like that , then you could take the , strength of each of those , times its little table , and you 'd add up the total evidence for " V " , " E " , and " A " . professor b: I i i No , no But I 'm saying is There There is a w  , if y if if you decide what 's what is appropriate , is probablistic evidence combination , you can write a function that does it . It 's a pui it 's actually one of the examples he 's got in there . But , anyway , s skipping skipping the question of exactly which functions now is it clear that you might like to be able to shortcut the whole conditional probability table . grad c: in some it seems very plausible in some sense , where we will be likely to not be observe some of the stuff . That 's one of the problems , is , W Is is , Where would th Where would it all come from ? grad c: Yeah . grad d: What ? grad c: I if it 's a a a discar Discourse Initial Phrase , we will have nothing in the discourse history . A are you saying that we 'll not be able to observe certain nodes ? That 's fine . The f the the the Bayes - nets in general are quite good at saying , " if you have no current information about this variable just take the prior for that . So , if you don't have any information about the discourse , you just use your priors of of whatever eh the discourse  , eh , basically whatever w it 's Probabilistically , whatever it would be . But the other problem is , how do you fill in all these numbers ? And I think that 's the one he was getting at . So , specifically in this case you have to f have this many numbers , grad e: Yeah . grad d: whereas in this case you just have to have three for this , three for this , three for this . grad d: So you have to have just three N ? So , this is much smaller than that . professor b:  grad e: So , you don't need da data enough to cover  , nearly as much stuff . grad a: So , really , i What a A Noisy - OR seems to kind of " neural - net - acize " these Bayes - nets ? professor b: Eh well to some No , no . So , " Noisy - OR " is a funny way of referring to this , because the Noisy - OR is only one instance . So Eh  grad a: Well , my point was more that we just eh With the neural net , right , eh , things come in , you have a function that combines them and professor b: Yeah , it it Tha - that 's true . It is a is also more neural - net - like , although  , it isn't necessarily sum  , s you know , sum of weights or anything like that . professor b:  i You could have , like the Noisy - OR function , really is one that 's essentially says , take the max . So And , I thi I think that 's the standard way people get around the  There are a couple other ones . But ,  The I think we definitely I think it 's a great idea tha to to pursue that . It  you you can always  see easily that that I 'm not grasping everything correctly , but what seemed attractive to me in im  in the last discussion we had , was that we find out a means of of getting these point four , point five , point one , of C - four , not because , you know , A is a Landmark or not , but we we we label this whatever object type , and if it 's a garden , it 's point three , point four , point two . If it 's , a town hall , it 's point two , point three , point five . And we don't want to write this down necessarily every time for something but ,  let 's see . grad d: It 'll be students Where else would it be stored ? That 's the question . So , i is Well , let me say something , guys , cuz there 's not There 's a pretty point about this we might as well get in right now . professor b: So that  , if you know about it let 's say , a particular town hall that , it 's one that is a monument , then , that would be stored there . If you don't , you look up the hierarchy , Eh so , you you you may or So , then you 'd have this little vector of , you know , Approach Mode or EVA Mode . professor b: or , link to or but but in any case i View it logically as being in the ontology . professor b: And , if yo As I say , if you know about a specific object , you put it there . So , when we get to Wu , The - e We 'll see w what he says about that . professor b: And , then if you If it isn't there , it 's higher , and if you don't know anything except that it 's a b it 's it 's a building , then up at the highest thing , you have the pr what amounts to a prior . If you don't know anything else about a building , you just take whatever your crude approximation is up at that level , grad d: Right . professor b: So , that 's a very pretty relationship between these local vectors and the ontology . And it seems to me the obvious thing to do , unless we find a reason to do something different . professor b: Does this make sense to you ? grad d: So professor b: Bhask - ? grad d: Yeah . So , we are but we we 're not doing the ontology , so we have to get to whoever is doing the u ultimately , professor b: Indeed . So , that 's another thing we 're gonna need to do , is is , to , either grad d: we have to get them to professor b: We 're gonna need some way to either get a p tag in the ontology , or add fields , or some way to associate Or , w It may be that all we can do is , some of our own hash tables that it Th - the th you know , there 's always a way to do that . It 's a just a question of grad a: Yeah , hash on object name to , you know , the probabilities or whatever . And , so , i  grad c: But it 's ,  Well , it strikes me as a What For If we get the mechanism , that will be sort of the wonderful part . And then , how to make it work is is the second part , in the sense that  , m the guy who was doing the ontology eh , eh , s ap apologized that i it will take him another through two to three days because they 're having really trouble getting the upper level straight , and right now . The reason is , given the craw bet  , the the the projects that all carry their own taxonomy and , on all history , they 're really trying to build one top level ontology ft that covers all the EML projects , and that 's , sort of a tough cookie , a little bit tougher than they figured . But , nevertheless , it 's going to be there by n by , next Monday and I will show you what 's what some examples from that for towers , and stuff . And , what I don't think is ever going to be in the ontology , is sort of , you know , the likelihood of , eh , people entering r town halls , and looking at town halls , and approaching town halls , especially since we are b dealing with a case - based , not an instance - based ontology . So , there will be nothing on on that town hall , or on the Berkeley town hall , or on the Heidelberg town hall , it 'll just be information on town halls . professor b: Well , they they they How ar What are they gonna do with instances ? grad c: But what professor b: you y grad c: Well , that 's Hhh .  , th the first , they had to make a design question , " do we take ontologies that have instances ? or just one that does not , that just has the types ? " professor b: OK . grad c: And , so , since the d decision was on types , on a d simply type - based , we now have to hook it up to instances .  this is professor b: But what i What is SmartKom gonna do about that ? grad c: one professor b: Cuz , they have instances all the time . grad c: Yeah , but the ontology is really not a SmartKom thing , in in and of itself . professor b: I understand , but is anybody doing anything about it ? grad c:  professor b: OK . grad c: No , but th the r eh I th I still think that there is enough information in there . So , in a sense , " I " used as Institutions for some s in some sense or the other . When we see what people have done , it may turn out that the easiest thing to do is to build a a separate thing that that just pools i i Like , i i it it may be , that , the the instance w That we have to build our own instance , things , that , with their types , grad d: Yeah , it 's Right , we can just assume professor b: and then it goes off to the ontology once you have its type . So we build a little data structure And so what we would do in that case , is , in our instance gadget have our E V And if we d there isn't one we 'd get the type and then have the E V As for the type . professor b: So , we 'd have our own little things so that whenever we needed one , we 'd just use the ontology to get the type , grad d:   professor b: and then would hash or whatever we do to say , " ah ! grad d:   professor b: If it 's that type of thing , and we want its EVA vector , pppt - pppt ! it 's that . And then But , the combination functions , and whether we can put those in Java Bayes , and all that sort of stuff , is ,  is the bigger deal . grad a: We could just steal the classes in JavaBayes and then interface to them with our own code . grad d:  professor b: Well , I me ye eh , yeah , the grad d: That requires understanding the classes in JavaBayes , I guess .  , you 've been around enough to  Just ? grad a: Well , it depends on professor b: there 's this huge package which which may or may not be consistent and you know . It 's b It It 's an inter sort of a kind of a it The thing is , it 's kind of an interpreter and i i it expects its data structures to be in a given form , and if you say , " hey , we 're gonna make a different kind of data structure to stick in there " grad a: Well , no , but that just means there 's a protocol , right ? That you could professor b: It may or may not . That 's the question is " to what extent does it allow us to put in these G functions ? " And I don't know . grad a: Well , no , but  What I  the So you could have four different Bayes - nets that you 're running , and then run your own write your own function that would take the output of those four , and make your own " G function " , is what I was saying . professor b: Yeah , that 's fine if it 's if it comes only at the end . But suppose you want it embedded ? grad a: Well , then you 'd have to break all of your Bayes - nets into smaller Bayes - nets , with all the professor b: Oh , that Yeah , that 's a truly horrible way to do d it . But , at that point you may say , " hey , Java Bayes isn't the only package in town . Let 's see if there 's another package that 's , eh , more civilized about this . " grad d:  professor b: Now , Srini is worth talking to on this , grad d: Mmm . professor b: cuz he said that he actually did hack some combining functions into But he doesn't remember at least when I talked to him , he didn't remember whether it was an e an easy thing , a natural thing , or whether he had to do some violence to it to make it work . I don't see why the , combining f functions have to be directly hacked into  , they 're used to create tables so we can just make our own little functions that create tables in XML . professor b: Well , I say that 's one way to do it , is is to just convert it int into a into a C P T that you zip It 's blown up , and is a it 's ,  it 's huge , but it doesn't require any data fitting or complication . I don't think  , the fact that it blown u blows up is a huge issue in the sense that  , OK . So say it blows up , right ? So there 's , like , the you know , ten , f ten , fifteen , things . It said , " Here 's the way you 'd take the logical f G function and turn it into a CPT . So , I will , e e before next week , @ @ p push push some more on on this stuff that Dekai Wu did , and try to understand it .  , you 'll make a couple of more copies of the Heckerman paper to give to people ? grad d: p Sure . And I I 'll I 'll think s through this , eh getting EVA vectors dynamically out of ontologies one more time because I s I I I 'm not quite sure whether we all think of the same thing or not , here . professor b: Alright , great ! And , Robert , thank you for coming in under He he 's been sick , Robert . grad a: I was thinking maybe we should just cough into the microphone and see if they can't th see if they can handle it


I think we 've met before , like , I remember talking to you about Aspect or something like that at some point or other .  , apart from that , sort of the old gang , Johno and Bhaskara have been with us from from day one grad g: Yay ! grad e: Hi . grad c: and  they 're engaged in in various activities , some of which you will hear about today . Ami is  our counselor and spiritual guidance and  also interested in problems concerning reference of the more complex type , phd a: Well . grad c: And  hopefully it is by by means of Keith that we will be able to get a b a better formal and a better semantic  idea of what a construction is and  how we can make it work for us . Additionally his interest  surpasses  English because it also entails German , an extra capability of speaking and writing and understanding and reading that language . And  , is there anyone who doesn't know Nancy ? Do you do you know Nancy ? grad g: Me ? grad e: I know Nancy . grad g: What ? grad b: The " I don't know myself " joke . grad e: Man ! grad g: About me or you ? grad b: About me . grad g: It 's a grad c: And  Fey is with us as of six days ago officially ? undergrad d: Officially , grad c: Officially , undergrad d: yeah . grad c: but in reality already  much much longer and   next to some some more or less bureaucratic  stuff with the the data collection she 's also the wizard in the data collection  , grad g: Of Oz . grad c: we 're sticking with the term " wizard " , undergrad d: Yes .  , so we 're about to collect data and  the  s the following things have happened since we last met . grad c: what happened is that  , " A " , there was some confusion between you and Jerry with the that leading to your talking to Catherine Snow , and he was  he he agreed completely that some something confusing happened . It it 's exactly how you interpreted it , sort of s grad e: The list of majors in the department ? undergrad d: M m Majors ? grad c: Ma - majors , majors . undergrad d: Something I don't know about these grad g: The department has many mayors . grad c: Majors and  just sending the the little write - up that we did on to those email lists undergrad d: OK . grad c:  undergrad d: So it was really Carol Snow who was confused , not me and not Jerry . And  we have a little description of asking peop subjects to contact Fey for you know recruiting them for our thing and  there was some confusion as to the consent form , which is basically that that what what you just signed grad g: Right . grad c: and since we have one already  grad g: Did Jerry talk to you about maybe using our class ? the students in the undergrad class that he 's teaching ? grad c:  well he said  we definitely " yes " , grad g: e grad c: however there is always more people in a in a facul  in a department than are just taking his class or anybody else 's class at the moment grad g: Yeah . grad g: OK , but th I guess it 's that  people in his class cover a different set so than the c is the CogSci department that you were talking about ? undergrad d: I guess . See grad g:  reaching out to ? undergrad d: that 's what I suggested to him , that people like like Jerry and George and et cetera just grad g: Cuz we have you know people from other areas grad c: Yeah . undergrad d: Yeah or even I could you know I could do the actual grad c:   grad g: Cuz  I I know how to contact our students , undergrad d: That 's generally the way it 's done . grad g: so if there 's something that you 're sending out you can also s  send me a copy , grad c: Yeah . grad g: me or Bhaskara could either of us could post it to  is it undergrad d: A mailing list . grad g: if it 's a general solicitation that you know is just contact you then we can totally pro post it to the news group grad c:   undergrad d: That 's grad g: OK , so you 'll send it or something so . grad c: As a matter of fact , if you undergrad d: I can send it . grad c: if undergrad d: I 'll send it , grad g: You can send it to me . grad c: How however I suggest that if you if you look at your email carefully you may think you may find that you already have it . grad c: Anyhow ,  the  Yeah , not only Also we will talk about Linguistics and of course Computer Science . grad c:  and then , secondly , we had , you may remember ,  the problem with the re - phrasing , that subject always re - phrase sort of the task that  we gave them , grad b: Right . grad c: and so we had a meeting on Friday talking about how to avoid that , and it proved finally fruitful in the sense that we came up with a new scenario for how to get the the subject m to really have intentions and sort of to act upon those , and  there the idea is now that next actually we we need to hire one more person to actually do that job because it it 's getting more complicated . So if you know anyone interested in in what i 'm about to describe , tell that person to to write a mail to me or Jerry soon , fast .  the idea now is to sort of come up with a high level of sort of abstract tasks " go shopping "  " take in  a batch of art "  " visit do some sightseeing " blah - blah - blah - blah - blah , sort of analogous to what Fey has started in in in compiling compiling here and already she has already gone to the trouble of of anchoring it with specific  o  entities and real world places you will find in Heidelberg . So out of these f s these high level categories the subject can pick a couple , such as if if there is a cop  a category in emptying your roll of film , the person can then decide " OK , I wanna do that at this place " , sort of make up their own itinerary a and and tasks and the person is not allowed to take sort of this h high level category list with them , but  the person is able to take notes on a map that we will give him and the map will be a tourist 's sort of schematic representation with with symbols for the objects . And so , the person can maybe make a mental note that " ah yeah I wanted to go shopping here " and " I wanted to maybe take a picture of that " and " maybe  eat here " and then goes in and solves the task with the system , IE Fey , and  and we 're gonna try out that Any questions ? grad g: so  y you 'll have those say somewhere what their intention was so you still have the the nice thing about having data where you know what the actual intention was ? grad c:   grad g: But they will  There 's nothing that says you know " these are the things you want to do " so they 'll say " well these are the things I want to do " and Right , so they 'll have a little bit more natural interaction ? grad c: Hopefully . grad f: So they 'll be given this map , which means that they won't have to like ask the system for in for like high level information about where things are ? grad c: Yeah it 's a schematic tourist map . So it 'll be  i it 'll still require the that information and An grad g: It w it doesn't have like streets on it that would allow them to figure out their way grad c: N not not not really the street network . grad e: So you 're just saying like what part of town the things are in or whatever ? grad c: Yeah a and  the map is more a means for them to have the buildings and their names and maybe some ma ma major streets and their names grad g:   grad c: and we want to maybe ask them , if you have get it sort of isolated street the the , whatever , " River Street " , and they know that they have decided that , yes , that 's where they want to do this kind of action  that they have it with them and they can actually read them or sort of have the label for the object because it 's too hard to memorize all these st strange German names . And then we 're going to have another we 're gonna have w another trial run IE the first with that new setup tomorrow at two and we have a real interesting subject which is Ron Kay for who those who know him , he 's the founder of ICI . grad c: And he also approached me and he offered to help  our project and he was more thinking about some high level thinking tasks and I said " sure we need help you can come in as a subject " and he said " OK " . Which I 'll hopefully sort of scrape together t But , thanks to Fey , we already have sort of a nice blueprint and I can work with that . No ? No more questions ? grad e: I 'm not sure I totally understand this grad g: So what 's the s this is what you made , Fey ? grad c:  ? grad e: but I 'm not sure I totally understand everything that 's being talked about grad g: Like so So it 's just based on like the materials you had about Heidelberg . grad c:  are you familiar with with the with the very rough setup of the data ? grad e: but I I imagine I 'll c just catch on . undergrad d: Based on the web site , yeah , at the grad g: Oh OK there 's a web site grad c: experiment ? undergrad d: Right . grad g: and then you could like  figure out what the cate undergrad d: It 's a tourist information web site , grad e: this is where they 're supposed to undergrad d: so . grad c: The question is just sort of how do we get the tasks in their head that they have an intention of doing something and have a need to ask the system for something without giving them sort of a clear wording or phrasing of the task . grad c: Because what will happen then is that people repeat repeat , or as much as they can , of that phrasing . The The goals that we 've d you guys have been talking about are this these you know identifying which of three modes  their question  concerns . grad g: So it 's like the Enter versus View grad c: Yeah , we we we will sort of get a protocol of the prior interaction , grad g:  - huh . grad c: right ? That 's where the instructor , the person we are going to hire ,  and the subjects sit down together with these high level things grad g:  - huh . grad c: and so th the q first question for the subject is , " so these are things , you know , we thought a tourist can do . grad c: And the person can say " yeah , sure sh this is something I would do . Yeah ? and then we can sort of this s instructor can say " well ,  then you you may want to find out how to get over here grad g:   grad g: So the interaction beforehand will give them hints about how specific or how whatever though the kinds of questions that are going to ask during the actual session ? grad c: No . Just sort of OK , what what what would you like to buy and then  OK there you wanna buy a whatever cuckoos clocks grad g: Yeah . grad c: So the task then for that person is t finding out how to get there , right ? grad g:   grad c: And we know that the intention is to enter because we know that the person wants to buy a cuckoos clock . grad g: OK , that 's what  so like those tasks are all gonna be  unambiguous about which of the three modes . phd a: Well , so the idea is to try to get the actual phrasing that they might use and try to interfere as little as possible with their choice of words . In a sense that 's exactly the the the idea , phd a:   grad c: which is never possible in a in a s in a lab situation , phd a: Well , u u the one experiment th that that that I 've read somewhere , it was they u used pictures . We had exactly that on our list of possible way things so we  I even made a sort of a silly thing how that could work , how you control you are here you you want to know how to get someplace , and this is the place and it 's a museum and you want to do some and and and there 's a person looking at pictures . grad c: However , not only was the common census were among all participants of Friday 's meeting was it 's gonna be very laborious to to make these drawings for each different things , phd a: Right . grad c: all the different actions , if at all possible , and also people will get caught up in the pictures . phd a: I I 'm I 'm not saying it 's necessary but but  i   i you might be able to combine you know text  and and some sort of picture and also  I think it it will be a good idea to show them the text and kind of chew the task and then take the test away the the the the the text away grad c:   phd a: so that they are not  guided by by by what you wrote , grad c: We will phd a: but can come up with their with their own grad c: Yeah , they will have no more linguistic matter in front of them when they enter this room . Then I suggest we move on to the to we have   the EDU Project , let me make one more general remark , has sort of two two side   actions , its  action items that we 're do dealing with , one is modifying the SmartKom parser and the other one is modifying the SmartKom natural language generation module . And  this is not too complicated but I 'm just mentioning it put it in the framework because this is something we will talk about now .  , I have some news from the generation , do you have news from the parser ? grad f: not grad c: By that look I grad f: Yes , I would really p It would be better if I talked about it on Friday .  , did you run into problems or did you run into not h having time ? grad f: Yeah .  , meaning that Tilman Becker , who does the German one , actually took out some time and already did it in English for us . And so the version he 's sending us is already producing the English that 's needed to get by in version one point one . grad f: So I take it that was similar to the what what we did for the parsing ? grad c: Yeah . I I it even though the generator is a little bit more complex and it would have been , not changing one hundred words but maybe four hundred words , grad f: OK . grad c: but this this is I guess good news , and the  the time and especially Bhaskara and  and  Oh do I have it here ? No . It 's the last week of April until the fourth of May so it 's twenty - sixth through fourth . So it 's it 's extremely important that the two of you are also present in this town during that time . grad b: Wait , what what are the days ? April twenty - sixth to the May fourth ? grad c: Yeah , something like that . grad c: There is a d Isn't finals coming up then pretty much after that ? grad f: Finals was that . grad g: Yeah w it doesn't really have much meaning to grad students but final projects might . grad g: That grad c: Anyway , so this is grad b: Well I 'll be here working on something . Guaranteed , it 's just  will I be here , you know , in  I 'll be here too actually but phd a:  . grad c: No it 's just  you know they 're coming for us so that we can bug them grad g: Ye grad c: and ask them more questions and sit down together and write sensible code and they can give some nice talks and stuff . But  grad b: But it 's not like we need to be with them twenty - four hours a day s for the seven days that they 're here . It Just a mail that , you know , he 's sending me the the the stuff soon grad g: OK . grad c: and I and that 's also it 's it 's going to produce the concept - to - speech  blah - blah - blah information for necessary for one point one in English based on the English , you know , in English . grad c: we 're done ! " grad g: So that was like one of the first l You know , the first task was getting it working for English . grad c: when they have  changed the language model of the recognizer and the dictionary , then we can actually a put it all together grad g:   grad c: and you can speak into it and ask for TV and movie information grad e: Toll . grad c: and then when if if something actually happens and some answers come out , then we 're done . grad c: Then  grad g: Are they is it using the database ? the German TV movie . So all the actual data might be German names ? grad c:  well actually th  grad g: Or are they all like American TV programs ? grad c:  well grad e: I want to see " Die Dukes Von Hazard " grad c: The OK , so you don't know how the German dialogue  the German the demo dialogue actually works . It works the first thing is what 's , you know , showing on TV , and then the person is presented with what 's running on TV in Germany on that day , on that evening grad g:  -  ,   grad c: and so you take one look at it and then you say " well that 's really nothing there 's nothing for me there " " what 's running in the cinemas ? " So maybe there 's something better happening there . grad c: And then you get you 're shown what movies play which films , and it 's gonna be of course all the Heidelberg movies and what films they are actually showing . grad c: N grad g: It 's a  so would the generator , like the English language sentence of it is " these are the follow you know the following films are being shown " or something like that ? grad c: Yeah , but it in that sense it doesn't make In that case  it doesn't really make sense to read them out loud . grad c: But  it 'll tell you that this is what 's showing in Heidelberg and there you go . grad c: And the presentation agent will go " Hhh ! " Nuh ? grad g: OK . And then you pick pick a movie and and and it show shows you the times and you pick a time and you pick seats and all of this . grad c: But it 's so this time we we are at an advantage because it was a problem for the German system to incorporate all these English movie titles . grad c: Nuh ? But in English , that 's not really a problem , grad g: Right . grad c: unless we get some some topical German movies that have just come out and that are in their database . What 's the next thing ? grad b: e grad c: This is very rough but this is sort of what  Johno and I managed to come up with . The idea here is that grad b: This is the  s the schema of the XML here , not an example or something like that . grad c: Yeah this is not an XML this is sort of towards an a schema , grad e: OK . The idea is , so , imagine we have a library of schema such as the Source - Path - Goal and then we have forced  motion , we have cost action , grad e:   grad c: And they 're gonna be , you know , fleshed out in in their real ugly detail , Source - Path - Goal , and there 's gonna be s a lot of stuff on the Goal and blah - blah - blah , that a goal can be and so forth . This is a the fact that we 're calling this " action schema " right now should not entail that we are going to continue calling this " action schema " . But what that means is we have here first of all on the in the in the first iteration a stupid list of Source - Path - Goal actions grad b: Actions that can be categorized with or that are related to Source - Path - Goal . grad c: So a push may be in in in both you know push  in this or this  grad g: Forced motion and caused action for instance , grad c: Exactly . So this is something that , you know , may also be a res As a result of your work in the future , we may find out that , you know , there 're really s these subtle differences between  even within the domain of entering in the light of a Source - Path - Goal schema , that we need to put in fill in additional structure up there . It it is It was sort of a it gave us some headache , how do we avoid writing down that we have sort of the Enter Source - Path - Goal that this But this sort of gets the job done in that respect and maybe it is even conceptually somewhat adequate in a sense that  we 're talking about two different things . We 're talking more on the sort of intention level , up there , and more on the this is the your basic bone  schema , down there . When you point at the screen is it your shadow that I 'm supposed to look at ? grad g: Yeah . Whereas I keep looking where your hand is , and it doesn't grad c: Well , that wouldn't have helped you at all . grad b: Basically , what this is is that there 's an interface between what we are doing and the action planner grad e: Spit right here . grad b: and right now the way the interface is " action go " and then they have the what the person claimed was the source and the person claimed as the goal passed on . grad b: And the problem is , is that the current system does not distinguish between goes of type " going into " , goes of type " want to go to a place where I can take a picture of " , et cetera . grad c: So this is sort of what it looks like now , some simple " Go " action from it from an object named " Peter 's Kirche " of the type " Church " to an object named " Powder - Tower " of the type " Tower " . Right ? grad g: This is the  what the action planner uses ? grad b: Right . grad g: And is that and tha that 's changeable ? or not ? grad c: Yeah , well grad g: Like are we adapting to it ? grad c: No . grad g: Or grad c: We This is the output , sort of , of the natural language understanding , grad g: Oh , yeah . grad c: And what we are going to do , we going to and you can see here , and again for Johno please please focus the shadow , grad b: OK . grad c:  we 're gon   here you have the action and the domain object and w and on on grad g: What did you think he was doing ? grad b: I just grad g: OK , sorry . grad b: Robert likes to be abstract and that 's what I just thought he was doing . grad c: so as you can see this is on one level and we are going to add another  " Struct " , if you want , IE a rich action description on that level . grad c: So in the future grad g: So it 's just an additional information grad c: Exactly . In the future though , the content of a hypothesis will not only be an object and an an action and a domain object but an action , a domain object , and a rich action description , grad g: Right ? that doesn't hurt the current way . grad f: So  you had like an action schema and a Source - Path - Goal schema , grad g:  . grad f: right ? So how does this Source - Path - Goal schema fit into the  action schema ? Like is it one of the tags there ? grad g: Yeah can you go back to that one ? grad b: So the Source - Path - Goal schema in this case , I 've if I understand how we described we set this up ,  cuz we 've been arguing about it all week , but  we 'll hold the the Well in this case it will hold the  the the features I guess . I 'm not it 's hard for me to exactly s So basically that will store the the object that is w the Source will store the object that we 're going from , the Goal will store the the f grad e:   grad b: we 'll fill those in fill those roles in , right ? grad g: OK . grad b: The S Action - schemas basically have extra See we so those are schemas exist because in case we need extra information instead of just making it an attribute and which which is just one thing we we decided to make it 's own entity so that we could explode it out later on in case there is some structure that that we need to exploit . grad g: OK , so th sorry I just don't kn    This is just  XML mo notational but  the fact that it 's action schema and then sort of slash action schema that 's a whole entit grad b: That 's a block , yeah . grad g: That 's a block , whereas source is just an attribute ? grad c: No , no , no . Source meaning Source will be  will have a name , a type , maybe a dimensionality , grad g: Oh , OK , OK . OK could it it could also be blocked out then as grad b: Yeah , the So grad g: OK . grad c: s Source it will be , you know we 'll f we know a lot about sources so we 'll put all of that in Source . grad c: But it 's independent whether we are using the SPG schema in an Enter , View , or Approach mode , right ? grad g:   We can talk about Paths being the fastest , the quickest , the nicest and so forth ,  or or and the Trajector should be coming in there as well . So I guess the question is when you actually fill one of these out , it 'll be under action schema ? Those are It 's gonna be one y you 'll pick one of those for grad b: Right . grad g: OK these are this is just a layout of the possible that could go play that role . grad b: Right , so the the the roles will be filled in with the schema grad c:  ? grad g: OK , go it . grad b: and then what actual a action is chosen is will be in the in the action schema section . This was in this case it 's all  clear , sort of obvious , but you can think of the Enter , View and Approach as each having their roles , right ? the  it 's it 's implicit that the person that 's moving is doing entering viewing and approaching , but you know the usual thing is we have bindings between sort of they 're sort of like action specific roles and the more general Source - Path - Goal specific roles . grad c: What do you what do you grad g: What 's that ? Oh I guess it I I may be just  reading this and interpreting it into my head in the way that I 've always viewed things grad c:  . But if it is , then the top block is sort of like  , you know , you have to list exactly what X - schema or in this action schema , there 'll be a certain one , that has its own s structure and maybe it has stuff about that specific to entering or viewing or approaching , but those could include roles like the thing that you 're viewing , the thing that you 're entering , the thing that you 're grad e: So very specific role names are " viewed thing " , " entered thing " grad g: whatever , you know , that which are think think of enter , view and approach as frames grad c:   grad g: and you can also describe them in a general way as Source - Path - Goal schema and maybe there 's other image schemas that you could you know add after this that you know , how do they work in terms of you know a force dynamics grad c:   So all of those have  basically f either specific frame specific roles or more general frame specific roles that might have binding . grad g: and it 's not  I don't know if you wanna have in the same level as the action schema SPG schema it it 's somewhere in there that you need to represent that there is some container and the interior of it corresponds to some part of the Source - Path - Goal  you know goal  goal I guess in this case . grad g: So  is there an easy way in this notation to show when there 's identity basically between things grad c: Yeah . grad g: and I di don't know if that 's something we need to invent or you know just grad b: The wa wasn't there supposed to be a link in the grad f: Right . grad b: I don't know if this answers your question , I was just staring at this while you were talking , sorry . grad b:  a link between the action schema , a field in the s in the schema for the image schemas that would link us to which action schema we were supposed to use so we could grad c: Yeah .  , well that 's that 's one one thing is that we can link up , think also that  we can have one or m as many as we want links from from the schema up to the s action  description of it . grad c: But the notion I got from Nancy 's idea was that we may f find sort of concepts floating around i in the a action description of the action f " Enter " frame up there that are , e when you talk about the real world , actually identical to the goal of the the S Source - Path - Goal schema , grad g: Exactly . grad c: and do we have means of of telling it within that a and the answer is absolutely . grad c: The way we absolutely have those means that are even part of the M - three - L A API , grad g: Yeah . This referencing thing however is of temporary nature because sooner or later the W - three - C will be finished with their X - path , specification and then it 's going to be even much nicer . Then we have real means of pointing at an individual instantiation of one of our elements here grad g:   grad c: and link it to another one , and this not only within a document but also via documents , grad g:   grad g: So you know happen to know how what what " sooner or later " means like in practice ? grad c: That 's but it 's soon . grad c: So it 's g it 's the spec is there and it 's gonna part of the M - three - L AP API filed by the end of this year so that this means we can start using it basically now . grad b: Basically references from the roles in the schema the bottom schemas to the action schemas is wha  I 'm assuming . Yeah ,  personally , I 'm looking even more forward to the day when we 're going to have X forms , which l is a form of notation where it allows you to say that if the SPG action up there is Enter , then the goal type can never be a statue . grad g: So you have constraints that are dependent on the c actual s specific filler , of some attribute .  , you know this , of course , does not make sense in light of the Statue of Liberty , grad g:  - huh . grad c: Yeah ? grad f: S So  , like are you gonna have similar schemas for FM grad g: Or the Gateway Arch in St . grad f: like forced motion and caused action and stuff like you have for SPG ? grad c: Yeah . grad f: And if so like can are you able to enforce that you know if if it 's if it 's SPG action then you have that schema , if it 's a forced motion then you have the other schema present in the grad c:  we have absolute No . We have absolutely no means of enforcing that , so it would be considered valid if we have an SPG action " Enter " and no SPG schema , but a forced action schema . grad g: Whi - which is not bad , because  , that there 's multiple sens  that particular case , there 's mult there there 's a forced side of of that verb as well . grad c: What 's also nice , and for a i for me in my mind it 's it 's crucially necessary , is that we can have multiple schemas and multiple action schemas in parallel . grad c: And  we started thinking about going through our bakery questions , so when I say " is there a bakery here ? " you know I do ultimately want our module to be able to first of all f tell the rest of the system " hey this person actually wants to go there " and " B " , that person actually wants to buy something to eat there . Nuh ? And if these are two different schemas , IE the Source - Path - Goal schema of getting there and then the buying snacks schema , nuh ? grad g: Would they both be listed here in grad c: Yes . grad c: ye Yeah , they they would both schemas would appear , so what is the  is is there a " buying s snacks " schema ? grad e: Snack action . grad c: What is the  have grad g: What ? grad c: the buying snack schema ? grad e: See . undergrad d: Buying buying his food grad e: I 'm sure there 's a commercial event schema in there somewhere . grad g: Yeah I I grad c: Yeah ? So  so we would we would instantiate the SPG schema with a Source - Path - Goal blah - blah - blah grad g: I see . grad c: and the buying event you know at which however that looks like , the place f thing to buy . Would you say that the like  you could have a flat structure and just say these are two independent things , but there 's also this sort of like causal , well , so one is really facilitating the other and it 's part of a compound action of some kind , which has structure . Now it 's technically possible that you can fit schema within schema , and schema within schemata grad g:  I I think that 's nicer for a lot of reasons but might be a pain so  grad c:  Well , for me it seems that  r Yes . grad g:  there are truly times when you have two totally independent goals that they might express at once , but in this case it 's really like there 's a purpo means that you know f for achieving some other purpose . grad c: Well , if I 'm if I 'm recipient of such a message and I get a Source - Path - Goal where the goal is a bakery and then I get a commercial action which takes place in a bakery , right ? and and and they they are obviously , via identifiers , identified to be the same thing here . grad c: No , no , just the Yeah ? grad g: Yeah because they 're two different things one of which is l you could think of one a sub you know pru whatever pre - condition for the second . And they 're independently they they are events which have very different characters as far as Source - Path - Goal whatever . grad g: So when you identify Source - Path - Goal and whatever , there 's gonna to be a desire , whatever , eating , hunger , whatever other frames you have involved , they have to match up in in nice ways . So it seems like each of them has its own internal structure and mapping to these schemas grad c:   grad g: you know from the other But you know that 's just That 's just me . grad c: Well , I think we 're gonna hit a lot of interesting problems grad g: Like I I grad c: and as I prefaced it this is the result of one week of arguing about it grad g:   grad e: Yeah  I I still am not entirely sure that I really fully grasp the syntax of this . grad b: Well it 's not it 's not actually a very actually , it doesn't actually grad c:  it occur it occurs to me that  ne grad e: You know , like what Right . grad c:  well I should have we should have added an ano an XML example , grad e: Yeah . grad c: and and this is on on a on on my list of things until next next week . grad c: It 's also a question of the recursiveness and and a hier hierarchy  in there . grad c: Do we want the schemas just blump blump blump blump ?  it 's if we can actually you know get it so that we can , out of one utterance , activate more than one schema , then we 're already pretty good , grad g:   grad c: right ? phd a: Well well you have to be careful with that   thing because   many actions presuppose some  almost infinitely many other actions . phd a: You have a further specific intentions to left to lift your right foot grad c:  ? phd a: and so    y you really have to focus on on on grad g: Right . phd a: and decide the level of of abstraction that that you aim at it kind of zero in on that , grad c: Yeah . phd a: and more or less ignore the rest , unless there is some implications that that you want to constant draw from from sub - tasks  that are relevant   but very difficult . grad g: M Th The other thing that I just thought of is that you could want to go to the bakery because you 're supposed to meet your friend there or som phd a: Yeah . grad g: you know so you like being able to infer the second thing is very useful and probably often right . grad b: Well the the the utterance was " is there a bakery around here ? " , grad g: But having them separate grad b: not " I want to go to a bakery . " grad g: Well maybe their friend said they were going to meet them in a bakery around the area . grad g: And I 'm , yeah I 'm I 'm inventing contexts which are maybe unlikely , phd a: Right . grad g: but yeah  like but it 's still the case that  you could you could override that default by giving extra information grad c:  -  , yeah . grad g: which is to me a reason why you would keep the inference of that separate from the knowledge of " OK they really want to know if there 's a bakery around here " , grad c: Yeah . grad c: Well there there there should never be a hard coded  shortcut from the bakery question to the  double schema thing , grad g: Right . grad c: how  And , as a matter of fact , when I have traveled with my friends we make these exactly these kinds of appointments . It 's I met someone at the bakery you know in the Victoria Station t you know train station London before , phd a: Right . It 's like phd a: So the Enter - View - Approach the the the EVA  , those are fixed slots in this particular action . Or or or or will it just   is it change grad e: Every SPG every SPG action either is an Enter or a View or an Approach , phd a: Right , right . grad e: right ? phd a: So so I  for for each particular action that you may want to characterize you would have some number of slots that define    you know in some way what this action is all about . So is it a fixed number or or do you leave it open it could be between one and fifteen  it 's it 's it 's flexible . grad c: the  Well , it sort of depends on on if you actually write down the the schema then you have to say it 's either one of them or it can be none , or it can be any of them . However the  it seems to be sensible to me to r to view them as mutually exclusive  maybe even not . grad g: J Do you mean within the Source - Path - Goal actions ? phd a:  ye   b I  I u I understand grad c: Yeah . grad g: Those three ? phd a:  but grad c: And  how how where is the end ? So that 's phd a: No , no . There a a actually by I think my question is simpler than that ,  is OK , so you have an SPG action and and it has three different   aspects  because you can either enter a building or view it or or approach it and touch it or something .  now you define  another action , it 's it 's called   s S P G - one grad c: Forced action or forced motion . And i in in a way similar to either Enter - View - Approach you may want to send a letter , read a letter , or dictate a letter , let 's say . So , h grad b: Oh the OK  maybe I 'd The  These actions I don't know if I 'm gonna answer your question or not with this , but the categories inside of action schemas , so , SPG action is a category . Real although I think what we 're specifying here is this is a category where the actions " enter , view and approach " would fall into because they have a related Source - Path - Goal schema in our tourist domain . Cuz viewing in a tourist domain is going up to it and or actually going from one place to another to take a picture , in this in a phd a: Right . Oh , s so it 's sort of automatic derived fr from the structure that that is built elsewhere . grad b: derived I don't know if I u grad e: This is a cate this a category structure here , grad b: Right . What are some types of action schemas ? Well one of the types of action schemas is Source - Path - Goal action . So if I want to go from outside to inside then you 'd have the roles that need to filled , where you 'd have a Source - Path - Goal set of roles . So you 'd the Source would be outside and Path is to the door or whatever , right ? phd a: Right . grad b: So if you wanted to have a new type of action you 'd create a new type of category . Then this category would we would put it or not necessarily We would put a new action in the m  in the categories that in which it has the  Well , every action has a set of related schemas like Source - Path - Goal or force , whatever , right ? grad e:   grad b: So we would put " write a letter " in the categories  that in which it had it w had  schemas u grad e: There could be a communication event action or something like that grad b: Exactly . grad b: And then later , you know , there the we have a communication event action where we 'd define it down there as grad g:  . So there 's a bit a redundancy , right ? in in which the things that go into a particular You have categories at the top under action schema and the things that go under a particular category are  supposed to have a corresponding schema definition for that type . So I guess what 's the function of having it up there too ?  I guess I 'm wondering whether You could just have under action schema you could just sort of say whatever you know it 's gonna be Enter , View or Approach or whatever number of things grad c:   grad g: and pos partly because you need to know somewhere that those things fall into some categories . And it may be multiple categories as you say which is  the reason why it gets a little messy grad c: Yeah . grad g:  but if it has if it 's supposed to be categorized in category X then the corresponding schema X will be among the structures that that follow . grad g: That 's like grad c: th this is this r grad g: OK , sorry . grad c: this is this is more this is probably the way that th that 's the way that seemed more intuitive to Johno I guess grad g: You didn't tell me to grad c: also for a while for grad g:  - huh . grad b: The the reason One reason we 're doing it this way is in case there 's extra structure that 's in the Enter action that 's not captured by the schemas , grad g: I it 's easy to go back and forth isn't it ?  - huh . grad b: right ? grad g: Which is why I would think you would say Enter and then just say all the things that are relevant specifically to Enter . grad b: Right , but grad e: Ri - You 'd like so you 're saying you could practically turn this structure inside out ? or something , or ? grad g:  Ye - I see what you mean by that , grad c: No basically w grad g: but I I don't if I would I would need to have t have that . grad c: Get get rid of the sort of SPG slash something  or the sub - actions category , grad g: Right . grad c:  and I agree that you know this is something we need to discuss , grad g: I in fact what you could say is for Enter , grad c: yeah . grad g: you could say " here , list all the kinds of schemas that on the category that grad e: List all the parent categories . So you would say enter and you 'd say my parent frames are such - and - such , h and then those are the ones that actually you then actually define and say how the roles bind to your specific roles which will probably be f richer and fuller and have other stuff in there . This sounds like a paper I 've read around here recently in terms of grad g: Yeah it could be not a coincidence . Like I said , I 'm sure I 'm just hitting everything with a hammer that I developed , grad e: Yeah . grad g: but  you know  it 's I 'm just telling you what I think , you just hit the button and it 's like grad c: And , I guess fr  grad e: Yeah  but there 's a good question here . Like ,   do you When do you need Damn this headset ! When you this  , eh grad g: Metacomment . I just don't see why you would  does th Who uses this  this data structure ? You know ? Like , do you say " alright I 'm going to  do an SPG action " . And then you know somebody ne either the computer or the user says " alright , well , I know I want to do a Source - Path - Goal action so what are my choices among that ? " And " oh , OK , so I can do an Enter - View - Approach " . It 's not like that , right ? It 's more like you say " I want to ,  I want to do an Enter . " grad b: Well only one of grad e: And then you 're more interested in knowing what the parent categories are of that . Right ? So that the  the  sort of representation that you were just talking about seems more relevant to the kinds of things you would have to do ? grad b: I 'd I grad g:  . grad b: if we if our if our module decided that Enter is the case , View and Approach will not be there . grad c: Well  it 's it sort of came into my mind that sometimes even two could be on , and would be interesting . grad c:  nevertheless  grad e: Mayb - Well maybe I 'm not understanding where this comes from and where this goes to . grad b: Well in that case , we can't we can't w if if grad c: l let 's let 's not grad g: OK . grad b: well the thing is if that 's the case we our I don't think our system can handle that currently . grad e: What are we doing with this ? grad c: No , not at all . " grad c: the I think the in some sense we we ex get the task done extremely well grad g: Run like this  grad c: because this is exactly the discussion we need need . grad g: No , this is the useful , grad c: and  and and I th I hope grad g: you know , don don't worry . grad c: And it 's gonna get more and more complex the the l complexer and larger our domains get . grad b: The r  the in terms of why is it 's laid out like this versus some other grad c: the people grad e: Yeah . grad b:  that 's kind of a contentious point between the two of us but this is one wa so this is a way to link  the way these roles are filled out to the action . grad b: Because if we know that Enter is a t is an SPG action , grad e:   grad b: we know to look for an SPG schema and put the appropriate fill in the appropriate roles later on . grad g: And you could have also indicated that by saying " Enter , what are the kinds of action I am ? " grad c:  -  , yeah . grad g: Right ? So there 's just like sort of reverse organization , right ? So like unless @ @ Are there reasons why one is better than the other  that come from other sources ? grad e: Again grad c: Yes because nobod no the modules don't grad g: Yeah .  grad c: This is this is a schema that defines XML messages that are passed from one module to another , grad g:   grad c: mainly meaning from the natural language understanding , or from the deep language understanding to the action planner . grad c: Now the the reason for for not using this approach is because you always will have to go back , each module will try have to go back to look up which  you know entity can have which  , you know , entity can have which parents , and then So you always need the whole body of of y your model  to figure out what belongs to what . grad c: nuh ? So you always send up " here I am I am this person , and I can have these parents " in every message . grad c: which e grad g: OK , so it 's just like a pain to have to send it . grad c: It may or may not be a just a pain it 's it 's I 'm completely willing to to to throw all of this away grad g: OK , I understand . grad c: and completely redo it , grad e: Well grad c: you know and and and it after some iterations we may just do that . grad e: I I would just like to ask  like , if it could happen for next time , just beca cuz I 'm new grad c:   grad e: and I don't really just I just don't know what to make of this and what this is for , and stuff like that , you know , so if someone could make an example of what would actually be in it , grad c: Yeah . grad e: like first of all what modules are talking to each other using this , grad c: Yeah , we I will promise for the next time to have fleshed out N XML examples for a a run through and and see how this this then translates , grad e: right ? And OK . grad c: And  is there more to be said ? I think  In principle what I I think that this approach does , and e e whether or not we take the Enter - View and we all throw up up the ladder  wha how do how does Professor Peter call that ? grad g: Yeah . grad c: The  hhh , silence su sublimination ? Throwing somebody up the stairs ? Have you never read the Peter 's Principle anyone here ? grad e: Nope . phd a: Oh ,  grad f: People reach their level of  max their level of at which they 're incompetent or whatever . grad c: OK , so we can promote Enter - View all all up a bit and and get rid of the  blah - blah - X - blah  asterisk sub - action item altogether . grad c: and we w we we will play around with all of them but the principal distinction between having the the pure schema and their instantiations on the one hand , and adding some whatever , more intention oriented specification  on parallel to that that this approach seems to be  workable to me . grad g: Although  roles grad b: So I I do I 'm I 'm not grad c: I 'm I 'm never happy when he uses the word " roles " , grad g: Yeah I I grad c: I 'm grad g: Yeah . I was going to grad b: I b  ROLLS so grad g: Bread rolls ? grad e: Oh you meant pastries , then ? grad b: Yeah , pastries is what I 'm talking about . grad e: Help ! grad g: I guess I 'll agree to that , then . grad g: I n Didn't you say something about Friday , grad c: Yeah . grad c: So it looks like you have not been partaking , the Monday at three o ' clock time has turned out to be not good anymore . So people have been thinking about an alternative time and the one we came up with is Friday two - thirty ? three ? What was it ? grad b: You have class until two , right ? so if we don't want him if we don't want him to run over here grad f:   grad c: Two - th Two - thirty - ish or three or Friday at three or something around that time . grad c: And I know that you have until three You 're busy ? grad e:  undergrad d: Yeah . Yeah  earlier on Friday is better but three you know  if it were a three or a three thirty time then I would take the three or whatever , grad c:   grad c: You are more than welcome if you think that this kind of discussion gets you anywhere in in your life then  you 're free to c undergrad d: It 's fascinating . " undergrad d: I 'm just glad that I don't have to work it out grad c:  undergrad d: because . grad c:  ? undergrad d: I 'm just glad that don't have to work it out myself , that I 'm not involved at all in the working out of it because . phd a: So it 's at Friday at three ? there that 's grad c: And  grad e: So already again this week , grad c: How diligent do we feel ? grad e: huh ? grad c: Yeah . So  clearly there 's I can talk about the  the parser changes on Friday at least , grad c: OK , Bhaskara will do the big show on Friday . grad g: And you guys will argue some more ? grad b: And between now and then yeah . grad g: and have some ? grad c: We will r grad e: Promise ? grad g: probably . grad g: And we 'll get the summary like , this the c you know , short version , like phd a: An - and I would like to second Keith 's request . grad g: S phd a: An example wo would be nice t to have kind of a detailed example . I 've I 've I 've I guess I 'm on record for promising that now . grad c: This is it and  grad b: The other good thing about it is Jerry can be on here on Friday and he can weigh in as well . and  if you can get that binding point also maybe with a nice example that would be helpful for Johno and me . let 's  yeah they 're grad c: Give us undergrad d: No problem , grad e: I think you 've got one on hand , undergrad d: yeah . grad c: Well the the the binding is technically no problem but it 's it for me it seems to be conceptually important that we find out if we can s if if there if there are things in there that are sort of a general nature , we should distill them out and put them where the schemas are . grad c: If there are things that you know are intention - specific , then we should put them up somewhere , a grad g: So , in general they 'll be bindings across both intentions and the actions . grad g: it 's like  Shastri would say you know binding is like an essential cognitive  process . grad g: So I don't think it will be isolated to one or the two , but you can definitely figure out where Yeah , sometimes things belong and So actually I 'm not sure I would be curious to see how separate the intention part and the action part are in the system . Like I know the whole thing is like intention lattice , or something like that , grad c:   grad g: right ? So is the ri right now are the ideas the rich rich the RAD or whatever is one you know potential block inside intention . grad g: and then that 's just one way to describe the the action part of it . grad c: it 's an an it 's it 's sort of grad g: Not just that you want to go from here to here , it 's that the action is what you intend grad c: Yeah . grad g: and this action consists of all com complicated modules and image schemas and whatever . And and there will be a a a relatively high level of redundancy grad g: So . which is , yeah , It 's fine grad c: so th so that if we want to get really cocky we we will say " well if you really look at it , you just need our RAD . grad c: Because you 're not gonna get anymore information out of the action a as you find it there in the domain object . grad c: But then again  in this case , the domain object may contain information that we don't really care about either . grad c: H But w we 'll see that then , and how how it sort of evolves . grad c:  if if people really like our our RAD ,  w what might happen is that they will get rid of that action thing completely , you know , and leave it up for us to get the parser input  grad g: Mmm . We know the things that make use of this thing so that we can just change them so that they make use of RAD . grad g: Is the But what is the " why " ? grad e: It 's rad , even ! grad b: Why ? grad g: Why ? grad e: It happened to c be what it stands for . grad c: ye no but i undergrad d: Just think of it as as " wheel " in German . grad c: but if you if you if you work in th in that XML community it is a great acronym grad g: Do you see what  ? Like grad c: because it e evokes whatever RDF grad g: Oh . grad c: RDF is the biggest thing right ? That 's the rich sort of " Resource Description Framework " grad e: Oh " rich de " grad g: Oh . grad c: and  and also So , description , having the word d term " description " in there is wonderful , grad g:   grad g: But what if it 's not an action ? grad c: It 's it 's rad , undergrad d: Yeah all the kids 'll love it .  are the are the sample data that you guys showed sometime ago like the things maybe maybe you 're gonna run a trial tomorrow . Cuz it 'd be nice for me to like look if I 'm thinking about examples I 'm mostly looking at child language which you know will have some overlap but not total with the kinds of things that you guys are getting . grad g: and maybe you 've posted it before but where would I look if I want to see ? grad c: Oh I You want audio ? grad g: You know . grad g: OK , well don't don't make it a high priority grad c: Yeah . grad g: I In fact if you just tell me like you know like two examples grad c:   grad g: y The the the representational problems are I 'm sure , will be there , grad c: OK