text:
Q: PowerBI Visualizations: Debug + Quick Compile I have been exploring the PowerBI visualizations github project and starting to make my own visualization but I'm having the following issues that is really slowing me down on the development:

*

*It takes almost 3m to run the "build", is there any other option for rapid development because when we are starting with the project 3m see the results of the code changed is too much...

*Is there any way to debug the visualizations inside the framework? I had setted the property "DEBUG" to false in the gulpfile.js but without success...

Thanks!

A: If you are a windows user then just use Visual Studio 2015 solution PowerBIVisuals.sln from src/ folder
It has continuous build support, so that when you open the solution it will run continuous_build_debug task.
If you are a Unix/Linux/Mac user then you'll need to manually run continuous_build_debug. (Run "gulp continuous_build_debug")
This task will run Watchers which will build project parts after you save changes to any file.


text:
Home / Featured News / Asia Pacific Cup – Day Two
Asia Pacific Cup – Day Two
Japan are the only undefeated team after two days of intense competition in the Asia Pacific Cup.
After beating the Aussie Spirit in the night game, Japan has moved clear at the top of the standings, having scored 26 runs and conceded only one.
Australia, China and Chinese Taipei are next in the standings, each with 3-1 win-loss records.
Chinese Taipei were the big movers on Day Two, winning games against New Zealand and Italy.
Purchase tickets here
Day Three schedule here
DAY TWO GAME REPORTS
China 1, All Stars 0
This was a classic pitchers' duel through five innings, with neither All Stars starter Vanessa Stokes nor China's Xinxing Zhao conceding a run.
But in the sixth inning, the Chinese team manufactured a run after leadoff hitter Huan Li was hit by a pitch and was sacrificed to second. Xiaoqing Wang then singled to centre field, scoring Huan Li for what would be the only run of the game.
The All Stars tried to stage a late comeback, and were given hope when Rachel Lack and Carmelle Sorensen both singled in the sixth inning and advanced into scoring position on a wild pitch. But reliever Lan Wang pitched out of the jam, ending the threat.
China 5, Italy 0
China completed the second leg of a winning double-header by overcoming a determined Italy.
Errors proved costly for the Italians, who made four of them as China won 5-0.
Chinese starting pitcher Lan Wang allowed just four hits and one walk, with four strikeouts, in seven innings.
Lan Wang had three of China's seven hits.
Chinese Taipei 7, New Zealand 4
After falling behind early, New Zealand staged a strong comeback in the late innings but fell short.
Chinese Taipei scored three runs in the second inning, two of them driven in by Hsuan Liu, to take the lead.
The Kiwis hit back with two runs in the fifth inning, helped by some sloppy defence and an RBI single by Erin Blackmore.
They kept up the pressure by adding runs in the final two innings, but Chinese Taipei scored three late runs to hold off the challenge.
Australia 3, All Stars 1
It's not often that the Aussie Spirit get to play against another team of Australians, so this game was always going to be special.
The underdog All Stars were first to score, courtesy of an Erin Thras RBI single that scored Rachel Lack in the first inning.
But the lead didn't last long. In the bottom of the inning, Stacey Porter belted a solo home run to right field to tie the scores.
Two fielding errors by the All Stars in the third inning led to Michelle Cox first getting on base, then later scoring to give the Spirit a 2-1 lead.
They added another run in the sixth, when Leigh Godfrey doubled and scored on Stacey McManus's single.
Both starting pitchers were excellent. Kaia Parnaby pitched a complete game for the Spirit, striking out eight while allowing just one run on four hits and one walk.
For the All Stars, Stephanie Trzcinski allowed only one earned run in 5.1 innings, giving up just three hits and one walk while striking out two.
Japan 10, New Zealand 0
The powerhouse Japanese lineup wasted no time stamping their authority on this game, scoring eight runs in the first inning.
Hitomi Kawabata, Yu Yamamoto and Natsuko Sugama each had two hits for Japan, who scored again in the second and third innings to end the contest early.
Misaki Katsumata and Miki Kozono shared the pitching duties, allowing five hits and no walks between them.
Chinese Taipei 8, Italy 1
Chinese Taipei capped off a great day with a comfortable win over Italy.
Feng-Chen Lin led the offence with two hits and three RBIs. Yi-Ting Yang also had two hits, as did Pei-Chun Lin.
Pitcher Ya-Ting Tu restricted Italy to three hits, with no walks and six strikeouts.
Japan 3, Australia 0
An impressive pitching performance by Yukari Hamamura carried Japan to an important victory over the Aussie Spirit.
The talented Spirit lineup could manage only two base hits against Hamamura, who struck out six and walked only one.
Natsuko Sugama was the batting star, with three hits including a two-run homer that iced the game in the seventh inning.
For the Spirit, only Stacey Porter and Georgia Casey hit safely.
Pitcher Tarni Stepto battled hard, restricting Japan to one through the first six innings until Sugama's blast in the seventh.
Standings after Day Two
Runs for
Runs against
Japan 4 4 0 26 1
Australia 4 3 1 18 4
China 4 3 1 10 11
Chinese Taipei 4 3 1 26 12
All Stars 4 1 3 8 14
Italy 4 0 4 2 23
New Zealand 4 0 4 4 29
lWatch Asia Pacific Cup final live on Fox Sports
Asia Pacific Cup – Day Threer

redpajama_set_name: RedPajamaCommonCrawl


My response, collected into a single blog post (and slightly edited to remove the names of the innocent and not-so-innocent) because I think I'm going to want to point at this sometime rather than recreate it when the topic comes up again.
I do not consider myself particularly imaginative. I know people who blow me away with the imagination they show. I have, however, developed some tools that help me fake it when doing game design — which is in many ways like writing, except that the only thing you can count on the characters doing is "not what you expect".
First, an 'entity definition template' that captures important points about each major entity (which can be a person, place, thing, event, whatever). Name, theme (what is it about) description (how do you know when it stands in front of you? How do you know where it's been/that it was here? How do you know it is coming?), what threats does it embody? (what does it threaten, and/or what is it threatened by?).
Second, a set of questions I ask while preparing the entity definitions. There are the normal 5Ws+H (Who, What, Where, When, Why, and How), but I tend to structure them in my mind as "Challenge and Response", "how does this entity interact with the world?", and add another component, 'Secret' ("what is not generally known about this entity?").
Between the two, I have enough to guide me through the creation of (I think) interesting entities and story elements. Having a set of common questions leads me to ideas I might not otherwise have come up with. If I need to, though, I have a third tool in my kit, random generators. Seventh Sanctum has a bunch of random generators that can prod my thoughts, and links to other sites with related materials and tools.
Oh, plus hundreds of pounds (weight or monetary, though I'm in Canada and we use dollars here) of gaming books full of idea tools.
Note that these are in some ways developed through procedural methods. As I said, I do not consider myself 'imaginative', I know people who regularly come up with cooler and weirder ideas than I do… but these techniques go a long way toward giving me something I can use to build a story around. At every stage while considering these questions I discover things I don't yet know that I might be interested in. The entities all have threats and secrets built into them, providing a tapestry of potential conflict to explore (seriously, I draw relationship diagrams to help me figure these things out).
When I'm developing something, I keep a log (separate text file) for the "thought of something but can't think about it right now because I'm busy!" ideas.
Well, it works for me.


text:
UK government rejects compensations for victims of Libya-funded IRA attacks
By Libyan Express Published: 16 Sep 2017
Shankill Road bombing: The history of collusion between MI5 and the IRA | The Independent
The government has rejected a call for a UK reparations fund for victims of Libyan-sponsored IRA attacks.
High Council of State decries bias of Arab League, calls for changing location…
The fund was originally suggested by a cross-party group of MPs on the Northern Ireland Affairs Committee.
They have described the government's response as unacceptable.
Former Libyan leader Colonel Muammar Gaddafi supplied arms to the IRA and, during the Troubles, Libyan Semtex plastic explosive was a key weapon in the group's bombing campaign.
MPs on the Northern Ireland Affairs Committee, which investigated the issue for many months during the last parliament, originally called for compensation for victims.
They also wanted a special fund set up and Libyan assets in the UK frozen.
IRALibyaUK
High Council of State decries bias of Arab League, calls for changing location of its…


Have you ever thought about, while combing your hair, who invented the hair comb or when was the first comb made? You will be surprised to know that the history or use of hair combs dates as 5,000 years ago! Combs are actually among the oldest tools found by archaeologists. As early as 5500 B.C. the ancient Egyptians carved out combs among other remnants of the emerging cultures. Similarly in China, combs were worn as hair accessories that reflected one's social status. However, historians have no way to know who the inventor of the first comb was. Archaeologists have found combs in settlements from 5,000 years ago in Persia. Hair combs are used by humans to separate tangled hairs, to keep their hair clean and to style their hair. They are also used as a decoration for hair.
Apart from using combs to part hair for coloring, conditioning and braiding, people especially in early times have also worn combs as decorative hair accessories that hold hair in a specific style. During the 1930s and 1940s, when hair updos were in the fashion, the decorative hair comb became a popular accessory.
The need or idea to invent combs, certainly, would have arisen from the need to keep the hair in order, neat, and with a tidy appearance, and in few cases with the intention of keeping the head free of lice and other parasites. The various designs and shapes of combs, from rudimentary prehistoric combs to the elaborated designs of middle Ages and further centuries, are testimonials of how people concerned about hair styles' appearance.


Our Passion came from very early on, its called being a petrol head. This personal interest in cars has grown over 40 years and has led us to the creation of Car-Iconics.
Our experience includes driving, collecting, racing and restoration of cars. Some of the cars and services we have personally experienced have not met our expectations. Car-Iconics is built around our personal experience of cars and the desire to provide outstanding cars and service to those customers who appreciate that the total cost of buying / owning a car is not just about the lowest buying price.
Our brand commitment is to ensure that to the best of our capability, our cars are fully prepared for you to drive and enjoy.


Welcome to the City of Richmond Department of Fire and Emergency Services website. Located in the heart of downtown Richmond, we have been serving the citizens of this great City since 1858. As the City's Fire Chief my hope is that, as you browse through our site, you find the information contained within each section both informative and possibly lifesaving as well. As you can see by the attached menu, we handle a myriad of duties and our personnel are constantly training and learning new skills to continue to keep our citizens safe. We strive every day to ensure Fire Safety in both the home and workplace. So, please sit back and relax as you browse this site and feel free to contact us on our website with any questions. At Richmond Fire we are "there when you need us, every day, every call, every time".
Firefighters will come to your home and perform a safety survey at no cost. Call (804)646-1526 for more information.
The City of Richmond provides and installs smoke detectors to citizens at no charge. Call Fire Prevention at (804)646-1526 to schedule an installation.


text:
Plantable Candles That Help You Do Your Part
Pass It On plantable candles 2.0 is back with new scents & product upgrades! Candle holders possess a 2nd life as planting pots.
[UPDATE] There's an error in the estimated delivery time - it should be July 2020 not July 2022. We will ship out the products in 2 months after the campaign ends. Thank you for your support!
Pass It On plantable candles pay tribute to iconic destinations that are endangered due to climate change.
Our candles are made of sustainable materials and are designed to encourage our users to give the candles a second life by repurposing them as planting pots.
Most importantly, for every purchase of a Pass It On candle, we will donate a portion of the proceeds to Eden Reforestation Projects to plant 10 trees.
New Scents, Product Upgrades And Better Rewards!
Thanks to your support towards our first successful campaign, we are back with new scents, products upgrades and better rewards!
(1) NEW SCENTS
For our two new scents, we draw inspiration from the lush Congo Basin and Venice's breezy waterways. Finished in two beautiful earth tones, these new scents expand our colour palette.
NEW: Congo Basin - Orchid & Vetiver
In the Democratic Republic of Congo, refugees from armed conflict are forced to live off the land with severe impacts to forests and wildlife. Palm oil plantations and other commercial agriculture operations are a growing threat, putting greater pressure on the Congo's forests and rivers.
NEW: Venice - Rose & Patchouli
As the city is built on marshland at the edge of a lagoon, its existence has always required maintaining a careful balance between the city and the natural world. But in recent years, rising sea levels due to climate change has threatened to throw off the balance.
(2) PRODUCT UPGRADES
Since our first campaign, we've gathered feedback and spent weeks enhancing our product to be even more socially responsible.
First, we realised that although many people loved the idea of planting our seed-infused candle lids, not many users did.
So here's the good news: We're providing you with soil!
For this batch, every candle will come with compressed soil pellets. All you have to do is add some water, and the pellets will expand into a handful of soil!
Second, we're upgrading our wicks to be twice the thickness so that it burns the candle more evenly, giving you stronger scent throw.
Lastly, our packaging is now made from FSC-certified cardboard. The FSC certification is considered the "gold standard" in eco terms for papermaking, as wood is harvested from forests that are responsibly managed, socially beneficial, environmentally conscious and economically viable.
(3) BETTER REWARDS
Previously, we gave away postcards; this time, we're giving away a bamboo candle cover to those who purchase 2 or more candles!
Bamboo is a kind of grass, which makes growing it about as easy as it gets. Being extremely adaptable and resilient, bamboo is a super sustainable alternative to wood or plastic.
Also, we're offering 4 sets of limited edition larger-sized candles for our ultimate changemakers! Each set will include an exclusive 185g candle in these beautiful cement pots:
Alps (large) & Venice (standard)
Amazon (large) & Congo (standard)
The Green Factor
Pass It On candles are made of sustainable materials. The candle wax is made with 100% soy wax, and does not contain any dyes, parabens, paraffin or phthalates.
Turn Pass It On candle holders into flower pots!
Our candle holders have been designed to be repurposed as gardening cement pots (which is why they're not glass jars), and our candle lids are made of biodegradable paper infused with wildflower seeds. So plant candle lid into pot after finishing your candle.
Packaging boxes are made of FSC-certified cardboard. No bubble wrap or plastic materials are used.
The Scents
Pass It On presents its new collection of scented candles, inspired by iconic destinations around the world that are endangered due to climate change.
The scents were created to remind us of these endangered destinations with the hope to encourage the current generation to do their part in protecting our environment.
Amazon - Goji & Tarocco Blood Orange
Brazil's Amazon is the largest rainforest in the world. It is home to the world's most diverse species, but expansion of agriculture has led to the destruction of the rainforest and wildlife.
Alps - English Pear & Freesia
Every year, the European mountain range loses around 3% of glacial ice, which means that by 2050, there might not be any more glaciers.
Great Barrier Reef - Wood Sage & Sea Salt
The world's largest coral reef, Australia's Great Barrier Reef, has decreased by more than half its size because of rising temperatures in the past 30 years.
Taj Mahal - White Tea & Ginger
The Taj Mahal is one of the most iconic buildings in the world, but some experts worry that the site could collapse because of erosion and pollution.
The purpose of creating this product stems from our mission to grow more trees on our planet to combat devastating levels of carbon emissions. With every purchase of a Pass It On candle, a donation will be made to Eden Reforestation Projects to plant 10 trees.
Pass It On partners with Eden Reforestation Projects providing locals employment for their reforestation projects.
Why Eden Reforestation Projects?
Eden Reforestation Projects is a 501c3 non-profit who goes beyond planting trees and adopts the mission to provide fair wage employment to impoverished villagers as agents of global forest restoration.
Beyond planting trees, Eden Reforestation adopts the mission to provide fair wage employment to impoverished villagers as agents of global forest restoration.
"We hire the poorest of the poor to grow, plant, and guard to maturity native species forest on a massive scale. Our "employ to plant methodology" results in a multiplication of positive socio-economic and environment measures."
Scientific studies have shown that Mangroves "sequester carbon at a rate two to four times greater than mature tropical forests", and contain "the highest carbon density of all terrestrial ecosystems."
Mangrove restoration on these islands are essential to reducing the effects of flooding, tsunami's and soil erosion
Trees are extremely efficient at carbon sequestration and one of the most effective tools in the fight against dangerous Greenhouse Gases that contribute to climate change and global warming.
To keep track of the number of trees being planted, we have created a Trillion Tree account to ensure traceability for each donation. To see our tracker on Trillion Trees,
While on a trip to Australia's Great Barrier Reef, our founder, Michelle, was both astounded by its beauty amidst the destruction and saddened by the dire state of our planet's largest coral reef.
When corals get stressed, from things such as heat or pollution, they react by expelling a marine algae, leaving a ghostly, transparent skeleton behind. This is known as 'coral bleaching'. If we continue to pollute the air and the ocean with carbon emissions at our high rate, coral reefs around the world will face a catastrophic future in coming decades – in our lifetime.
A trip to Southern Great Barrier Reef in Australia, triggered a chain of emotions and actions, birthing Pass It On just 1 month after. Thankfully, areas such as Lady Elliot Island in Australia, is still very much protected and many new initiatives and sustainable resorts have been sprouting up to conserve what's left of the Great Barrier Reef. Despite the efforts of many, Michelle felt that more could and had to be done. Like what 16 year old Greta Thunberg aptly put it across
"We already have the facts and solution. All we have to do is wake up and change."
pinkypoll


text:
Le dodécaèdre adouci ou icosidodécaèdre adouci est un solide d'Archimède.

Le dodécaèdre possède 92 faces dont 12 sont des pentagones et les 80 autres sont des triangles équilatéraux. Il possède aussi 150 arêtes et 60 sommets. Il a deux formes distinctes, qui sont les images dans un miroir (ou énantiomorphes) l'une de l'autre.

Relations géométriques

Le dodécaèdre peut être engendré en prenant les douze faces pentagonales du dodécaèdre, en les tirant de telle façon qu'aucune ne se touchent, puis en leur donnant toutes une petite rotation de leurs centres (toutes en sens horaire (Sh) ou toutes en sens anti-horaire (Sah)) jusqu'à ce que l'espace entre elles puisse être rempli par des triangles équilatéraux.

Coordonnées cartésiennes
Les coordonnées cartésiennes pour les sommets d'un dodécaèdre adouci sont toutes les permutations paires de

 ,

 ,

 ,

  et

 ,
avec un nombre pair de signes plus, où

et
 ,
où  est le nombre d'or et
 est la solution réelle de , qui est le nombre

ou approximativement 1,7155615.

On notera que, parmi les 6 permutations de 3 coordonnées, les permutations paires sont les 3 permutations circulaires.

Prendre  les mêmes permutations avec un nombre impair de signes plus, donne une autre forme, l'énantiomorphe de celle-ci.

Références
 dont la seule référence était  Robert Williams, The Geometrical Foundation of Natural Structure: A Source Book of Design, 1979 .

Voir aussi

Article connexe
Polyèdre adouci

Liens externes
Dodécaèdre adouci sur MathCurve.

Polyèdre chiral
Polyèdre adouci
Polyèdre uniforme
Solide d'Archimède

redpajama_set_name: RedPajamaWikipedia


The European Investment Bank (EIB) has provided a loan worth approximately EUR 38 million to Szczecin and Świnoujście Seaports Authority to improve the seaports' nautical access and quay infrastructure.
According to the Minister of Maritime Economy and Inland Navigation, Marek Gróbarczyk, this financial support will enable the improvement of the access infrastructure to the port, safer navigation and better environmental protection.
Today's loan is a part of a larger framework financing program agreed by the EIB and the Polish government last year and amounting to some EUR 298 million.
This program is expected to support employment in the Polish seaports, and, more importantly, in the industries depending on them for efficient logistics and production. The maritime infrastructure investments are expected to catalyze further investments from the private operators and concessionaires in TEN-T seaports.
The seaports in Szczecin and Świnoujście are part of the Baltic-Adriatic core network corridor within the multimodal TEN-T Core Network. The corridor extends via Poland, the Czech Republic and Slovakia and through eastern Austria to the Slovenian port of Koper and the Italian ports of Trieste, Venice and Ravenna.
Posted on February 28, 2017 with tags EIB, loan, Poland.


text:
Q: Import 50+ tables from MySQL to SQL Server using SSIS I need too import more than 50 tables from a MySQL database. the connection details are known and I'm able to connect after creating a dsn using connection details. When I try to use the Import Export Wizard however, there are some type conversion issues i need to individually address(Eg: Destination type is shown as '22' or '13').
This is cumbersome for 50+ tables. Is there an easier alternative to this? I just need the data in string format in my MSSQL tables.

A: Also, you can use the DataConversion data flow transformer. It's easy to use and configure. You can use it to get the data (almost) in the same format/data types as in the MySQL db. Good luck!

A: I would start from the Import Export Wizard, then save the resulting SSIS Package and edit it using Visual Studio / BIDS / SSDT BI.
In the real world you often need to "individually address" specific issues, so the above steps meet that challenge quickly, with a minimal amount of coding.

redpajama_set_name: RedPajamaStackExchange


make sure you really feel free to get hold of us. We're searching for Curtain Rod 40 Inch , curtain rod 40 inch , curtain rod 30 inch , We are sincerely looking forward to creating superior cooperative relationships with clients from in the home and overseas for creating a vibrant foreseeable future together.
"So that you can supply you with comfort and enlarge our company, we also have inspectors in QC Workforce and guarantee you our greatest service and item for Curtain Rod 40 Inch , curtain rod 40 inch , curtain rod 30 inch , We always adhere to follow the honesty mutual benefit common development after years of development and the tireless efforts of all staff now has perfect export system diversified logistics solutions extensive meet customer shipping air transport international express and logistics services. Elaborate one-stop sourcing platform for our customers!


The South African wine industry is one of those new regions you might be hearing about more and more. It's ironic, because even though it's considered New World, South Africa's wine tradition is over 300 years old. The irony is not lost on young winemakers in the region, who are finding inspiration in the past. Cinsault, a grape that has risen and fallen in popularity over time, is now being embraced as the new darling varietal. With these wines coming stateside and leaving a big impression on the wine community, you should be paying attention to Cinsault.
Cinsault (or Cinsaut) originated in the south of France. Cultivated as "Hermitage" since 1850 in South Africa, it was discovered in the early 20th century to be the same as the Mediterranean grape, according to Marietta Louw, the head of the Plant Improvement Organisation, a research group owned by Bosman Family Vineyards. Cinsault used to be one of the most widely planted varieties. However, in the mid-20th century, the rising popularity of Bordeaux varieties changed Cinsault's status, and vineyards were ripped up to make way for the French grape du jour.
But winemakers encountered a big issue with their new plantings: desirable Cabernet Sauvignon was rather expensive to grow and produce. So they turned to the oldie but goodie that they had just recently torn up: Cinsault. Until 1973, wines only needed to contain 25 percent of a certain grape to be labeled single varietal, so Cinsault could be blended into the Cabernet with no one the wiser. "It filled the gap, literally," says Adam Mason, winemaker at Mulderbosch. Cinsault's large berries produced a good amount of juice, which was blended, or even co-fermented, with the star grape to increase volume and generate a higher profit.
In an unexpected plot twist, many of these older Cabs are now showing beautifully due to the high Cinsault content, and lauded for their elegance and age-worthiness.
The up-and-coming winemaking generation wasn't the only one to take notice; sommeliers and wine writers' enthusiasm over these structured and refined older vintages also raised interest in this underdog grape. With groups like In Pursuit of Balance raising the collective consciousness around balanced wine styles, Cinsault is poised to move from the chorus into a starring role.
The wine shows red berry fruits and some brambly character. It's lighter bodied and fresh, with good acidity.
Lots of red fruits and spice notes come through on the palate. Light in body and color, it's called "Cool" because the winery recommends serving chilled.
Fennel and clove spices jump out on the nose, while the palate has hints of orange peel and a long finish of raspberries.
A more intense style, but still retains the red fruits (cherries, strawberries). It's spicier, with some wild herb notes. More medium-bodied.
Very light in color; cranberries, strawberries, and a bit of spice come through on the nose. This light-bodied wine in delectably juicy and refreshing.


WISE welcomes the 2018 theme for International Women's Day, #PressforProgress. WISE especially welcomes the UN Women theme for 2018, Time is Now: Rural and Urban Activists Transforming Women's Lives, and hopes that this re-centring of the struggle on the agency and voices of rural and urban activism will lead to a reclamation of the women's rights agenda within development discourse by those who pioneered it.
WISE Development has recently undertaken a scoping study to inform investment and operational strategy for the Open Society Foundations (OSF) Economic Advancement Program (EAP). The EAP will seek to increase economic benefits for female workers, particularly those working in the informal sector, and to increase the voice of women in economic policy making.
WISE Director Georgia Taylor's work with WHO on "Midwives Voices, Midwives Realities" cited in WHO statement to UN Commission on the Status of Women 2017.
The Director General of WHO, Dr Margaret Chan, has highlighted the findings of a report commissioned by WHO and partners, co-authored by WISE Director Georgia Taylor, in her end of year message.
WHO and partners are calling for an end to the discrimination, harassment and lack of respect that hinder midwives' ability to provide quality care to women and newborns.
WISE director Dinah Bennett's recent visit to China fortunately coincided with a milestone event in the progression of women's empowerment through a unique contact sport which is a growing phenomenon globally.


text:
Spaghetti Warehouse to Close Iconic West End Location After 47 Years
Original location to close after nearly five decades
Published at 1:57 pm on October 14, 2019
NBC 5 News
After more than 45 years, an iconic Dallas restaurant is closing its doors in the West End.
The Spaghetti Warehouse said in a post on Facebook that this Sunday, Oct. 20, will be the restaurant's last day. The West End location is the original for the chain, which has only one other North Texas location in South Arlington, near the Parks Mall.
Meanwhile, the restaurant is asking fans to share their favorite West End Spaghetti Warehouse memories by leaving a comment in their post about the closure. The restaurant will celebrate two memories each week for four weeks, and these guests will receive a $50 Gift Card or a Family Feast for 4 redeemable at the Arlington restaurant.
With just a week to go before the doors close forever, there's been no mention of what will happen to the old East Dallas trolley car located in West End's main dining room -- a popular seating spot for diners over the decades.
Spaghetti Warehouse locations still remain in San Antonio; Syracuse, New York; and in Ohio in Akron, Columbus, Dayton and Toledo.


For many of us, a large portion of our daily lives is spent in the office. It is said that over a lifetime, an average person will spend over 90,000 hours at work! With all the time we spend in our place of work, it is bound to have a huge impact on your quality of life.
If you are in an office environment, this is a lot of time spent indoors! It is, thus, important to make the experience as pleasing and gratifying as possible.
One of the most simple and effective ways of doing this is through the use of plants. Not only do they improve the aesthetics of any office space, they have mental and physical benefits to the people working within.
There is growing evidence from various research on people-nature relationship that yes, exposure to nature, and plants is beneficial to health and well being. And this certainly applies to the workplace.
It is said that the use of plants, in any form, reduces workplace stress, improves concentration and enhances performance of employees.
Reduced Stress Levels. In a 2010 study conducted by the new University of Technology in Sydney, there was a significant reduction of stress among workers when plants were introduced to the workplace.
Consider these results – a 37% drop in tension and anxiety; 58% decrease in feelings of depression or dejection; a 44% decrease in anger and hostility; and a 38% reduction in fatigue!
It is said that the addition of green and flowering plants contribute largely to lifting employee spirits and promote wellbeing and performance.
Increased Productivity. Even with only a handful of plants, employees' productivity is increased by as much as 15%. In fact, it is said that adding just one plant per square meter improved memory retention.
People who see plants from their desk everyday are psychologically engaged. This leads to more happiness in the workplace which in turn, translates to increased productivity.
Healthier Work Environment. A majority of people in the US and other developed countries spend 90% of their time indoors, mostly in places where air is just re-circulated.
Increasingly, indoor air quality is becoming a concern.
While you may think that being indoors shields you from air pollution. This is, in fact, a misconception. Indoor air can be 12 times more likely to be polluted than outside air due to compounds found in clothing, furnishings, paints and building materials.
Through the natural processes of plants, they remove as much as 87% of volatile organic compounds and toxins from the air every 24 hours. Plants have the ability to purify indoor air by pulling contaminants into the soil where they are converted into plant food.
Aside from the air purifying effect of plants, they also are excellent humidifiers. Staying inside an air conditioned environment for long periods can cause dry skin, colds, sore throats and dry cough.
Plants release 97% of their water intake into the air. Put several of them in a room and this results in improved humidity. In a study conducted in Norway, sickness rates dropped by as much as 60% in offices where natural elements such as plants and foliage were incorporated.
Reduced Noise Levels. Plants have the excellent ability to absorb sounds thus reducing the distracting effects of background office chatter and sounds. This helps increase employee concentration and productivity.
It is important to carefully plan which indoor plants will thrive best in your work environment. Inside Plants is an expert in the field of indoor plantscaping.
With our office plant service, we help you design your indoor landscaping and offer advice on what plants best suit your particular situation, install them and ensure they are well established. And we help maintain your plants so you protect your investment.
Purchasing or renting plants for your office space is an investment, not only in the ambiance and making the space inviting, but in your employee health. The benefits you reap are worth much more than the dollars you put in making your space "green".
As an expert office plant service company, Inside Plants, we help you achieve a beautiful plantscape.
Comprehensive and Custom Design. Taking into consideration key factors such as your office interior design, natural lighting available and budget, we design a plantscape that suits your needs and situation. You have the option to purchase or rent suggested plants and planters.
Delivery and Installation. Once you're good with our proposed plan, we work hard to translate this blueprint to reality . Not only do we install your plants, we carefully monitor and make sure that they are well established.
Our new plants are given careful attention to ensure they will thrive in their new environment. If needed, we make changes until we are sure that your plants will flourish.
Office Plant Monthly Service. Our dedicated and highly skilled office plant service technicians will ensure that "you're not caught with your plants down".
Under our team of certified plant technicians, Inside Plants ensures that the investment you've made to make your office space a green sanctuary is well taken care of.
Make your office or commercial space a beautiful green oasis. Inside Plants provides the best indoor plants and services to make your vision into a reality.
Call us at (951)371-4637 or (760)322-9909 to get started.


You already spend your hard earned money to bring leads to your website with the intent to turn those leads into sales. For most businesses this results in converting a small percentage of them into actual sales. Wouldn't it be nice to covert more of those leads? We use technology to capture more of your leads while simultaniously turning them into higher qualified buyers. Turning your leads into qualified buyers saves you time and results in more satisfied customers. How do we do this? We use content to educate your customers in a way that feels natural to them and deliver it to them on their terms. Your services will be perceived to have a higher value to your customers and they will already feel comfortable with how your business operates.
We can review your website to see what potential leads you could be losing.


Happy Spring! The school year is winding down and some of us only have 50 days or less until summer break. We can do this!
This month we are unveiling our 2019 summer professional learning line up. We will be offering seminars on reading, writing, & math. Check them out at the bottom of this flyer, or on our website.
April is the month to celebrate poetry! Ignite your students love for reading poetry by showcasing a variety of poetry forms. Some of my favorites to share are lyrical poems(songs), narrative poems (stories), and Haikus (syllables).
A fun way to engage students in reading and writing poetry, in addition to building their fluency skills, is through Poetry SLAMS! A poetry slam is an opportunity for students to read poetry and perform a dramatic interpretation. This requires them to think deeply about the meaning of the poem and the author's craft. Check out Brod Bagert's YouTube channel for examples of expressive poetry reading.
Each day of the week you can have students focus on a different aspect of the poem that will help them gain a deeper understanding of its meaning. On Friday, students SLAM (or perform) the poem for an audience. This can be accomplished live or by recording and publishing the video on your class website or by using apps like SeeSaw or Flipgrid. To help students improve their dramatic presentation, use our fluency rubric as a guide for coaching. Slams are a fun way to develop fluency and extend comprehension.
Expert journals can be a personal or community journal in which students share their expertise and answer questions they either have themselves or that their peers have posed about that topic. Your students can make a simple expert journal by folding and tearing several sheets of paper and then stapling them together. For a more structured journal, like a community expert journal to keep in your room, you can use composition notebooks.
In my experience, students love creating these expert journals. It allows them to practice explaining information through expository text writing, and they enjoy it because they get to choose a topic that they already know a lot about, or that they want to know a lot about.
They get to write for a stated purpose and with an understanding of their intended audience.
To begin an expert journal writing activity, have students choose a topic, and then jot down what they think they already know about it, and what they need to learn or verify. Remind them that they need facts, examples, explanations, and details to engage their audience. They must also anticipate audience questions and interests.
When creating expert journals, students are faced with lots of choices and are able to make decisions about what they want to share with their audience. Choice is a motivational tool that drives learning towards autonomy.
This student decided to make a table of contents after we discussed organizing the information. Once he did, others followed his lead!
We used an Alphaboxes chart to discuss topics we know a lot about or want to know more about.
1. How many ways can you describe the number?
Give a time limit and have students write down as many different ways to describe the number as they can think of. For example, if the number is 27, they could say it has 2 tens and 7 ones, it is odd, 20+7, 30-3, etc.
Students will then compare ideas with a partner or small group. They can earn a point for every way they described the number that their partner did not.
2. The answer is______. What is the problem?
Given the number, students can write equations or word problem for the given number. For example, let's say the number is 21. Students can write 3 x 7=21 or something like "Jack had 3 boxes of cupcakes from the Magnolia Bakery. Each box had 7 cupcakes in it. How many cupcakes did he have to share with his two sons?"
3. Write a story or letter from the perspective of the number of the day. The students should determine an audience and a message. For example, if the number is 9, you might write a letter to 7 explaining what they have in common and how they are different. Be sure and read the story 7 Ate 9 by Tara Lazar, a mentor text for playing with numbers and words.
When students are developing skills or strategies, they sometimes need coaching to move them to mastery. The Kagan strategy Rally Coach can be a perfect micro-intervention for helping students achieve the daily learning targets. Students take turns demonstrating the skill and coaching, correcting, and praising each other. Pair students who need to develop the skill with students who need to reinforce (close to mastery) the skill. This takes just a few minutes and can help students identify their misconceptions quickly so that they can move forward in becoming proficient in the skill. Both of the students benefit from thinking as a coach and coach-ee, and it is a great way to deepen their understanding of the moves needed to correctly utilize the skill.
Looking for ways to motivate and transform the writers in your classroom this coming year? Join Randi Anderson for a 2 hour seminar packed with ideas for engaging students in authentic writing activities. Learn about the power of discussions and daily quick writes, learn how to use Youtube to spark interests, and so much more!


LEDs included! Full frame kit ready for your loving hands to build and then torture it. Carbon and aluminum construction for optimal strength and performance. Make them jealous.
Frame kit only - you will need to add all your own electronics (like our Build'N'Fly kit).
Room for approximately 6 standard 30.5mm boards, tab ESCs will fit. Can sub in one stack of 3 20mm mounting boards.


happy 2013. let's read through the Bible!
2013. time to read the most popular book ever!
i've read the Bible my whole life. yet i've not read the Bible in its entirety.
over on instagram i started posting pics of what i'd been doing and others wanted to join in.
download the app i have shown in the second picture below. its called Bible in One Year by Alpha International.
join me on instagram and use the hashtag #afbibleinayear (don't forget the "f") is stands for august fields Bible in a Year. you can search by that hashtag on instagram and see how it's going for others and share and encourage eachother.
i also like to doodle and draw in my journal.
i'm excited to connect with more people and work through this together. my husband is on board as well and i even set a daily reminder on my phone for 10pm. this is when we share a favorite/standout/meaningful part of our study that day.
*amazon links are affiliate but are products i have tried and love!


text:
\section{Introduction}

In the field of human-robot interaction (HRI), natural language has been used as an interface to communicate a human's intent to a robot~\cite{kollar2013generalized,howard2014natural,branavan2009reinforcement,matuszek2010following}. Much of the work in this area is related to specifying simple tasks or commands for robot manipulation, such as picking up and placing objects. As robots are increasingly used in complex scenarios and applications, it is important to develop a new generation of motion planning and robot movement techniques that can respond appropriately to diverse, attribute-based NLP instructions for HRI, e.g., instructions containing negation based phrases or references to position, velocity, and distance constraints. Furthermore, we need efficient techniques to automatically map the NLP instructions to such motion planners.

Humans frequently issue commands that include sentences with orientation-based or negation constraints such as ``put a bottle on the table and keep it upright'' or ``move the knife but don't point it towards people,'' or sentences with velocity-based constraints such as ``move slowly when you get close to a human.'' To generate robot actions and movements in response to such complex natural language instructions, we need to address two kinds of challenges:

\noindent 1. The accurate interpretation of attribute-based natural language instructions and their grounded linguistic semantics, especially considering the environment and the context. For example, a human may say ``move a little to the left'' or ``do not move like this,'' and the robot planner needs to learn the correct interpretation of these commands that include spatial and motion-based adjectives, adverbs, and negation.

\noindent 2. The motion planner needs to generate appropriate trajectories based on these complex natural language instructions. This includes appropriately setting up the motion planning problem  based on different motion constraints (e.g., orientation, velocity, smoothness, and avoidance) and computing smooth and collision-free paths.

\begin{figure}[t]
  \centering
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/stop1.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/stop2.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/stop3.png}
  }
  \caption{The Fetch robot is moving a soda can on a table based on NLP instructions. Initially, the user gives the ``pick and place" command. However, when the robot gets closer to the book, the person says \emph{``don't put it there''} (i.e. negation) and the robot uses our dynamic constraint mapping functions and optimization-based planning to avoid the book. Our approach can generate appropriate motion plans for such attributes.}
  \label{fig:stop}
\end{figure}
At a high level, natural language instructions can be decomposed into task description and attributes. Task descriptions are usually verbs or noun phrases that describe the underlying task performed by a robot. The attributes include various adjectives, adverbs, or prepositional phrases that are used to specify additional conditions the robot must (or must not) satisfy. For example, these conditions may specify some information related to the movement speed, the orientation, the physical space characteristics, or the  distances. Therefore, it is important to design motion planners that consider these robotic task descriptions and robot motion constraints.

\noindent {\bf Main Results:}  We present an algorithm for generating parameterized constraints for optimization-based motion planning from complex, attribute-based natural language instructions. We use  {\em Dynamic Grounding Graphs} (DGG) to parse and interpret the commands and to generate the constraints. Our formulation includes the latent parameters in the grounding process, allowing us to model many continuous variables in our grounding graph. Furthermore, we present a new dynamic constraint mapping that takes DGG as the input and computes different constraints and parameters for the motion planner. The appropriate motion parameters are speed, orientation, position, smoothness, repulsion, and avoidance. The final trajectory of the robot is computed using a constraint optimization solver. Overall, our approach can automatically handle complex natural language instructions corresponding to spatial and temporal adjectives, adverbs, superlative and comparative degrees, negations, etc. Compared to prior techniques, our overall approach offers the following benefits:

\begin{itemize}
\item The inclusion of latent parameters in the grounding graph allows us to model continuous variables that are used by our mapping algorithm. Our formulation computes the dynamic grounding graph based on conditional random fields.
\item We present a novel dynamic constraint mapping used to compute different parametric constraints  for  optimization-based motion planning.
\item Our grounding graphs can handle more complex, attribute-based natural language instructions, and our mapping algorithm uses appropriate cost functions as parameters over the continuous space. Compared to prior methods, our approach is much faster and better able to handle more complex and attribute-based natural language instructions.
\end{itemize}

We highlight the performance of our algorithms in a simulated environment and on a 7-DOF Fetch robot operating next to a human. Our approach can handle a rich set of natural language commands and can generate appropriate paths. These include complex commands such as  picking (e.g., \emph{``pick up a red object near you"}), correcting the motion  (e.g., \emph{``don't pick up that one"}), and negation (e.g., \emph{``don't put it on the book"}).

\section{Related Work}
Most algorithms used to map natural language instruction to robot actions tend to separate the problem into two parts: parsing and motion planning computation. In this section, we give a brief overview of prior work in these areas.

\nocite{dragan2013legibility}

\subsection{Natural Language Processing}

Duvallet et al.~\cite{duvallet2016inferring} use a probabilistic graphical learning model called Generalized Grounding Graphs (G$^3$) on a ground vehicle for a navigation problem given natural language commands. Branavan et al.~\cite{branavan2009reinforcement,branavan2012learning} use reinforcement learning to learn a mapping from natural language instructions and then apply it to sequences of executable actions. Matuszek et al.~\cite{matuszek2010following} use a statistical machine translation model to map natural language instructions to path description language, which allows a robot to navigate while following directions. Duvallet et al.~\cite{duvallet2013imitation} use imitation learning to train the model through demonstrations of humans following directions. Paul et al.~\cite{paul2016efficient} propose the Adaptive Distributed Correspondence Graph (ADCG). Arkin et al.~\cite{arkin2015towards} further extend DCG, proposing the Hierarchical Distributed Correspondence Graph (HDCG), which  defines constraints as discrete inequalities and grounds word phrases to corresponding inequalities. Chung et al.~\cite{chung2015performance} use HDCG on ground vehicles to implement navigation commands and demonstrate performance improvements over G$^3$ in terms of running time, factor evaluations, and correctness. Oh et al.~\cite{oh2016integrated} integrate HDCG with their navigating robot system, measuring performance in terms of completion rates and comparing them to human behaviors. Scalise et al.~\cite{scalise2018natural} collected a corpus of natural language instructions from online crowdsourcing that specify objects of interest for \emph{``picking up''} command. The dataset could be used as a training dataset in our method.

\subsection{Robot Motion Planning in Dynamic Environments}

Many replanning algorithms have been suggested to generate collision-free motion plans in dynamic environments. Fox et al.~\cite{fox1997dynamic} propose the dynamic window approach to compute optimal velocity in a short time window. Optimization-based motion planners~\cite{STOMP:2011,zucker2013chomp,GPUITOMP} solve a constrained optimization problem to generate smooth and collision-free robot paths. We present an automatic scheme that generates the motion planning problem from NLP instructions.

\begin{figure}[t]
  \centering
  \includegraphics[width=\linewidth]{figs/overview.png}
  \caption{
  The overall pipeline of our approach highlighting the NLP parsing module and the motion planner. Above the dashed line (from left to right): Dynamic Grounding Graphs (DGG) with latent parameters that are used to parse and interpret the natural language commands, generation of optimization-based planning formulation with appropriate constraints and parameters using our mapping algorithm. We highlight the high-level interface below the dashed line. As the environment changes or new natural language instructions are given, our approach dynamically changes the specification of the constraints for the optimization-based motion planner and generates the new motion plans.
  }
  \label{fig:pipeline}
\end{figure}

There is some work on integrating optimization-based motion planning with NLP in 2D workspaces. Silver et al.~\cite{silver2013learning} develop an algorithm for learning navigation cost functions from demonstrations. Howard et al.~\cite{howard2014natural} use a probabilistic graphical model to generate motion planning constraints for a 2D navigation problem. Compared to these methods, our approach can handle 3D workspaces and high-dimensional configuration spaces to generate robot motions corresponding to complex NLP instructions. Other techniques focus on efficiency in human-robot collaborative tasks. Markov Decision Processes (MDP) are widely used to compute the best robot action policies~\cite{nikolaidis2013human,koppula2016anticipatory}. These techniques are complementary to our approach.

\section{Dynamic Grounding Graphs}
\label{sec:dgg}
Fig.~\ref{fig:pipeline} shows the basic pipeline of our approach. When natural language commands are given as input, the NLP module (upper left) creates an optimization problem for a motion planning module (upper middle). The robot motion trajectory is then computed from the motion planning module (upper right). As the planned trajectory is executed (bottom right), the result is fed back to the NLP module. In this section, we present the algorithms used in the NLP module.

We extend the ideas of the Generalized Grounding Graphs (G$^3$) model and the Distributed Correspondence Graph (DCG) model~\cite{howard2014natural} by including the latent variables in the grounding graph and using them to compute the constraints for motion planning. The input to our algorithm is the natural language instruction. We do not account for any errors due to voice recognition. From a natural language command input, we construct a factor graph, as shown in Fig.~\ref{fig:nlp_overview}(a), which is based on the parsing of the command. For each node of the parse tree, we generate three types of nodes: word phrase node $\lambda$, grounding node $\gamma$, and correspondence node $\phi$.

The input sentence $\Lambda$ is parsed using the NLTK library~\cite{bird2006nltk}. The word phrase of each node in the parse tree is denoted as $\lambda_i$ for $i=1,2,\cdots$. Children of $\lambda_i$ are $\lambda_{i1}$, $\cdots$, $\lambda_{im}$. The root node of the parse tree is $\lambda_1$. For example, in Fig.~\ref{fig:nlp_overview}(a), the input sentence is \emph{``Put the cup on the table.''} The parse tree has the root word phrase $\lambda_1=$\emph{``Put''}. Its noun $\lambda_2=$\emph{``the cup"} and the preposition $\lambda_3=${``on,''} which are the children nodes of the root node. The noun phrase $\lambda_4=$\emph{``the table''} is the child node of $\lambda_3$. Similarly, in Fig.~\ref{fig:nlp_overview}(b), the command \emph{``Don't put it there''} is decomposed into 4 noun phrase nodes. The word phrase $\lambda_1=$\emph{``Don't''} is a negation of the verb and its child node is $\lambda_2=$\emph{``put.''} $\lambda_3=$\emph{``it''} and $\lambda_4=$\emph{``there''} are the children nodes of $\lambda_2$. Note that this parse tree is different from the parse tree in Fig.~\ref{fig:nlp_overview}(a).

Our goal is to compute a mapping from a natural language sentence $\Lambda$ to the cost function parameters $H$, given the robotic environment $E$ where the robot is operating. $E$ is a representation of the environment, which is composed of obstacle positions, orientations, and the robot's configuration. Feature vectors are constructed in the factor graph from the description of the environment. $H$ is a real-valued vector that contains all cost function parameters used in the optimization-based motion planner. It also includes the weights of different types of cost functions used in the optimization formulation. For example, the end-effector position cost function (Eq. (\ref{eq:target_cost})) requires the 3D coordinates of the target position as parameters. The repulsion cost function (Eq. (\ref{eq:repulsion_cost})) requires the repulsion source position and the constant from the exponential function.

We first compute the groundings $\gamma_i$ of each word phrase $\lambda_i$. The grounding of each word phrase is the mapping from the word phrase to its meaning in the real world. Groundings can be objects, locations, motions, tasks, or constraints. In our model, the grounding $\gamma_i$ depends on its work phrase $\lambda_i$ and its children grounding nodes $\gamma_{i1}$, $\cdots$, $\gamma_{im}$, where the tree structure of the grounding nodes follows the parse tree. Correspondence node $\phi_i$ indicates the correct matching between the word phrase $\lambda_i$ and the grounding $\gamma_i$. It is a binary variable; $\phi_i$ is $true$ if the word phrase and the grounding match and $false$ if they do not.

\subsection{Latent Parameters}

A key novel component of our approach is the inclusion of latent variables in the grounding graph. Our primary goal is to compute the best cost function parameters $H$ to be used directly for optimization-based motion planning. We denote $H \in \mathbb{R}^h$, a real vector of size $h$, as a collection of cost function parameters. In this case, the size $h$ and the number of cost function parameters depend on the types of cost functions that are used. \footnote{In this paper, we set a maximum $h=22$ to fully specify the smoothness, the end-effector position, the end-effector orientation, the end-effector speed, and the repulsion cost functions. It is a sum of three terms: $5$ for weights, $16$ for positions and orientations, and $1$ for an exponential constant.} From the predicted groundings $\gamma_i$, the cost function parameters in the motion planning formulation (Fig.~\ref{fig:nlp_overview}(b)) are inferred through the latent variable $H$. $H$ contains all the cost function parameters (e.g., weights of cost functions, locations, and orientations).

In Fig.~\ref{fig:nlp_overview}(b), the resulting constraint-based motion planning problems are shown. We use the collision avoidance cost function as the default smoothness cost function and the target location cost function, though weights can vary. The target location, whose 3D coordinates are the cost function parameters, is set on the surface of the table. The cost function parameter node $H$ contains the weights of the parameters and the 3D coordinates of the target location. In the bottom of Fig.~\ref{fig:nlp_overview}(b), where a new \emph{``Don't"} command is given, a repulsion cost function is added. Thus, the cost function weight and the location of the repulsion source (below the robot's end-effector position) are added to $H$.

\subsection{Probabilistic Model}
\label{subsec:probabilistic_model}
We present a new probabilistic model to compute $H$, $\Lambda$, and $E$. We pose the problem of finding the best cost parameters as an optimization problem:
\begin{equation}
\begin{aligned}
& \underset{\mathbf{H}}{\text{maximize}}
& & p(H | \Lambda, E).
\end{aligned}
\end{equation}
However, modeling the probability function without decomposing the variables and some assumptions about independence is difficult due to the high-dimensionality of $H$, $\Lambda$, and $E$ and the dependencies between them. To simplify the problem, the natural language sentence is decomposed into $n$ word phrases based on a parse tree, i.e.
\begin{equation}
p(H | \Lambda, E) = p(H | \lambda_1, \cdots, \lambda_n, E).
\end{equation}
Like G$^3$, we introduce the intermediate groundings $\gamma_i$ of word phrases $\lambda_i$ and correspondence variables $\phi_i$. The correspondence variables $\phi_i$ are binary random variables.
The value $1$ indicates that the word phrase $\lambda_i$ correctly corresponds to the grounding $\gamma_i$. $0$ means an incorrect correspondence.

We assume the conditional independence of the probabilities so that we can construct a factor graph (see Fig.~\ref{fig:nlp_overview}(a)). With the independence assumptions, a single factor is connected to a word phrase node and its children grounding nodes, which contain information about the sub-components. These independence assumptions simplify the problem and make it solvable by efficiently taking advantage of the tree structure of the probabilistic graphical learning model. Formally, the root grounding node $\gamma_1$ contains all the information about a robot's motion. The factor that connects $\gamma_1$ and $H$ implies that, from the root grounding node, the cost function parameters $H$ are optimized without any consideration of other nodes. Other factors connect $\gamma_i$, $\phi_i$, $\lambda_i$, children grounding nodes $\gamma_{ij}$ and the environment $E$, where the parent-child relationship is based on a parse tree constructed from the natural language sentence. This graphical representation corresponds to the following equation:
\begin{align}
p(H | \lambda_1, \cdots, \lambda_n, E)
= p(H | \gamma_1, E) \prod_i p(\gamma_i | \lambda_i, \phi_i, \gamma_{i1}, \cdots, \gamma_{im}, E).
\end{align}
For the root factor connecting $H$, $\gamma_1$ and $E$, we formulate the continuous domain of $H$. We compute the Gaussian Mixture Model (GMM) on the probability distribution $p(H | \gamma_1, E)$ and model our probability with non-root factors as follows:
\begin{align}
 & p(\gamma_i | \lambda_i, \phi_i, \gamma_{i1}, \cdots, \gamma_{im}, E) \nonumber \\
=& \frac{1}{Z} \psi_i(\gamma_i, \lambda_i, \phi_i, \gamma_{i1}, \cdots, \gamma_{im}, E) \nonumber \\
=& \frac{1}{Z} \exp(- \theta_i^T f(\gamma_i, \lambda_i, \phi_i, \gamma_{i1}, \cdots, \gamma_{im}, E)), \label{eq:one_sample}
\end{align}
where $Z$ is the normalization factor, and $\theta_i$ and $f$ are the log-linearization of the feature function. The function $f$ generates a feature vector given a grounding $\gamma_i$, a word phrase $\lambda_i$, a correspondence $\phi_i$, children groundings $\gamma_{ij}$, and the environment $E$. The information from the robotic environment is used in the feature function $f$ and in the log-linearized feature function $f$. The attributes of objects in the robotic world such as shapes and colors are encoded as multidimensional binary vectors, which indicate whether the object has a given attribute.

The probability distribution of the latent variable $H$ is modeled with $m$ pairs of Gaussian distribution parameters $\mu_i$ and $\sigma_i$ with weights $\omega_i$, as follows:
\begin{align}
p(H | \gamma_1, E) \sim \sum_{i=1}^{m} \omega_i \mathcal{N}(\mu_i, \sigma_i^2).
\label{eq:gmm}
\end{align}

\noindent \emph{\bf Word phrases.}
The feature vector includes binary-valued vectors for the word and phrase occurrences, and Part of Speech (PoS) tags. There is a list of words that could be encountered in the training dataset such as {\em \{put, pick, cup, up, there, $\cdots$ \}}. If the word phrase contains the word {\em ``put,''} then the occurrence vector at the first index is set to 1 and the others are set to 0. If the word phrase is {\em ``pick up,''} then the occurrence vector values at the second, while the fourth is set to 1 and others are set to 0. This list also includes real-valued word similarities between the word and the pre-defined seed words. The seed words are the pre-defined words that the users expect to encounter in the natural language instructions. We used Glove word2vec~\cite{pennington2014glove} to measure cosine-similarity (i.e. the inner product of two vectors divided by the lengths of the vectors) between the words. The measurement indicates that the words are similar if the similarity metric  value is near $1$, that they have opposite meanings if the similarity metric is near -1, and that they have a weak relationship if it is near 0. This provides more flexibility to our model, especially when it encounters new words that are not trained during the training phase.

\noindent \emph{\bf Robot states.}
From the robot state, we collect the robot joint angles, the velocities, the end-effector position, the end-effector velocity, etc. This information can affect the cost function parameters even while processing the same natural language commands. For example, if the robot is too close to a human under the current configuration, then the cost function for end-effector speed $C_{speed}$ or smoothness $C_{smoothness}$ will be adjusted so that the robot does not collide with the human. We also store information about the objects that are close to the robot. This information includes object type, position, orientation, shape, dimension, etc.

\subsection{Factor Graph using Conditional Random Fields}
We represent our dynamic grounding graph as a factor graph. We build a factor graph based on the probabilistic model described in Section~\ref{subsec:probabilistic_model} and use that for training and for inferring the meaning of given commands. In particular, we use Conditional Random Fields (CRF)~\cite{sutton2012introduction} as a learning model for factor graphs because CRFs are a good fit for applying machine learning to our probabilistic graph model with conditional probabilities.

During the training step of CRF, we solve the optimization problem of maximizing the probability of the samples in the training dataset over the feature coefficients $\theta_i$ and the GMM parameters $\omega_i$, $\mu_i$ and $\sigma_i$ for every parse tree structure. By multiplying Eq. (\ref{eq:one_sample}-\ref{eq:gmm}) for all training samples, the optimization problem becomes
\begin{align}
\underset{\substack{\theta_1, \cdots, \theta_n, \\ \omega_1, \cdots, \omega_m, \\ \mu_1, \cdots, \mu_m, \\ \sigma_1, \cdots, \sigma_m }}{\text{maximize}} & \prod_k p(H^{(k)} | \gamma_1^{(k)}, E^{(k)}) \label{eq:training_gmm} \\
& \prod_i \frac{1}{Z^{(k)}} \exp(\theta_i^T f(\gamma_i^{(k)}, \lambda_i^{(k)}, \phi_i^{(k)}, \gamma_{i1}^{(k)}, \cdots, \gamma_{im}^{(k)}, E^{(k)})), \label{eq:training_crf}
\end{align}
where superscripts $(k) = 1 \cdots D$ mean the indices of the training samples. The joint optimization problem Eq. (\ref{eq:training_gmm}-\ref{eq:training_crf}) of the GMM and the CRF is a hard problem. So, we separate the problem into two and solve each one separately to maximize the objective. To solve Eq. (\ref{eq:training_gmm}), the training samples of continuous variable $H$ is collected under the same conditional variable $\gamma_{1}^{(k)}$. Then, we solve the problem with the collection of $H$'s via Expectation Maximization (EM) method. Eq. (\ref{eq:training_crf}) is a tree-structured CRF problem.

At the inference step, we used the trained CRF factor graph models to find the best groundings $\Gamma$ and the cost function parameters $H$ by solving the CRF maximization problem
\begin{align}
\underset{H, \gamma_1, \cdots, \gamma_n}{\text{maximize}} & p(H | \gamma_1, E)
\prod_i \frac{1}{Z} \exp(\theta_i^T f(\gamma_i, \lambda_i, \phi_i, \gamma_{i1}, \cdots, \gamma_{im}, E)) .
\label{eq:inference}
\end{align}
When the nodes $H$, $\gamma_1, \cdots, \gamma_n$ are optimized, they create a tree structure in the factor graph, meaning that we can solve the optimization problem efficiently using dynamic programming. Each factor depends on its parent and children varying variables and other fixed variables connected to it. This implies that we can solve the sub-problems in a bottom-up manner and combine the results to solve the bigger problem corresponding to the root node.

\section{Dynamic Constraint Mapping With NLP Input}
\label{sec:planning}

\begin{figure}[t]
  \centering
  \subfloat
  {
    \includegraphics[width=0.28\linewidth]{figs/dgg1.png}
  }
  \subfloat
  {
    \includegraphics[width=0.31\linewidth]{figs/prob1.png}
  }
  \subfloat
  {
    \includegraphics[width=0.31\linewidth]{figs/robot1.png}
  }
  \addtocounter{subfigure}{-1}
  \addtocounter{subfigure}{-1}
  \addtocounter{subfigure}{-1}
  \\
  \subfloat[][]
  {
    \includegraphics[width=0.28\linewidth]{figs/dgg2.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.31\linewidth]{figs/prob2.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.31\linewidth]{figs/robot2.png}
  }
  \caption{{\bf  Factor graphs for different commands:} In the environment in the right-hand column, there is a table with a thin rectangular object on it. A robot arm is moving a cup onto the table, but we want it to avoid moving over the book when given NLP instructions.
  (a) The  command \emph{``Put the cup on the table"} is given and the factor graph is constructed (left).
  Appropriate cost functions for the task are assigned to the motion planning algorithm (middle) and used to compute the robot motion  (right).
  (b) As the robot gets close to the book, another command \emph{``Don't put it there"} is given with a new factor graph and cost functions.
 }
  \label{fig:nlp_overview}
\end{figure}

We use an optimization-based algorithm~\cite{Park:2012:ICAPS} to solve the cost minimization problem. The function and constraints of this cost minimization problem come from DGG, as explained in Sec.~\ref{sec:dgg}. In this section, we present our mapping algorithm, Dynamic Constraint Mapping, which maps the word phrase groundings to proper cost function parameters corresponding to natural language instructions.

\subsection{Robot Configurations and Motion Plans}
\label{subsec:planning}

We denote a single configuration of the robot as a vector $\mathbf{q}$, which consists of joint-angles or other degrees-of-freedom. A configuration at time $t$, where $t \in \mathbb{R}$, is denoted as $\mathbf{q}(t)$. We assume $\mathbf{q}(t)$ is twice differentiable, and its derivatives are denoted as $\mathbf{q}'(t)$ and $\mathbf{q}''(t)$. The $n$-dimensional space of configuration $\mathbf{q}$ is the configuration space $\mathcal{C}$. We represent bounding boxes of each link of the robot as $B_i$. The bounding boxes at a configuration $\mathbf{q}$ are denoted as $B_i(\mathbf{q})$.

For a planning task with a given start configuration $\mathbf{q}_0$ and derivative $\mathbf{q}'_0$, the robot's trajectory is represented by a matrix $\mathbf{Q}$, whose elements correspond to the waypoints~\cite{Zucker:IJRR:2012,STOMP:2011,zucker2013chomp,Park:2012:ICAPS}:
\begin{equation}
\mathbf{Q} = \begin{bmatrix}
    \mathbf{q}_0 & \mathbf{q}_1 & & \mathbf{q}_{n-1} & \mathbf{q}_n \\
    \mathbf{q}'_0 & \mathbf{q}'_1 & \cdots & \mathbf{q}'_{n-1} & \mathbf{q}'_n \\
    t_0 = 0 & t_1 & & t_{n-1} & t_n = T
\end{bmatrix}.
\end{equation}
The robot trajectory passes through $n+1$ waypoints $q_{0}, \cdots, q_{n}$, which will be optimized by an objective function under constraints in the motion planning formulation. Robot configuration at time $t$ is interpolated from two waypoints. Formally, for $j$ such that $t_j \leq t \leq t_{j+1}$, the configuration $\mathbf{q}(t)$ and derivative $\mathbf{q}'(t)$ are cubically interpolated using $\mathbf{q}_j$, $\mathbf{q}'_j$, $\mathbf{q}_{j+1}$, and $\mathbf{q}'_{j+1}$.

The $i$-th cost functions of the motion planner are $C_i(\mathbf{Q})$. Our motion planner solves an optimization problem with non-linear cost functions and linear joint limit constraints to generate robot trajectories for time interval $[0, T]$,
\begin{equation}
\begin{aligned}
& \underset{\mathbf{Q}}{\text{minimize}}
& & \sum_i w_i C_i(\mathbf{Q}) \\ \label{optimize}
& \text{subject to}
& &
\begin{array}{l}
  \mathbf{q}_{min} \leq \mathbf{q}(t) \leq \mathbf{q}_{max}, \\
  \mathbf{q}'_{min} \leq \mathbf{q}'(t) \leq \mathbf{q}'_{max}
\end{array}
\, \, 0 \leq \forall t \leq T.
\end{aligned}
\vspace{-6pt}
\end{equation}
In the optimization formulation, $C_i$ is the $i$-th cost function and $w_i$ is the weight of the cost function.

\subsection{Cost Functions}

The overall optimization formulation is given in Eq. (\ref{optimize}). To formulate the constraints, we use the following cost functions, which are designed to account for various attributes in the NLP instructions.
In our formulation, we use many types of cost functions such as collision avoidance, robot smoothness, robot end-effector speed, target positions, and target orientations. These cost functions are used to handle many attributes of natural language instructions. Each cost function has its weight and may also have other cost function parameters, if necessary. For example, the robot end-effector speed cost function has parameters corresponding to the direction and the magnitude of the speed, which impose a constraint on the final computed trajectory. If the weight of the end-effector speed cost function is higher than the others, then it contributes more to the overall objective function in the optimization formulation. If the weight is low, then the end-effector speed cost will be compromised and has a lesser impact on the path planner.

The cost functions $C_i$ and the latent parameter $H$ are closely related. $H$ is a collection of parameters that describe all types of $C_i$ and the weights $w_i$. The cost function parameters of $C_i$ and the weights $w_i$ are all real-valued. Those real values are appended to construct the real-valued vector $H$.

\subsection{Parameterized Constraints}

To handle various attributes, we use the following parameterized constraints in our optimization formulation.

\noindent \emph{\bf Collision avoidance:} By default, the robot should always avoid obstacles.
\begin{align}
C_{collision}(\mathbf{Q}) = \int_0^T \sum_i \sum_j \textit{dist}(B_i(t), O_j)^2 dt,
\end{align}
where $\textit{dist}(B_i(t), O_j)$ is the penetration depth between a robot bounding box $B_i(t)$ and an obstacle $O_j$.

\noindent \emph{\bf Smoothness:} We penalize the magnitude of a robot's joint angle speed to make the  trajectory smooth. This corresponds to the integral of the first derivative of joint angles over the trajectory duration, as follows:
\begin{align}
C_{smoothness}(\mathbf{Q}) = \int_0^T \sum_i \mathbf{q}'(t)_i^2 dt.
\end{align}
This function is useful when we need to control the speed of the robot. When the robot should operate at a low speed (e.g. when a human is too close), or we don't want abrupt movements (e.g., for human safety), the smoothness cost can have high weights so that the robot moves slowly without jerky motions.

\noindent \emph{\bf End-effector position:} A user usually specifies the robot's target position to make sure that the robot reaches its goal. This cost function penalizes the squared distance between the robot's end-effector position and the target position over the trajectory duration as
\begin{align}
C_{position}(\mathbf{Q}) = \int_0^T || \mathbf{p}_{ee}(t) - \mathbf{p}_{target} ||^2 dt, \label{eq:target_cost},
\end{align}
where $\mathbf{p}_{ee}(t)$ is the robot end-effector position at time $t$ and $\mathbf{p}_{target}$ is the target position.
The target position $\mathbf{p}_{target}$ is considered as a cost function parameter. In the mapping algorithm, a position grounding node encodes the target position parameter. This parameter can be a 3D position or the current object position in the environment. Typically, the target position is specified by an object name in the sentence, such as \emph{``pick up the cup''} or \emph{``move to the box.''} In these cases, the grounding nodes for \emph{``the cup"} and \emph{``the box''} are interpreted as the current 3D coordinates of the target positions, which are the parameters of this cost function.

\noindent \emph{\bf End-effector orientation:} Robotic manipulation tasks are sometimes constrained by the end-effector orientation. This cost function penalizes the squared angular differences between the end-effector orientation and the target orientation over the trajectory duration.
\begin{align}
C_{orientation}(\mathbf{Q}) = \int_0^T \textit{angledist}(\mathbf{q}_{ee}(t), \mathbf{q}_{target})^2 dt \\
C_{upvector}(\mathbf{Q}) = \int_0^T \textit{angledist}(\mathbf{n}_{up}(t), \mathbf{n}_{target})^2 dt,
\end{align}
where $\mathbf{q}_{ee}(t)$ is the quaternion representation of the robot end-effector's orientation at time $t$, $\mathbf{q}_{target}$ is the end-effector orientation that we want the robot to maintain, $\mathbf{n}_{up}$ is the normal up-vector of the robot's end-effector, and $\mathbf{n}_{target}$ is the target up-vector. As with the \emph{end-effector position} cost, the target orientation $\mathbf{q}_{target}$ is the cost function parameter. The target orientation usually depends on the object the robot picked up. For example, when the robot is doing a peg-hole insertion task under the command \emph{``insert that into the hole,''} the orientation of the robot's end-effector $\mathbf{q}_{ee}$ should be constrained near the hole. If the robot arm is holding a cup of water, it should be upright so it does not spill the water. In this case, $\mathbf{n}_{target}$ is set to $(0, 0, 1)$.

\noindent \emph{\bf End-effector speed:} This cost function penalizes the robot's end-effector speed and direction:
\begin{align}
C_{speed}(\mathbf{Q}) = \int_0^T || \mathbf{v}_{ee}(t) - \mathbf{v}_{target} ||^2 dt,
\end{align}
where $\mathbf{v}_{ee}(t)$ is the robot's end-effector speed at time $t$, and $\mathbf{v}_{target}$ is the target speed. The parameters of this cost function correspond to $\mathbf{v}_{target}$. In some cases, we must restrict the robot's end-effector velocity, e.g., if a user wants to pick up a cup filled with water and doesn't want to spill it. Spilling can be prevented by limiting the end-effector speed, making the robot move more slowly.

\noindent \emph{\bf Repulsion:} The repulsion functions are commonly represented as potential fields
\begin{align}
C_{repulsion}(\mathbf{Q}) = \int_0^T \exp \left( - c || \mathbf{p}_{ee}(t) - \mathbf{p}_{r} || \right) dt, \label{eq:repulsion_cost}
\end{align}
where $\mathbf{p}_{r}$ is the position to which we don't want the robot to move. The coefficient $c > 0$ suggests how much the cost is affected by $||\mathbf{p}_{ee}(t) - \mathbf{p}_{repulsive}||$, the distance between the end-effector position and the repulsion source. The cost function is maximized when the end-effector position is exactly at the repulsion source, and it decreases as the distance between the end-effector and the repulsion position increases.  For example, if the command is \emph{``Don't put the cup on the laptop,''} we can define a repulsion cost with the laptop position as the repulsion source. The cost function is inversely proportional to the distance between the end-effector and the laptop.

\section{Implementation and Results}

We have implemented our algorithm and evaluated its performance in a simulated environment and on a 7-DOF Fetch robot. All the timings are generated on a multi-core PC.with Intel i7-4790 8-core 3.60GHz CPU and a 16GB RAM. We use multiple cores for fast evaluation and parallel trajectory search to compute a good solution to the constrained optimization problem~\cite{Park:2012:ICAPS}.

\subsection{Training DGGs for Demonstrations}

We describe how the training dataset for our DGG model was generated. The training dataset for DGGs requires three components: a natural language sentence, a robotic environment, and the cost function parameters for optimization-based motion planners.

For each demonstration, we write tens of different sentences that specify the take goals the constraints for the motion plans with different nouns, pronouns, adjectives, verbs, adverbs, preposition, etc. For each sentence, we generate a random robotic environment and an initial state for the robot. In addition, the robot joint values and joint velocities are randomly set as initial states. We collect tens or hundreds of random robotic environments. For a natural language sentence, a random robotic environment, and a random initial state for the robot, the cost function parameters are assigned manually or synthesized from other examples. Crowdsourcing such as Amazon Mechanical Turk can be alternatively used to assign cost function parameters. Hundreds of multiple data samples are generated from generated data samples by switching the correspondence variable in the DGG model from $1$ (true) to $0$ (false) and changing the grounding variables to the wrong ones to match the false correspondence variable. The training dataset is created with up to $100,000$ samples in our experiments. When the cost function parameters are determined, the optimization-based motion planner is used to compute a feasible robot trajectory. In the optimization-based motion planning algorithm, there are some waypoints through which the robot trajectory should pass. For the robot's safety, we check if the robot trajectory with the given cost function parameters is in-collision and appropriately set a higher value of the coefficient of the collision cost and compute a new trajectory. This process is repeated until the trajectory is collision-free. The training step took up to an hour with $100,000$ training samples in our experimental settings, though the training time can vary depending on the complexity of tasks, environments, and natural language instructions.

We use different training data for each scenario. For the scenarios shown in Fig.~\ref{fig:sim_1}, the initial pose of the robot in front of the table and the positions of the blue and red objects on the table are randomly set. For \textit{``Pick up''} commands, appropriate cost function parameters are computed so that the robot picks up a blue or red object depending on the given command. Similarly, in Fig.~\ref{fig:sim_2}, the position and orientation of the laptop is initialized randomly. Given the \textit{``Put''} command, we create an end-effector position cost function so that the robot places the object on the table; and a repulsive cost function to avoid the laptop position.

\subsection{Simulations and Real Robot Demonstrations}

\begin{figure}[t]
  \centering
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/pick1.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/pick2.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/pick3.png}
  }
  \caption{The simulated Fetch robot arm reaches towards one of the two red objects.
  (a) When a command \emph{``pick up one of the red objects''} is issued, the robot moves to the red object on the right because of the DGG algorithm.
  (b) If the user doesn't want the robot to pick up the object on the right, he/she uses a command \emph{``don't pick up that one.''} Our DGG algorithm dynamically changes the cost function parameters.
  (c) The robot approaches the object on the right and stops.
  }
  \label{fig:sim_1}
  \vspace*{-0.05in}
\end{figure}

\begin{figure}[t]
  \centering
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/laptop1.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/laptop2.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/laptop3.png}
  }
  \caption{In this simulated environment, the human instructs the robot to \emph{``put the cube on the table''} (a). As it approaches the laptop (b), the human uses a negation NLP command \emph{``don't put it there,''} so the robot places it at a different location (c).}
  \label{fig:sim_2}
\end{figure}

\begin{figure}[ht]
  \centering
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/sim1.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/sim2.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/sim3.png}
  }
  \caption{A 7-DOF Fetch robot is operating in a simulated environment avoiding an obstacle.
  (a) In a traditional optimization-based motion planner, the planner gets stuck at a local minimum.
  (b),(c) Using natural language commands as guidance, the user guides the robot out of the minimum and towards the goal position.}
  \label{fig:sim_obstacle}
\end{figure}

We evaluate the performance on optimization problems that occur in complex environments composed of multiple objects. Based on the NLP commands, the robot decides to pick an appropriate object or is steered towards the goal position in a complex scene.  In particular, the user gives NLP commands such as \emph{``move right,''} \emph{``move up,''} \emph{``move left,''} or \emph{``move down''} to guide the robot. For each such command, we compute the appropriate cost functions.

\begin{figure}[t]
  \centering
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/dont1.jpg}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/dont2.jpg}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/dont3.jpg}
  }
  \caption{The Fetch robot is taking real-time commands from the human and moves the soda can on the table.}
  \label{fig:dont}
\end{figure}

\begin{figure}[t]
  \centering
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/stop1.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/stop2.png}
  }
  \subfloat[][]
  {
    \includegraphics[width=0.32\linewidth]{figs/stop3.png}
  }
  \caption{The Fetch robot is moving a soda can on a table. Initially, the user gives the ``pick and place'' command. However, when the robot gets closer to the book, the person says \emph{``don't put it there''} and the robot avoids the book using appropriate cost functions and optimization.}
  \label{fig:stop}
\end{figure}

We also integrate our NLP-based planner with ROS and evaluated its performance on the 7-DOF Fetch robot.
In a real-world setting, we test its performance  on different tasks corresponding to: (1)  moving a soda can on the table from one position to another; (2) not moving the soda can over the  book. With a noisy point cloud sensor on the robot, the thin book is not recognized as a separate obstacle by the robot, though the human user wants the robot to avoid it. All the instructions used in these tasks have different attributes, which makes it hard for prior methods. In Fig.~\ref{fig:dont}, the two sub-tasks are specified in one sentence at the beginning, as \emph{``move the can on the table, but don't put it on the book"}. The cost function is used to move the robot's end-effector to the surface of the table. Another cost function penalizes the distance between the book and the end-effector. In Fig.~\ref{fig:stop}, only the first sub-task is given at the beginning. This results in the robot moving the can on the book. As the robot gets too close to the book, the person says \emph{``stop,''} then says \emph{``don't put it there.''} The robot recomputes the cost functions and avoids the region around the book.

\subsection{Analysis}
We evaluate the performance based on the following metrics:

\noindent \emph{\bf Success Rate:} The ratio of successful task completion among all trials. Failure includes colliding with the obstacles due to an incorrect mapping of cost function parameters, violating constraints specified by natural language commands, and not completing the task due to some other reason.

\noindent \emph{\bf Trajectory Duration:} The time between the giving of the first NLP command and the robot's successful completion of the task after trajectory computation. A shorter duration implies a higher performance.

\noindent \emph{\bf Trajectory Smoothness Cost:} A cost based on evaluating the trajectory smoothness according to standard metrics and dividing it by the trajectory duration. A lower cost implies a smoother and more stable robot trajectory.

{\small
\begin{table}[t]
\caption{Planning performances with varying sizes of training data for the  scenario in Fig.~\ref{fig:sim_1} with 21 different NLP instructions.
}
\vspace*{-0.12in}
\centering
\scalebox{0.9}{
\begin{tabular}{|c|c|c|c|}
\hline
\# Training Data & Success Rate &  Duration &  Smoothness Cost \\ \hline
  1,000 &  5/10 & 23.46s (5.86s) & 8.72 (5.56) \\ \hline
  3,000 &  9/10 & 16.02s (3.28s) & 2.56 (0.64) \\ \hline
 10,000 & 10/10 & 13.16s (1.24s) & 1.21 (0.32) \\ \hline
 30,000 & 10/10 & 12.81s (0.99s) & 0.78 (0.12) \\ \hline
100,000 & 10/10 & 12.57s (0.97s) & 0.72 (0.10) \\ \hline
\end{tabular}
}
\label{table:training_data}
\end{table}
}
{\small
\begin{table}[t]
\caption{Running time (ms) of our DGG and motion planning modules for each scenario.
}
\vspace*{-0.12in}
\centering
\scalebox{0.9}{
\begin{tabular}{|c|c|c|c|c|}
\hline
Scenarios & Instructions & $|H|$ & \begin{tabular}[x]{@{}c@{}}DGG\\Time\end{tabular} & \begin{tabular}[x]{@{}c@{}}Planning\\Time\end{tabular} \\ \hline
Pick up an object (Fig. 4) & 10 & 12 & 32ms & 93ms \\ \hline
Don't put on the laptop (Fig. 5) & 20 & 13 & 16ms & 98ms \\ \hline
Move around obstacle & 45 & 9 & 16ms & 95ms \\ \hline
Static Instructions (video) & 20 & 18 & 73ms & 482ms \\ \hline
Dynamic Instructions (Fig. 1) & 21 & 18 & 58ms & 427ms \\ \hline
\end{tabular}
}
\label{table:time}
\vspace*{-0.2in}
\end{table}
}
Table~\ref{table:training_data} shows the results on our benchmarks with varying numbers of training data samples in the simulation environment shown in Fig.~\ref{fig:sim_1}. When the number of training data samples increases, the success rate also increases while the trajectory duration and the trajectory smoothness cost decrease. Table~\ref{table:time} shows the running time of our algorithm and the distances from the obstacle on the table in the real-world scenarios. We use 8 parallel threads for parallel trajectory search in the motion planning module.

Table~\ref{table:appendix_dgg} and~\ref{table:appendix_trajectory} show the examples of the dataset. In Table~\ref{table:appendix_dgg}, DGGs with word phrase nodes, grounding nodes and correspondence variables are shown. The correspondence variables of the graphs on the left column are all \emph{true}, and the groundings are matched correctly. Whereas, the correspondence variables on the right column are mixed with \emph{true} and \emph{false}. The groundings are not matched if the correspondence variable is \emph{false}. Many data samples are generated by flipping the correspondence variables between \emph{true} and \emph{false} to increase the accuracy of the DGG inference step. In Table~\ref{table:appendix_trajectory}, the latent variables are shown for the examples of the grounding graphs and the environment. The cost function weights and other necessary cost function parameters are manually set in the data generation program.

\section{ Benefits and Comparisons}
Most prior methods that combine NLP and motion planning have focused on understanding natural language instructions to compute robot motion for simple environments and constraints. Most of these methods are limited to navigation applications~\cite{oh2016integrated,chung2015performance,duvallet2016inferring} or simple settings~\cite{branavan2012learning}, or they are not evaluated on real robots~\cite{arkin2015towards}. Nyga et al.~\cite{nyga2012everything,nyga2018cloud,nyga2017instruction,nyga2017no} use probabilistic relation models based on knowledge bases to understand natural language commands that describe visual attributes of objects. This is complementary to our work. Broad et al.~\cite{broad2017real} extedd DCG for a robot manipulator so that it will handle natural language correction for robot motion in realtime. In our approach, the goal is to generate appropriate high-DOF motion trajectories in response to attribute-based natural language instructions like negation, distance or orientation constraints, etc. Unlike prior methods, the output of our NLP parsing algorithm is directly coupled with the specification of the motion planning problem as a constrained optimization method.

It may be possible to extend prior methods~\cite{kollar2013generalized,howard2014natural} to handle attribute-based NLP instructions. For example, distance attributes require a number of constraints in the motion planning formulation. In natural language instructions such as \emph{``Pick up the blue block and put it 20 cm to the left of the red block''} or \emph{``Pick up one of the two blocks on the rightmost, and place it 10 inches away from the block on the leftmost,''} the exact distance specifications are the distance attributes. Prior methods that use G$^3$, DCG, or the Hybrid G$^3$-DCG models have only been evaluated with a small number of attributes (distance, orientation, and contact) to solve constrained motion planning problems. These prior techniques use discretized constraints~\cite{howard2014natural}, each of which can be active (i.e. $f(x) > 0)$), inverted ($f(x) < 0$), or ignored (i.e. not included). Therefore,  it is not possible to represent an explicit  constraint corresponding to the value of the  continuous variable \emph{distance}  in their formulation.

\begin{comment}
\section{ Benefits and Comparisons}
In this section, we compare our approach with prior methods and highlight the benefits in terms of handling attribute-based NLP instructions. We evaluated the end-to-end performance of our approach including the NLP, mapping and the motion planning modules on different benchmarks. Most prior methods that combine NLP and motion planning have focused on understanding natural language instructions to compute robot motion for simple environments and constraints. Most of these methods are limited to navigation applications~\cite{oh2016integrated,chung2015performance,duvallet2016inferring} or used in simple settings~\cite{branavan2012learning} or not evaluated on real robots~\cite{arkin2015towards}. In our approach, the goal is to generate appropriate high-DOF motion trajectories in response to attribute-based natural language instructions like negation, distance or orientation constraints, etc. Unlike prior methods, the output of our NLP parsing algorithm is directly coupled with the specification of the motion planning problem as a constrained optimization method.

Our algorithm demonstrates many advantages over Howard et al.~\cite{howard2014natural}.
They use a discrete set of constraints and their planning algorithm determines whether those constraints are activated or not. In the worst case, the complexity of their search space can grow exponentially as more constraints are added. On the other hand, we use appropriate cost functions  as parameters over the continuous domain. Our constraint setup algorithm based on dynamic constraint mapping is different and our formulation can easily handle complex dynamic environments. As a result, we can handle complex instructions with attributes in real-time (see Table II).

It may be possible to extend prior methods~\cite{kollar2013generalized,howard2014natural} to handle attribute-based NLP instructions.
For example, distance attributes require a number of constraints in the motion planning formulation.
Lets consider natural language instructions such as: \emph{``Pick up the blue block and put it 20 cm to the left of the red block''} or \emph{``Pick up one of the two blocks on the rightmost, and place it 10 inches away from the block on the leftmost,''} where the exact distance specifications are the distance attributes.
\footnote{These are based on NLP instructions for manipulation tasks available at \url{https://github.com/tmhoward/h2sl}. We added distance attributes to the original instructions to evaluate the performance.}
Prior methods that use G$^3$, DCG and the Hybrid G$^3$-DCG models have only been evaluated with a small number of attributes (distance, orientation, and contact) to solve constrained motion planning problems. These prior techniques use discretized constraints~\cite{howard2014natural}, each of which can be active (i.e. $f(x) > 0)$), inverted ($f(x) < 0$)  or ignored (i.e. not included). Therefore,  it is not possible to exactly represent an explicit  constraint corresponding to the value of the  continuous variable \emph{distance}  in their formulation. One workaround is to discretize the continuous \emph{distance} and to create multiple constraints for each discretized values. However, a large number of constraints increases the runtime overhead of their planner. However, our model can represent this \emph{distance} attribute precisely with only one constraint using the latent parameter $H$ and the corresponding cost function.

Moreover, we highlight the experimental results with a 7-DOF Fetch robot and demonstrate that our approach can handle the uncertainties that arise from real robot integration, which include speech recognition, visual sensor errors, and control errors.
In the real robot experiment, the NLP parsing algorithm is successfully integrated with the realtime motion planning algorithm to set up proper parameters in the formulation for our optimization-based motion planning.
\end{comment}
\section{Limitations, Conclusions and Future Work}
We present a motion planning algorithm that computes appropriate motion trajectories for a robot based on complex NLP instructions. Our formulation is based on two novel concepts: dynamic grounding graphs and dynamic constraint mapping. We highlight the performance in simulated and real-world scenes with a 7-DOF manipulator operating next to humans. We use a high dimensional optimization algorithm and the solver may get stuck in local minima, though we use multiple initializations to solve this problem. Furthermore, the accuracy of the mapping algorithm varies as a function of the training data.

As future work, we would like to overcome these limitations and evaluate the approach in challenging scenarios with moving obstacles while performing complex robot tasks. More work is needed to handle the full diversity of a natural language, especially for rare words, complicated grammar styles, and hidden intentions or emotions in human speech. We plan to incorporate stronger natural language processing and machine learning methods such as those based on semantic parsing, neural sequence-to-sequence models, etc. We also plan to collect more natural language data from a variety of sources such as recipes or demonstration videos.

redpajama_set_name: RedPajamaArXiv


In fall 2012, a subcommittee of Des Moines Council directed staff to draft a bylaw banning front yard fruit and vegetable gardens.
There is hope, as some Councillors consider banning front yard vegetable gardens to be an infringement on property rights…which is, of course, absolutely true. The right to express yourself through your gardening is guaranteed under the US Constitution. This right has already been affirmed by the Courts in cases against native plant gardens. Native plant gardens are an essential adjunct to food gardens as they provide a range of habitat and nourishment for pollinators throughout the growing season.