WildVine Jazz is made up of Tim Wildman on piano and Jock Irvine on double-bass. They perform classic and contemporary jazz standards that are easy to listen to and will get your toes tapping at the same time. Never too loud, WildVine Jazz is an excellent compliment to your restaurant or private function. The group can perform in a variety of formats from a duo, to a five piece band featuring vocals and horns.
Nothing compares with live music, and WildVine Jazz is an affordable way to add musicians to your special function for memories that will last a lifetime. Please contact us for more information.
We just finished our new Christmas CD, all proceeds will of which will go to the Frank L. Irvine Fund for Justice and Nonviolence. This fund, which was established by South Congregational Church, Concord NH, and is administered by the NH Conference of the United Church of Christ, will be used to assist the conference in bringing speakers and preachers who are recognized advocates for justice and Nonviolence to conference-wide gatherings.
Click on the video, above, to hear a two-minute sample.


Belinda was amazing! Always helpful and cheerful to lend a hand. I look forward to continue using you all and Belinda again in the future!
Belinda made my first time home buying experience a pleasure! I highly recommend Belinda and GMFS Mortgage. Belinda was very professional, patient, thorough, and nice!
I want to Thank Belinda Reine for being so patient with our process! She was very pleasant to work with. When we were have some problems/set backs she called and checked on us, and reassured us that everything was going to work out and she was there if we needed her.
Belinda Reine was so very helpful. She was very professional. Belinda made us feel like we were a part of her family. I would recommend her to everyone. Our experience was the best we have ever had with a mortgage company. Thank you Belinda and GMFS Mortgage for making us feel so comfortable.


text:
18 Apr Q&A WITH RYAN MATHER: PROSPECTIVE TOWN FOUNDATION DEGREE STUDENT
In March 2020, the Huddersfield Town Foundation announced the launch of its Community Football Coaching and Development Foundation Degree.
The two-year programme, which is in conjunction with the EFL Trust and University of South Wales, will see students gain an industry recognised coaching qualification.
On completion of the course, participants will have the opportunity to add a third year of study and upgrade the qualification to a BSC in Football Coaching, Development and Administration.
Set to start the course in September after receiving an unconditional offer to the programme is 24-year-old Ryan Mather.
Ryan is a lifelong Huddersfield Town supporter and founder of the LGBT supporters group, Proud Terriers, which was established in 2016.
Here he explains more about why he applied for the course:
Can you tell us why you have applied to study for the Town Foundation's Degree in Community Football Coaching and Development?
I absolutely love football. It's the first thing I think about when I wake up in the morning and it's the last thing I think about when I go to sleep. This is the perfect degree for me to make my dream of becoming a coach come true.
I also feel totally honoured to be studying with the Town Foundation. They do some truly inspiring work.
What are your long-term career aspirations?
Ultimately, I would love to work for a professional football club as the main coach. It's a big dream, but I believe anything is possible if you work hard!
What do you think will be most challenging about the course?
I think the most challenging thing will be the difficulty of the assignments. However, I love a challenge and I am prepared to put in plenty of hard work to achieve the best results possible.
The Town Foundation looks forward to welcoming Ryan to the programme and helping him to achieve his ambition and build upon his prior involvement with the Club.
Proud Terriers was initially founded as a Twitter page and has gone from strength to strength with the support of Town's Supporter Services Team.
In speaking of the Proud Terriers, Ryan commented:
"Being an openly gay football fan myself, I know first-hand how tough and intimidating it can be on the terraces. I wanted to create a safe space for Huddersfield Town fans who are LGBT to feel welcome at the game, but the group is accepting of everybody who wishes to support us, regardless of sexuality".
More information about Proud Terriers can be found on their Twitter page: @ProudTerriers.
If you would like to learn more about or register your interest in the Community Football Coaching and Development Foundation Degree, please contact Head of Education and Sport, Katie Suttron via katie.suttron@htafcfoundation.com or 07947 564331.

redpajama_set_name: RedPajamaCommonCrawl


The concept of bitcoin as a payment method for online gambling is seeing a meteoric rise.
Gambling using bitcoin at bitcoin-friendly online casinos offers multiple benefits, including lower house edges than traditional online casinos (1% to 2% lower, because of lower operating costs from payment processors), faster deposits and withdrawals, and the option to deposit, withdraw, and gamble anonymously.
How did they come to be, and how did they evolve into a stellar payment method for real money online gambling?
In its purest form, bitcoin is a fully decentralized virtual form of currency. This means that it does not exist in the physical world (like real cash in a bank does). This also means that it does not run through a central banking system, such as the United States Federal Reserve.
Bitcoin is used by over 20 million individuals and business of all sizes all over the world. Bitcoin transactions, sales, and purchases have dollar amounts well into the hundreds of millions every day, all without the need for a middleman, bank, or credit card company to get involved in the transactional process.
Situated firmly amongst these individuals and business are bitcoin-accepting online casinos, each offering its own unique array of betting games, from blackjack to slots to roulette and beyond.
There are casinos online that specifically do business solely in bitcoin. These casinos have a considerably lowered house edge, sometimes by up to 2%. Many bitcoin casinos also go by the concept of provable fairness, using algorithms offering transparency, and showing that there is no way for the website to game their players. This process stretches also to live bitcoin casino gaming as well.
It should be standard though, that when searching for a top-notch bitcoin casino, you do your due diligence and check the casino's historical reputation. There are hundreds of bitcoin casinos across the internet, all with their own special set of offerings.
3 of the highest rated bitcoin casinos are Ignition, Bovada, and Slots.lv.
Ignition is a bitcoin casino that is relatively new to the scene, and it offers the fastest "cage cashier" transactional payment process around. In general, the Ignition casino offers a safer and more anonymous gambling process, with an overall procedural audit performed by the Kahnawake Gaming Commission. This ensures a safe and fair game when you start playing for real bitcoin.
Try playin through Bovada using their simplistic and minimalist transactional process when placing a withdrawal or committing to a deposit. The site, developed and launched in 2007, has quickly grown into one of the largest and most widely-used online casino currently accepting bitcoin payments and transactions. Bovada is offering a $3,000 maximum welcome bonus for new players to their service.
Slots.lv offers a wide selection of table games and virtual slot machines from the top gaming software providers available. A popular Vegas-style online casino, Slots.lv has recently begun accepting bitcoin and other cryptocurrencies as payment methods. Players are offered a wide array of rewarding, creative bonus promotions, and all transactions are not only anonymous through bitcoin but also secure with SSL.
Other higher-quality options for bitcoin casinos, in addition to the 3 listed above, are Lincoln Casino and Sloto Cash.
Lincoln Casino offers a $1,500 bonus to new players, and has an 8-day average turnaround time for payouts via bitcoin.
Sloto Cash offers a $500 bonus to new players, and also has an 8-day average payout time for bitcoin payments.
Some online bitcoin casinos are unsafe and unfair.
Balzac Casino has consistently engaged in dishonest casino practices. They offer no information regarding their licensing on their site.
Grand Reef has loose gameplay when in free-play mode, but tightens up when you play for real money. They also have slow payout times for bitcoin withdrawals.
Euro Play Casino has many complaints regarding a lack of timely customer support and unresolved issue. They've been reported numerous times for unprofessional business practices.
Avoid these options, regardless of how enticing their bonuses might be.
Our in-house team inspects online casinos to ensure that we only recommend the best bitcoin online casinos.
Variety of Games: Top bitcoin online casinos will generally support hundreds of various betting games, table games, card games, dice games, and slot machines. This ensures that players will always have an ideal game for their playing style and preferences.
Quick Player Payouts: Technological advancements are such that it should not be an issue for online casinos to release winnings in 2 or 3 days. This requires strong organization and a carefully developed payment process.
Security and Safety: All top bitcoin casinos must use the strongest encryption levels available an the most developed and latest online security parameters to protect players' personal data and finances.
Promotions and Bonuses: These top-rated bitcoin online casinos need to provide a high value for the loyalty of their players. Whether that comes in the form of welcome bonuses, weekly offers, monthly promotions, or random giveaways, there needs to be incentive for bitcoin users to keep playing.
Options for Banking: Besides the security of players' personal data, it's important that bitcoin casinos ensure that money transmits securely both to and from the casino and that there remains a variety of payment methods and transmission preferences to fit any budget and preference.
Strong Customer Service: One of the most important aspects of a solid bitcoin offering online casino is a wide variety of options for players to contact customer support. Whether this comes from phone chat, email correspondence, ticket tracking, live chat or smoke signals, players need the assurance that their questions, comments, and concerns can be addressed quickly and effectively.
Most bitcoin wallets require a user to have the login information for your wallet to access. Additionally, depending on the wallet type, they may need access to your physical machine to retrieve funds (this may be a computer or phone). These factors make hacking into a bitcoin wallet far more difficult than other payment methods such as PayPal or a regular bank account.
How quickly do deposits and withdrawals happen with bitcoin?
Unlike traditional banking methods that can be used at most online and physical casinos, both payments and withdrawals at bitcoin-offering casinos are made instantaneously and completely anonymously across the block chain.
How do bitcoin casinos compare to regular online casinos? Are there downsides?
For those interested in immediate transactions, lower edges for the house, and the privacy offered by bitcoin, traditional banking methods don't hold a candle to bitcoin casino offerings. The only true downside to gambling using bitcoin is that the value of the bitcoin can fluctuate wildly, even as quickly as from hour to hour. If the value of a bitcoin drops significantly while you have bitcoin invested in an online casino, your withdrawal value will drop in parallel.
What fees are involved in online bitcoin casino transactions?
Bitcoins can be bought and sent peer-to-peer across the Internet without going through a bank or clearing house. The best online Bitcoin casinos have fees that are therefore relatively low. The current market price for a Bitcoin is always changing due to the supply and demand for it.


Catherine Lévy, who created Dorette, was born in Paris where she has lived and worked ever since. At a very young age, she would make jewellery from what was close at hand: chick peas from the kitchen, pebbles extracted from the gravel, rabbit poo she'd collect from the mountains, or snail shells (that also came from the kitchen). These tendencies naturally directed her to Les Ateliers (l'Ecole nationale supérieure de la creation industrielle), where she studied design. She then joined forces with Sigolène Prébois to create objects for homes, signed Tsé & Tsé associées. Their vases, garlands, plates and bowls have been successful all over the world for twenty years. Dorette jewellery never weighs heavily on fingers, never scratches necks or pulls on ears… The multi-coloured stones, both modest and precious, brut diamonds and glass set in gold and silver, diffuse a singular murmur and are always cheerful company. Each one is different, neither ancient nor modern, they are light objects that inspire pleasure, wit and affection.


text:
Q: What is the difference between shelve and submit on perforce? I am using recently perforce due to the company I am working with is using perforce.
The only SCM I have worked with was Git and only for 3 months (as I am junior) and I know the basic commands.
I find now that P4 has shelve and submit (I just knew push on git) so as far as I understood from the docs, shelve would be the equivalent to git stash and submit the equivalent to push?

A: The git analogies are not perfect because Perforce is a client-server system and git is a distributed system.
shelve takes your changes and stores them on the server, but they are still in a separate changelist and are not permanent. A shelved change can still be further modified, or even deleted, or it can be made permanent by submitting the shelf.
submit takes your changes and stores them on the server, making them permanent and visible to others; your changes will be received by other users of these files the next time they sync their workspace from the server.

redpajama_set_name: RedPajamaStackExchange


text:
Q: Platform Specific Service Causes an Exception - MvvmCross, Xamarin.iOS I am developing an application in which I need to use platform specific services. I referred so many stuff in the internet and couldn't find a way to get this to work. Even the MvvmCross documentation has not been updated I guess.
My platform specific service is to pop up dialogs in each platform in their native appearances.
Since I am new to MvvmCross I couldn't think of a way to make this work. Any help would be highly appreciated.
Note: MvvmCross version is 6.4.2
IPlatformSpecificDialogs.cs in Core/Common Project
public interface IPlatformSpecificDialogs
{
    void Alert(string message, string title, string okButtonText);
}

IOSDialogs.cs in iOS Project
public class IOSDialogs : IPlatformSpecificDialogs
{
    public IOSDialogs()
    {
    }

    public void Alert(string message, string title, string okButtonText)
    {
        Console.WriteLine("worked");
    }
}

Setup.cs in iOS Project
public class Setup : MvxIosSetup<App>
{
    public Setup() : base()
    {
    }

    protected override IMvxApplication CreateApp()
    {
        return App.Instance;
    }

    protected override void InitializeLastChance()
    {
        base.InitializeLastChance();
        Mvx.IoCProvider.RegisterSingleton<IPlatformSpecificDialogs>(() => new IOSDialogs());
    }
}

MyView.cs in iOS Project
[MvxFromStoryboard("Main")]
[MvxModalPresentation(WrapInNavigationController = false)]
public partial class MyView : BaseView<MyViewModel>
{
    public MyView(IntPtr handle) : base(handle)
    {
    }

    public override void ViewDidLoad()
    {
        base.ViewDidLoad();
    }

    protected override void BindControllers()
    {
        var set = this.CreateBindingSet<MyView, MyViewModel>();
        set.Bind(AlertButton).To(vm => vm.AlertCommand);
        set.Apply();
    }
}

MyViewModel.cs in Core/Common Project
public class MyViewModel : BaseViewModel
{
    private readonly IPlatformSpecificDialogs _platformSpecificDialogs;

    public MyViewModel(IPlatformSpecificDialogs platformSpecificDialogs) : base()
    {
        _platformSpecificDialogs = platformSpecificDialogs;
    }

    public override Task Initialize()
    {
        return base.Initialize();
    }

    public IMvxCommand AlertCommand => new MvxCommand(PopupAlert);
    private void PopupAlert()
    {
        _platformSpecificDialogs.Alert("Title","Messege", "OK");
    }
}

Exception I get when I navigate to MyView

This is the Inner Exception Messege

Failed to resolve parameter for parameter platformSpecificDialogs of
  type IPlatformSpecificDialogs when creating
  MyProject.Core.ViewModels.MyViewModel. You may pass it as an argument.

This is the Inner Exception StackTrace

at MvvmCross.IoC.MvxIoCContainer.TryResolveParameter (System.Type
  type, System.Reflection.ParameterInfo parameterInfo, System.Object&
  parameterValue) [0x00020] in
  D:\a\1\s\MvvmCross\IoC\MvxIoCContainer.cs:686    at
  MvvmCross.IoC.MvxIoCContainer.GetIoCParameterValues (System.Type type,
  System.Reflection.ConstructorInfo selectedConstructor,
  System.Collections.Generic.IDictionary'2[TKey,TValue] arguments)
  [0x0003a] in D:\a\1\s\MvvmCross\IoC\MvxIoCContainer.cs:646    at
  MvvmCross.IoC.MvxIoCContainer.IoCConstruct (System.Type type,
  System.Collections.Generic.IDictionary'2[TKey,TValue] arguments)
  [0x0002b] in D:\a\1\s\MvvmCross\IoC\MvxIoCContainer.cs:416    at
  MvvmCross.IoC.MvxIoCContainer.IoCConstruct (System.Type type)
  [0x00000] in D:\a\1\s\MvvmCross\IoC\MvxIoCContainer.cs:363    at
  MvvmCross.IoC.MvxIoCProvider.IoCConstruct (System.Type type) [0x00000]
  in D:\a\1\s\MvvmCross\IoC\MvxIoCProvider.cs:149    at
  MvvmCross.ViewModels.MvxDefaultViewModelLocator.Load (System.Type
  viewModelType, MvvmCross.ViewModels.IMvxBundle parameterValues,
  MvvmCross.ViewModels.IMvxBundle savedState,
  MvvmCross.Navigation.EventArguments.IMvxNavigateEventArgs
  navigationArgs) [0x00000] in
  D:\a\1\s\MvvmCross\ViewModels\MvxDefaultViewModelLocator.cs:56

A:
Failed to resolve parameter for parameter platformSpecificDialogs of
  type IPlatformSpecificDialogs when creating
  MyProject.Core.ViewModels.MyViewModel. You may pass it as an argument.

The exception you are experiencing is due to a timing issue. To resolve it rather than placing the platform-specific dialogs registration in InitializeLastChance() place it in InitializeFirstChance()

Since I am new to MVVMCross I couldn't think of a way to make this
  work. Any help would be highly appreciated.

In terms of native dialogs I would suggest having a look at ACR User Dialogs you can either use the cross-platform library directly or use it for inspiration in creating your own custom implementation.

Explanation
MvvmCross startup life cycle consists of a two-phase (Documentation link).

*

*InitializePrimary - Runs on the main sync context (aka main thread). Initializes the IoC, Logging mechanism and other core parts.

*InitializeSecondary - Runs on the background (never on the main thread). Constructs some other platform services like bindings, the App class and calls Initialize on it. It finally registers Views / ViewModels lookups.

The initialized methods calls are as follows (Documentation link)
    // InitializePrimary
    InitializeFirstChance();                // Earliest possible hook
    InitializeDebugServices();
    InitializePlatformServices();
    InitializeSettings();
    InitializeSingletonCache();

    // InitializeSecondary
    PerformBootstrapActions();
    InitializeStringToTypeParser();
    InitializeViewModelFramework();
    var pluginManager = InitializePluginFramework();
    InitializeApp(pluginManager);           // App start up trigger here
    InitialiseViewModelTypeFinder();
    InitializeViewsContainer();
    InitiaiseViewDispatcher();
    InitializeViewLookup();
    InitialiseCommandCollectionBuilder();
    InitializeNavigationSerializer();
    InitializeInpcInterception();
    InitializeLastChance();                 // Last possible hook (async process)

You can see here InitializeFirstChance() occurs as the first hook within the InitializePrimary phase while  InitializeLastChance() occurs as the last hook in the
InitializeSecondaryphase. The issue arises where within InitializeSecondary the method InitializeApp(pluginManager) triggers the startup sequence for the app. This triggers the creation of your first view model which in your current case has a dependency on a service that has not yet been registered against the IoC provider. Leading to the exception showing that it can not resolve the parameter.
InitializeLastChance() usage would be fine if you don't have an immediate requirement for the service. Note, that InitializePlatformServices() is currently marked as deprecated in favour of InitializeFirstChance() (Documenation link).


text:
Quantity it made april dolphins set quality set
| Miami Dolphins Cheap Jerseys
| Posted in
Can you even get auto financing with bad credit?She hugs admirers, poses for selfies and warmly thanks person after person for attending.The running back became a focal point on offense for the first time in his career.
Laquincy Rideau had 15 points for South Florida .And, yes, injuries were a huge factor last season, but all teams deal with them to a degree .Specializing in One Conference It would be silly for me to start off with anything but the most important betting strategy.I do not evaluate it.It does not fully explain Collins' time with Special Olympics.He thinks he should get in.
FWIW, yesterday's news of Jefferson's restructure would seem to indicate that they aren't close to a deal w CJ Mosley.He missed the entire offseason https://www.cheapsnfljerseyhours.com program, and while he got up to speed during jerseys cheap training camp to take the field for the opener vs.There is plenty of room for passengers in this 3-row SUV, not to mention a substantial 23 cubic feet of cargo room .Mayers, who allowed an earned run on two hits while recording a strikeout over an inning against the Reds on Saturday, owns a 2 ERA and has one hold over 7 innings thus far this season.It's amazing that not a single team in the NHL Could there be some closure for big-name free agents on the horizon?
Carlesimo for the first time in four years since he tried to choke his former Golden State coach.You might not have to.The Canucks reassigned Gaunce to AHL Syracuse on Tuesday, Joe Smith of The Athletic reports.Yes, having Deion Jones on the field makes a huge difference for this defense.Jeff Van Gundy already is jumping ship from the LeBron James-Los Angeles Lakers experiment.Both the Blackhawks and Red Wings had exceptional seasons in 2008.
But he has accomplished so much.Oversized Telluride badging can be seen on the edge of the hood and on the liftgate.Sportsnet's Steve Dangle also agrees, believing Kapanen is too valuable to be traded for a back-end 'upgrade.' Instead, Dangle believes the Leafs should trade draft picks rather than roster players.Glasgow's currently ranked 18th among left guards by Pro Football Focus.Also, the Tacoma is a top choice for off-roading and has one of the highest payload capacities among classmates.Now I need to find out how the 2 door format works for installing a car seat.
Certainly, there's an agenda at work here: was the first modern EV to hit the market, and the company has been pushing its Intelligent Mobility platform, which has put in the hands of more drivers than perhaps any other brand.He was part of a bronze medal winning Swedish national team at the World Championship and was named not only the best defenceman in the tournament, but Sweden's best player as well.Justin Wright-Foreman had 29 points and six rebounds for the Pride .The puck hit a Penguins player and bounced out to Atkinson, who made no mistake.It will be a good duel between goalies, as Francouz will be faced by Vasili Koshechkin.However, it comes loaded with many high-tech safety and driver assistance aids that are pricey options on most rivals.
Inside, there's an 8-inch color touch screen and more leather seat options.If more than two clubs are tied, the higher percentage of available points earned in games among those clubs, and not including any odd games, shall be used to determine the standing.Kyle Walker: Is it all-wheel drive?I'm really not trying to be; I'm merely pointing out facts, stats, trends and try not to overreact and become a prisoner of the moment �?which so many fans do.
Good lord, he was really coming on again at the end of last season.Fans throughout the region celebrate Red Friday by donning red in support of their favorite team �?in offices, schools and homes.


I am really stumped on how to get this Nitrate down. I have a 220 that was down for about 10 months. I dont know what I was thinking but it had 1/2 old water and the other half i made new water. I was trying to just keep everything i had since there was tons of rock and sand and just keep stirring it up and letting it all filter back through. i have used stability and prime and in the past month i have done a whole bunch of water changes. I just did a 110 gallon water change last night and today the nitrate is still Blood red. i know the best answer is to start all over but if there is a way to fix this witout having to lose all that money in rock sand, and salt I am willing to try anything. there are no fish in there its been re cycling for about a month and a half now. Someone told me to use like 3 tablespoons of bleack and it will kill all the bad bacteria and then to add more stability and prime and it will filter out. it just scares me putting bleach in the tank? any opinions?
Consider adding chaeto to help export some of the nitrates - need something to adsorb them... Have you done any filter changes? are you using carbon?
Hi guys i did a 50% water change last night and it did nothing. I havent used carbon yet but i am changing the filter bags every day but the tank is crystal clear. no amonia or nitrite just rediculous nitrate. its got good lighting metal halides and T5's also i have brown algea growing on it. what is the Chaeto that you meantioned? also I didnt want to put anything in there just to kill it. what do you mean live shrimp? like the cleaners or the pepermint. i just figured they would die with high nitrate like that?
Take a sample to a friend or LFS to verify the test kit is correct. A long term fix could be a deep sand bed or even a remote DSB would help.
agree with confirming the test.
throw some chaeto in there to uptake the nutrients. Grow and harvest wekkly.
consider vodka dosing. YES, it works even in a tank without fish. At least it did for me.
ok thats now twice i someone has mentioned Chaeto. what is it? do you mean the green stuff you put in a refugium? also you mean like drinking vodka? to pour in the tank?
How are the phosphates? If you have a good skimmer, and adequate N and P, carbon dosing and skimming may remove those nutrients to a large degree. Research Vodka dosing, VSV, and/or Bio pellets for more info. JMTC & GL!
And welcome to Reef Central!
How would 3 tablespoons of bleach do anything to that volume of water?
And no I would not kill everything because nitrates can be dealt with over time.
Maybe if you listed your equipment and had a few pics people could provide additional ideas to help you.
Are your rocks covered in the algae or are they clean? Next water change save some of the water and take a few pieces of rock out and swish it really hard to see if you have a build up within your rocks.
You are wanting to address the source of the issue rather than symptoms.
Hang in there. You seem like a patient person and that is key in this.
You should maybe look into an algae scrubber. From what I have researched, they do an amazing job at removing nutrients(phosphate and nitrate) Because of that you will not have any algae in your DT. Definantly verify your test kit readings with a friend or LFS. What test kit are you using?
Get your test kit confirmed, first and foremost.
Secondly, the only way you can get your nitrates down quickly is by doing water changes. Chaeto (or other forms of macroalgae) will work, but it will take an eternity for macro alone to get your blood-red nitrates down to acceptable levels.
Since there are no fish in the tank currently, I would recommend just draining all/most of the water and replace with freshly made saltwater. Make sure your water source is nitrate-free, of course, i.e., use RO/DI water. As long as the live rock and sand stay moist, any nitrifying bacteria will be fine.
I would then test for nitrates so you have a good baseline. If it goes up after a few days, then nitrates must be leeching from the rocks. If this is the case, and you don't want to get rid of the LR, you'll just have to continue doing water changes. The suggestion for using macroalgae should help now, too. You'll have to keep doing this until all the nitrates leech out of the rocks.
You have 2 choices, IMO.
Choice #1. Start anew. Drain the tank and nuke everything. Wash all the rocks, give them a vinegar bath, rinse and repeat until they are clean and white. Same thing with all equipment. Throw away the sand. Then start all over again as a new system. This is the cheapest and most effective way to get going again after 10 months of neglect.
Choice #2. Invest in a large amount of salt, then do a series of 100% water changes every other day. Make sure to stir the substrate well, and turkey baste everything that you can turkey baste off the rocks before each water change. Rinse all mechanical filters like socks or sponges with each water change as well. I would keep the SG around 1.020-1.022 rather than 1.026, where it is high enough to preserve life forms while still saving some salt money. When the nitrate level stays stable at less than 10, you can stop the 100% water changes and see what happens. Not the best resource, but here is an interesting link that addresses high toxic levels of nitrate: http://saltaquarium.about.com/od/nitratecontrol/ss/sbsnitratereduction.htm.
What brand test kit and how old is the test kit?
Try the test kit on a batch of new saltwater, should be 0.
After 10 mths of sitting there, just start over.
His choice 1 explains it well. Just really make sure your rock is clean.


Dancer19501 hasn't written any comments yet.
Dancer19501 hasn't received any messages yet.
Dancer19501 has no friends yet.
Dancer19501 has no fans yet.
Dancer19501 has no favorite users yet.
Dancer19501 has no favorite companies yet.


It's time to get together with the family and enjoy a great day out at the Warralily Market. There will be stalls full of handmade crafts, produce and goodies, food trucks and lots of free activities for the kids. Make sure you RSVP to the event here to keep up to date on stall holder info and acitivities.


text:
\section{INTRODUCTION}

For real-world manipulation tasks, a robot needs to deal with unknown, stochastic, and contact-rich environments as well as occluded and noisy sensor data. To approach these challenging constraints, a common simplification is to omit time dependency by decomposing actions into single-step, open-loop manipulation primitives. While this is done for classical analytical approaches, it is even more important for learned robotic manipulation. Recent innovations allow robots to learn tasks by interacting with its environment and maximizing the obtained reward. Since data consumption is the fundamental limitation in most learning tasks, the task itself needs to be as simple as possible. Single-step primitives allow to reduce costly training, in particular in the real world.

However, the discarded time dependency is often useful for practical applications. Amongst others, it first allows to plan longer manipulation sequences or second, optimize for time-dependent multiple-step criteria. In this work, we propose to keep learning the manipulation primitives in a single-step and data-efficient manner, while introducing a transition model on top. This visual transition model is learned from pairs of images \textit{before} and \textit{after} an executed manipulation action, and is then able to \textit{predict} the resulting state of an action. As a transition model adds errors into the system, we further predict the uncertainty of the transition model. By estimating the final uncertainty of the manipulation action, the robot is able to manipulate in a \textit{risk-aware} manner.

In this work, we address the task of \textit{bin picking} using a flange-mounted depth camera. It highlights several challenges of robotic grasping, e.g. unknown and partially hidden objects, as well as an obstacle-rich environment. On the basis of our prior work, we learn both the task of grasping as well as simple pre-grasping manipulation like shifting or pushing in real-world experiments \cite{berscheid_shifting_2019}.

\begin{figure}[t]
    \centering

\newcommand\IncludeGraphic[1]{
    \includegraphics[trim=88 42 65 50, clip, width=0.29\linewidth]{figures/simple-example-large-1/#1}
}

\begin{tikzpicture}
    \node (c) {};

	\node[inner sep=0, left of=c, node distance=58] (front-left) {\includegraphics[trim=10 56 26 54, clip, width=0.45\linewidth]{figures/front-left-sm.jpg}};

	\node[inner sep=0, right of=c, node distance=58] (front-right) {\includegraphics[trim=10 56 26 54, clip, width=0.45\linewidth]{figures/front-right-sm.jpg}};

	\node[below of=c, node distance=110] (2)                {\IncludeGraphic{result-1.jpg}};
	\node[left of=2, node distance=0.338\linewidth] (1)      {\IncludeGraphic{result-0.jpg}};
	\node[right of=2, node distance=0.338\linewidth] (3)     {\IncludeGraphic{result-2.jpg}};

	\node[] at ($(front-left.north west) + (0.35, -0.35)$) {\small \circled{\textbf{a}}};
	\node[] at ($(front-right.north east) + (-0.4, -0.4)$) {\small \circled{\textbf{b}}};

	\node[below of=3, node distance=57] (4) {\IncludeGraphic{result-3.jpg}};
	\node[below of=2, node distance=57] (5) {\IncludeGraphic{result-4.jpg}};
	\node[below of=1, node distance=57] (6) {\IncludeGraphic{result-5.jpg}};

	\coordinate (diff) at (-0.1, -0.1);
	\tikzstyle{number} = [rounded rectangle, draw=black, fill=white, anchor=west]
	\node[number, text width=0.8cm] (cam) at ($(1.north west) + (diff)$) {\small 1};
	\node[number] at ($(2.north west) + (diff)$) {\small 2};
	\node[number] at ($(3.north west) + (diff)$) {\small 3};
	\node[number] at ($(4.north west) + (diff)$) {\small 4};
	\node[number] at ($(5.north west) + (diff)$) {\small 5};
	\node[number] at ($(6.north west) + (diff)$) {\small 6};

	\draw [thick, ->] (1.east) to (2.west);
	\draw [thick, ->] (2.east) to (3.west);
	\draw [thick, ->] (3.south) to (4.north);
	\draw [thick, ->] (4.west) to (5.east);
	\draw [thick, ->] (5.west) to (6.east);

	\draw [fill=black!92] ($(cam.west) + (0.45, -0.09)$) rectangle +(0.3, 0.18);
	\draw [fill=black!92] ($(cam.west) + (0.75, 0)$) ++(0, 0.03) -- ++(0.15, 0.07) -- ++(0, -0.2) -- ++(-0.15, 0.07);
\end{tikzpicture}

	\caption[]{After a single measurement (a), the robot predicts the next visual state using a learned generative transition model (1-6) and is able to grasp objects blindly considering the predicted uncertainty (b). Image (1) was taken by the flange-mounted depth camera and shows the chosen grasps (white) and the relevant input window for the algorithm (red). \footnotemark[2]}
	\label{fig:front-page}
\end{figure}

We see our contributions as follows: First, we introduce a transition model for predicting both the visual state \textit{and} its pixel-wise uncertainty after a manipulation action. Second, we propagate this uncertainty through the manipulation model. For the latter, we use a common fully-convolutional reward estimation for planar manipulation. The robot is then able to plan actions on the predicted state regarding their uncertainty. Third, we evaluate the novel abilities in various real-world robotic experiments, e.g. by increasing the \ac{PPH} in bin picking tasks.

\footnotetext[2]{Supplementary material is published at \url{https://pantor.github.io/learning-transition-for-manipulation}}

\section{RELATED WORK}

Grasping as a foundation for robotic manipulation has been of great interest since the beginning of robotics research. Bohg et al.~\cite{bohg_data-driven_2014} differentiate between analytical and data-driven approaches. Historically, grasps were synthesized commonly based on analytical constructions of force-closure. For known objects, sampling and ranking based on model-based grasp metrics is still widespread \cite{miller_graspit!_2004}. However, as modeling grasps itself is challenging, it is even harder to do so for pre-grasping manipulation. Moreover, we will focus on the case of manipulation without object model.

\textbf{Learning for manipulation}: In recent years, learning as a purely data-driven method has achieved great progress for manipulation. Kroemer et al.~\cite{kroemer2019review} gave a survey about the variety of existing approaches. From our perspective, two major trends have emerged: First, an \textit{end-to-end} approach using a step-wise, velocity-like control of the end effector \cite{levine_learning_2016, jang_end--end_2017, kalashnikov_2018_qt, quillen_deep_2018}. Second, the usage of predefined single-step \textit{manipulation primitives} promises a less powerful but more data-efficient learning. This is often combined with planar manipulation and a \ac{FCNN} as a grasp quality \cite{mahler_dex-net_2017} or (more general) an action-value estimator \cite{berscheid_shifting_2019, zeng_learning_2018}. Similar to \textit{Dex-Net} \cite{mahler_dex-net_2017} for bin picking, these approaches map a single image to a single grasp point. Based on the data source, learning for manipulation can be grouped using simulation \cite{quillen_deep_2018}, analytical metrics \cite{mahler_dex-net_2017}, or real-world interaction \cite{berscheid_shifting_2019, zeng_learning_2018}. In particular for real-world learning without any transfer gap, data efficiency often limits manipulation to single-step primitives in practice \cite{berscheid_improving_2019}.

\textbf{Transition model}: In this work, we focus on the \textit{transition model} and its applications for robotic manipulation. Here, model-based \ac{RL} promises data-efficiency by incorporating a learned transition model into the policy training. While model-based \ac{RL} have successfully been applied to either non-robotic tasks \cite{atkeson1997comparison, kaiser2019model, hafner2019learning} or low-dimensional non-visual robotic tasks \cite{abbeel_2007_application}, the high-dimensional visual state space hinders adaption for robotic manipulation. Nevertheless, some ideas were explored in this domain: Byravan et al.~\cite{byravan_2017_se3} learned a model predicting rigid body motions from depth images, enabling to generate further rendered images from an updated state model. Boots et al.~\cite{boots_2014_learning} learned an image-to-image transition model of a moving robot arm. More recently, Finn et al.~\cite{finn2019foresight} learned a model estimating pixel flow transformations on images. Combined with model-predictive control, the robot performed nonprehensile manipulation with unknown objects. Similarly, Ebert et al.~\cite{ebert2018visual} learned a visual transition model to infer actions to reach an image-specified goal for manipulation tasks.

\textbf{Image-to-image translation}: The core task of a visual transition model is image-to-image translation. In recent years, two major approaches emerged in the field of machine learning: First, \ac{GAN} based methods allow to generate data samples similar to a training set \cite{goodfellow_generative_2014}. On top, the \textit{Pix2Pix} architecture by Isola et al.~\cite{isola_image-to-image_2016} was used in a variety of applications. It is based on a conditional \ac{GAN} combined with a U-Net architecture for the generating \ac{NN}. However, it fails to capture stochastic output distributions, as it is a deterministic mapping and very sensitive to mode collapse \cite{isola_image-to-image_2016}. Second, \ac{VAE} based methods try to capture the complete data distribution via a probabilistic latent space. Combining \acp{GAN} and \acp{VAE}, Zhu et al.~\cite{zhu_multimodal_2017} introduced the \textit{BicycleGAN} that enables multi-modal output distributions and uncertainty estimation. Furthermore, we'll give a brief introduction, but focus on the integration of the BicycleGAN architecture within our robotics application.

\section{LEARNING A TRANSITION MODEL}

\begin{figure}[t]
    \centering
    \vspace{2mm}

    \begin{subfigure}[t]{\linewidth}
    \centering
\begin{tikzpicture}[font=\small, scale=0.75]
    \newcommand{\layer}[6][] {
		\draw[#1] (#2,#3 / 2,#4 / 2) -- ++(#5,0,0) -- ++(0, 0, -#4) -- ++(-#5, 0, 0) -- cycle;
		\draw[#1] (#2,#3 / 2,#4 / 2) -- ++(#5,0,0) -- ++(0, -#3, 0) -- ++(-#5, 0, 0) -- cycle;
		\draw[#1] (#2 + #5,-#3 / 2,-#4 / 2) -- ++(0,0,#4) -- ++(0,#3,0) -- ++(0,0,-#4) -- cycle;
	}

	\newcommand{\window}[7][] {
		\draw[#1] (#2 + #5,-#3 / 2 + #6,-#4 / 2 + #7) -- ++(0,0,#4) -- ++(0,#3,0) -- ++(0,0,-#4) -- cycle;
	}

	\layer[fill=white, rotate around x=-75]{-1.7}{3.6}{3.6}{0}{}
	\layer[fill=white, rotate around x=-50]{-1.3}{3.6}{3.6}{0}{}
	\layer[fill=white, rotate around x=-25]{-0.9}{3.6}{3.6}{0}{}
	\node[] () at (-1.2, -1.5) {$\cdots$};

	\layer[thick, fill=white]{0}{3.6}{3.6}{0.07}{}
	\begin{scope}[canvas is yz plane at x=0.07]
		\node[transform shape, rotate=-90] (a) {\includegraphics[width=3.6cm, height=3.6cm]{figures/2-cropped}};
	\end{scope}

    \draw[blue, thick, fill=white] (1.4, -1.2) rectangle (1.6, 1.2);
    \draw[blue, thick, fill=white] (1.9, -0.7) rectangle (2.1, 0.7);
    \draw[blue, thick, fill=white] (2.4, -0.4) rectangle (2.7, 0.4);
    \draw[blue, thick, fill=white] (3.0, -0.2) rectangle (3.4, 0.2);
    \draw[blue, thick, fill=white] (3.7, -0.1) rectangle (4.5, 0.1);

    \draw[thick, ->] (0.8, 0) -- (1.4, 0);
    \draw[thick, ->] (1.6, 0) -- (1.9, 0);
    \draw[thick, ->] (2.1, 0) -- (2.4, 0);
    \draw[thick, ->] (2.7, 0) -- (3.0, 0);
    \draw[thick, ->] (3.4, 0) -- (3.7, 0);
    \draw[thick, ->] (4.5, 0) -- (5.0, 0);

	\layer[thick, fill=white]{5.63}{3.6}{3.6}{0.07}{}
	\begin{scope}[canvas is yz plane at x=5.7]
		\node[transform shape, rotate=-90] (a) {\includegraphics[width=3.6cm, height=3.6cm]{figures/heatmap}};
	\end{scope}
\end{tikzpicture}
    \caption{Manipulation Model $M$: The fully-convolutional manipulation \acf{NN} efficiently estimates the reward for a large number of possible actions. For a single rotation and single primitive type, the result can be interpreted as a reward heatmap.}
    \label{fig:manipulation-model-arch}
    \end{subfigure}
~
    \begin{subfigure}[t]{\linewidth}
    \centering
\begin{tikzpicture}[font=\small, scale=0.75]
    \newcommand{\layer}[6][] {
		\draw[#1] (#2,#3 / 2,#4 / 2) -- ++(#5,0,0) -- ++(0, 0, -#4) -- ++(-#5, 0, 0) -- cycle;
		\draw[#1] (#2,#3 / 2,#4 / 2) -- ++(#5,0,0) -- ++(0, -#3, 0) -- ++(-#5, 0, 0) -- cycle;
		\draw[#1] (#2 + #5,-#3 / 2,-#4 / 2) -- ++(0,0,#4) -- ++(0,#3,0) -- ++(0,0,-#4) -- cycle;
	}

	\newcommand{\window}[7][] {
		\draw[#1] (#2 + #5,-#3 / 2 + #6,-#4 / 2 + #7) -- ++(0,0,#4) -- ++(0,#3,0) -- ++(0,0,-#4) -- cycle;
	}

	\layer[thick, fill=white]{0}{3.6}{3.6}{0.07}{}
	\begin{scope}[canvas is yz plane at x=0.077]
		\node[transform shape, rotate=-90] (a) {\includegraphics[width=3.6cm, height=3.6cm]{figures/before}};
	\end{scope}

	\window[red, thin]{0}{1.4}{1.4}{0.08}{0.55}{-0.3}

    \draw[blue, thick, fill=white] (1.5, -1.2) rectangle (1.8, 1.2);
    \draw[blue, thick, fill=white] (2.1, -0.5) rectangle (2.4, 0.5);
    \draw[blue, thick, fill=white] (2.7, -0.2) rectangle (3.3, 0.2);
    \draw[blue, thick, fill=gray!20] (3.52, -0.6) rectangle (3.82, 0.4);
    \draw[blue, thick, fill=white] (3.6, -0.5) rectangle (3.9, 0.5);
    \draw[blue, thick, fill=gray!20] (4.12, -1.3) rectangle (4.42, 1.1);
    \draw[blue, thick, fill=white] (4.2, -1.2) rectangle (4.5, 1.2);

    \draw[thick, ->] (0.8, 0) -- (1.5, 0);
    \draw[thick, ->] (1.8, 0) -- (2.1, 0);
    \draw[thick, ->] (2.4, 0) -- (2.7, 0);
    \draw[thick, ->] (3.3, 0) -- (3.6, 0);
    \draw[thick, ->] (3.9, 0) -- (4.2, 0);
    \draw[thick, ->] (4.5, 0) -- (5.0, 0);
    \draw[thick, ->, dotted] (2.4, -0.35) -- (3.6, -0.35);
    \draw[thick, ->, dotted] (1.8, -1.0) -- (4.2, -1.0);

	\layer[thick, fill=white]{6.0}{3.6}{3.6}{0.07}{}
	\begin{scope}[canvas is yz plane at x=6.07]
		\node[transform shape, rotate=-90] (a) {\includegraphics[width=3.6cm, height=3.6cm]{figures/after}};
	\end{scope}
	\window[red, thin]{6.0}{1.4}{1.4}{0.08}{0.55}{-0.3}
\end{tikzpicture}
    \caption{Transition Model $T$: The \textit{U-Net} architecture predicts the next image state $s_{t+1}$ depending on the prior state $s_{t}$, the action $a_t$, and its reward $r_t$. The result of the predicted window around the action (red) is then patched into the original image.}
    \label{fig:transition-model-arch}
    \end{subfigure}

    \caption{We make use of two \acp{NN}, one for the \textit{manipulation} policy (a) and one for the \textit{transition} model (b).}
    \label{fig:smy_label}
\end{figure}

From \Acf{RL}, we adopt the concept of a \ac{MDP} $(\mathcal{S}, \mathcal{A}, T^*, r, p_0)$ with the state space $\mathcal{S}$, the action space $\mathcal{A}$, the transition distribution $T^*$, the reward function $r$ and the initial configuration $p_0$. Similar to other data-driven approaches, \ac{RL} is limited by its data consumption, and even more so for time-dependent tasks resulting in \textit{sparse rewards}. For this reason, we simplify manipulation to a single time step problem. Then, a solution to this \ac{MDP} is a policy $\pi: \mathcal{S} \mapsto \mathcal{A}$ mapping the current state $s \in \mathcal{S}$ to an action $a \in \mathcal{A}$. In this work, we explicitly learn a transition model $T: \mathcal{S} \times \mathcal{A} \mapsto \mathcal{S}$ approximating the true transition distribution $T^*$.

\subsection{Learning for Robotic Manipulation}

\begin{figure}[t]
	\centering
	\vspace{0.8mm}

	\centering
\begin{tikzpicture}[x={(-0.5cm, -0.5cm)}, y={(1cm, -0.1cm)}, z={(0cm, 1cm)}, scale=0.8]
	\definecolor{bincolor}{RGB}{80, 80, 255}
	\tikzstyle{facestyle} = [fill=bincolor!8, draw=gray, thin, line join=round]
	\tikzstyle{ann} = [black, inner sep=1pt]
	\tikzstyle{coordaxis} = [black, thick, ->]

	\begin{scope}[canvas is yx plane at z=0.2]
		\path[facestyle, fill=bincolor!15] (-0.5,0) rectangle (1.5,2);
	\end{scope}

	\begin{scope}[canvas is zy plane at x=0]
		\path[facestyle] (0.2,-0.5) rectangle (1,1.5);
	\end{scope}

	\begin{scope}[canvas is zx plane at y=-0.5]
		\draw[facestyle] (0.2,0) rectangle (1,2.1);
	\end{scope}

	\begin{scope}[canvas is zy plane at x=2.1]
		\path[facestyle] (0.2,-0.5) rectangle (1,1.5);
	\end{scope}

	\begin{scope}[canvas is zx plane at y=1.5]
		\draw[facestyle] (0.2,0) rectangle (1,2.1);
	\end{scope}

	\begin{scope}[canvas is xy plane at z=1.5]
		\draw[step=5mm, very thin, gray!40] (-1.5,-1.5) grid (2,2);
	\end{scope}

	\draw[coordaxis] (0,0,1.5) -- +(1.8,0,0) node[right] {$x$};
	\draw[coordaxis] (0,0,1.5) -- +(0,1.8,0) node[right] {$y$};
    \draw[coordaxis] (0,0,1.5) -- +(0,0,1.8) node[above] {$z$};

    \draw[coordaxis, <-] (0.2,-0.2,2.8) arc(-20:280:.4) node[left] {$a$};

	\begin{scope}[shift={(0, 0, 2.0)}, rotate around z=-28]
		\draw[black, fill=black] (0.4,-0.1,0) -- ++(0,0,-0.5) -- ++(0,0.2,0) -- ++(0,0,0.5) -- cycle;
		\draw[black, fill=black] (-0.4,-0.1,0) -- ++(0,0,-0.5) -- ++(0,0.2,0) -- ++(0,0,0.5) -- cycle;

		\draw[black, fill=white] (-0.5,0.25,0) -- ++(1,0,0) -- ++(0,0,0.5) -- ++(-1,0,0) -- cycle;
	    \draw[black, fill=white] (0.5,-0.25,0) -- ++(0,0.5,0) -- ++(0,0,0.5) -- ++(0,-0.5,0) -- cycle;
	    \draw[black, fill=white] (-0.5,-0.25,0.5) -- ++(0,0.5,0) -- ++(1,0,0) -- ++(0,-0.5,0) -- cycle;
	    \draw[black, thin, fill=gray!30, dashed] (0,0,0.5) circle (0.2);
	    \draw[gray!60, thin, dashed] (0.173,-0.1,0.5) -- ++(0,0,1);
	    \draw[gray!60, thin, dashed] (-0.173,0.1,0.5) -- ++(0,0,1);

	    \draw[black, thin] (0.4,0,-0.5) -- +(0,2.2,0);
	    \draw[black, thin] (-0.4,0,-0.5) -- +(0,2.2,0);
	    \draw[black, thin, <->] (-0.4,2,-0.5) -- ++(0.8,0,0) node[below right, midway] {$d_m$};

	    \draw[red, fill=red] (0,0,-0.5) circle (0.08);
	\end{scope}
\end{tikzpicture}

	\caption{The manipulation \acf{NN} maps an orthographic depth image to an action type (\textit{grasp} or \textit{shift}) and a planar pose $(x, y, z, a)$ with gripper width $d_m$.}
	\label{fig:task-space}
\end{figure}

Similar to our prior work, we make use of six so-called \textit{manipulation primitives}:
\begin{enumerate}
    \item \textbf{Grasps} with four different pre-shaped gripper widths $d_m$. Given a grasp point, the robot approaches on a trajectory parallel to its gripper. It then closes the gripper, (possibly) lifts an object, and measures the grasp success by its internal force sensors. We define a binary reward $r$ for these primitives equal to the grasp success.
    \item \textbf{Shifts} in two different, horizontal directions (relative to the gripper). The robot approaches a given manipulation point, and moves its closed gripper for a few \si{cm}, leading to a shifting or pushing motion. The reward is defined by the difference of the maximal grasp reward before and after the action within a pre-defined neighborhood of the action.
\end{enumerate}
We simplify the action space to planar manipulation. Then, each action is described by five parameters: The spatial coordinates $(x, y, z, a)$ (as shown by Fig.~\ref{fig:task-space}) and the manipulation primitive type $p_t$. \\

Let $s_t \in \mathcal{S}$ be an orthographic image of the scene at time step $t$. We align the task-space coordinate system with the image frame by using a top-down view of the scene. This way, orthographic images have a key advantage over other projections: Affine transformations of the image $s$ correspond to planar poses $(x,y,a)$ of the robot.

We assume that manipulation (often) depends only on \textit{local} information. We introduce the cropped window $\hat{s} \in \hat{\mathcal{S}} \subset \mathcal{S}$ at the given affine transformation $(x, y, a)$. Its side length is slightly larger than the largest object size. We use a resolution of $(32 \times 32)$ pixels for the image window. Similar to our prior work \cite{berscheid_shifting_2019, berscheid_improving_2019}, the system is learned in a self-supervised manner to maximize the corresponding reward $r$. Therefore, a manipulation \ac{NN} called $M$ is trained to predict the reward $r$ of the image windows $\hat{s}_t$ with given primitive $p_t$. We make use of a fully-convolutional \ac{NN}: During inference, the same manipulation model $M(s_t)$ estimates rewards $\psi_t$ for a large number of poses efficiently in a sliding-window approach (Fig.~\ref{fig:manipulation-model-arch}). Rewards for different primitives $p_t$ are implemented as multiple channels in the output layer. The height $z$ is calculated trivially by a model-based controller. The final action $a_t$ is selected greedily by the policy $\pi$.

\subsection{Simplifications for the Transition Model}

\begin{figure*}[t]
	\centering
	\vspace{1.6mm}

	\definecolor{bis}{RGB}{14, 58, 106}
    \definecolor{bis1}{RGB}{194, 144, 54}
    \definecolor{bisp1}{RGB}{246, 192, 77}
    \definecolor{biN}{RGB}{85, 135, 168}
    \definecolor{biQ}{RGB}{246, 192, 77}
    \definecolor{biz}{RGB}{254, 181, 22}
    \definecolor{biL}{RGB}{224, 230, 243}

\newcommand{0.1\linewidth}{0.17\linewidth}
\newcommand\IncludeGraphic[1]{%
    \includegraphics[trim=88 42 65 50, clip, width=0.1\linewidth]{figures/simple-example-1/#1}
}

\begin{tikzpicture}
    \tikzstyle{title} = [text width=0.1\linewidth, align=left, node distance=50, font=\bfseries]
    \tikzstyle{captiontop} = [text width=0.1\linewidth, align=left, node distance=36]
    \tikzstyle{captionbottom} = [text width=0.1\linewidth, align=left, node distance=38]
    \tikzstyle{image} = [node distance=31]

    \node[] (1c) {};
	\node[image, above of=1c] (1r) {\IncludeGraphic{result-0.png}};
	\node[image, below of=1c] (1u) {\IncludeGraphic{uncertainty-0.png}};

	\node[captiontop, above of=1r] {\small \textbf{Measurement} $s_{t=0}$};
    \node[captionbottom, below of=1u] {\small Uncertainty $\Sigma_{t=0}^2$};

    \node[draw=black, fill=bis, text=white, minimum width=6mm, minimum height=7mm] (xpsi) at ($(1r.east) + (0.6, -0.3)$) {\small $M$};
    \node[draw=black, fill=bis, text=white, minimum width=6mm, minimum height=7mm] (xpsigrad) at ($(1u.east) + (0.6, 0.3)$) {\small $\frac{\partial M}{\partial s}$};

    \node[draw=black, fill=biL, right of=1c, node distance=100, minimum width=5mm, minimum height=12mm] (xpi) {\small $\pi$};

    \node[draw=black, fill=bisp1, minimum width=5mm, minimum height=26mm] (xt) at ($(1r.east) + (2.9, -0.5)$) {\small $T$};
    \node[draw=black, fill=biL, rounded rectangle, minimum size=6mm] (xplus) at ($(1u.east) + (3.5, -0.3)$) {\small $+$};

	\node[right of=1c, node distance=214] (2c)  {};
	\node[image, above of=2c] (2r) {\IncludeGraphic{result-1.png}};
	\node[image, below of=2c] (2u) {\IncludeGraphic{uncertainty-1.png}};

	\node[captiontop, above of=2r]  {\small \textbf{Prediction} $t=1$};

	\node[right of=2c, node distance=100] (3c)  {};
	\node[image, above of=3c] (3r) {\IncludeGraphic{result-2.png}};
	\node[image, below of=3c] (3u) {\IncludeGraphic{uncertainty-2.png}};
	\node[captiontop, above of=3r] {\small $t=2$};

	\node[right of=3c, node distance=100] (4c)  {};
	\node[image, above of=4c] (4r) {\IncludeGraphic{result-3.png}};
	\node[image, below of=4c] (4u) {\IncludeGraphic{uncertainty-3.png}};
	\node[captiontop, above of=4r] {\small $t=3$};

	\begin{scope}[->]
        \draw ([yshift=3mm]1r.east) |- ([yshift=8mm]xt.west);
        \draw ([yshift=-3mm]1r.east) |- (xpsi.west);

        \draw ([yshift=3mm]1u.east) |- (xpsigrad.west);
        \draw ([yshift=-3mm]1u.east) |- (xplus.west);

        \draw (xpsi.east) -| +(0.2, -0.5) |- ([yshift=3mm]xpi.west);
        \draw (xpsigrad.east) -| +(0.2, 0.5) |- ([yshift=-3mm]xpi.west);

        \draw ([yshift=3mm]xpi.east) |- ([yshift=-3mm]xt.west);
        \draw ([yshift=-3mm]xpi.east) |- ([yshift=-9mm]xt.west);

        \draw ([yshift=8mm]xt.east) |- ([yshift=3mm]2r.west);
        \draw ([yshift=-9mm]xt.east) -| (xplus.north);

        \draw (xplus.east) |- ([yshift=-3mm]2u.west);
    \end{scope}

	\begin{scope}[thick, ->, decoration={post length=4pt}, rounded corners=2.5mm]
	    \draw ($(2r.east)!0.5!(2u.east)$) -- ($(3r.west)!0.5!(3u.west)$);
	    \draw ($(3r.east)!0.5!(3u.east)$) -- ($(4r.west)!0.5!(4u.west)$);
    \end{scope}

    \node[] at ($(xt.west) + (-0.5, 1.05)$) {$\hat{s}_t$};
    \node[] at ($(xt.east) + (0.45, 1.05)$) {$\hat{s}_{t+1}$};
    \node[] at ($(xt.east) + (0.6, -0.68)$) {$\hat{\sigma}_{s, t+1}^2$};

    \node[] at ($(xplus.west) + (-1.0, -0.3)$) {$\hat{\Sigma}_{t}^2$};

    \node[] at ($(xpi.east) + (0.26, 0.47)$) {$r_t$};
    \node[] at ($(xpi.east) + (0.26, -0.52)$) {$p_t$};

    \node[] at ($(xpsi.east) + (0.44, -0.06)$) {$\psi_t$};
    \node[] at ($(xpsigrad.east) + (0.55, 0.02)$) {$\sigma_{a, t}$};

    \draw[double, densely dashed, ->] ($(4r.north) + (0.7, 0.12)$) -- ++(0.8, 0);
\end{tikzpicture}

	\caption{Iterative multiple-step predictions of the state $s_t$ with corresponding uncertainty $\Sigma_t^2$ given an initial measurement $s_{t=0}$. The computational flow during a prediction step is as follows: The manipulation \acf{NN} $M$ estimates rewards $\psi_t$ for a discrete set of action primtives $p_t$, its gradients are used to propagate the cumulative input uncertainty $\Sigma_t^2$ towards the estimated reward $\sigma_{a,t}$. The policy $\pi$ chooses the final action $a$ with corresponding reward $r$ using the lower bound of the estimated reward. Then, the transition model $T$ predicts the new state $\hat{s}_{t+1}$ around the action $a$ with pixel-wise uncertainties $\hat{\sigma}_{s,t+1}^2$. The uncertainties are summed up ranging from low (blue) to high (red).}
	\label{fig:example-iterative-predictions}
\end{figure*}

Similar to the local simplifications for manipulation, the transition model $T$ learns only to predict a window $\hat{s}$ around the action pose. While the window size could in principle match to the overall image $s$, we limit the window size to twice the maximal object size. This way, we presume a principle of locality for the state transitions, but it allows to remove all spatial dependencies from the transition model. To estimate the distribution over $s_{t+1}$, the state image $s_{t}$ is patched with $\hat{s}_{t+1}$ at the affine transformation $(x, y, a)$
\begin{align}
	p(s_{t+1}) &= T(s_t, a_t) \approx T(\hat{s}_t, p_t, r_t)
\end{align}
with the manipulation primitive type $p_t$. Additionally, we condition the transition model on the reward $r_t$ of the action. This is motivated by following arguments:
\begin{itemize}
    \item By using the estimated reward $\psi_t$ instead, the calculations of this difficult and high-level feature can be outsourced from the transition model to the already learned manipulation \ac{NN}. This is in particular important for grasp actions: Here, the next states depends strongly on the binary grasp success. Therefore, we train the manipulation model to learn the grasp success explicitly (as it is measurable) and reuse this information in the transition model.
    \item Real reward measurements $r_t$ can be used in the transition model. For example, the grasp reward can be quickly measured by the robot's gripper.
    \item Note that there is no loss of generality by condition the transition model on the reward, as we can substitute the manipulation model as a reward predictor.
\end{itemize}
If not stated otherwise, we simplify the transition model to
\begin{align}
	p(s_{t+1}) &\approx T \left( \hat{s}_t, p_t, M(\hat{s}_t) \right)
\end{align}
Let $\hat{s}_{t+1}^\prime$ denote the prediction with highest probability.

\subsection{Generative Transition Model Architecture}

We employ the BicycleGAN architecture for our transition model \cite{zhu_multimodal_2017}. In a nutshell, the architecture is based on a generator, discriminator, and an encoder \ac{NN}. The generator tries to mimic images from the underlying distribution, taking a noise vector $z$ from a prior latent distribution as input. While playing a min-max-game, the discriminator aims to distinguish the generated data from the real data. The BicycleGAN introduces an encoder to calculate a representation from an image within the latent space $z$. By constraining the latent distribution between the real and generated images, as well as a similarity to a Gaussian, the architecture is able to output a multi-modal distribution. This way, the BicycleGAN architecture is in principle able to capture the stochastic physics as a transition distribution. \\

We use the generator \ac{NN} as the transition model $T$. It uses a \textit{U-Net} architecture with $(64 \times 64)$ pixel input and output size (Fig.~\ref{fig:transition-model-arch}). We extend the BicycleGAN architecture by additionally condition the generator on the manipulation primitive type $p_t$ (as a one-hot encoded vector) and the reward $r_t$.

\subsection{Uncertainty Prediction}

Given the image distribution $p(s_{t+1})$, we are able to estimate a pixel-wise uncertainty of the prediction by sampling from its latent space $z$. Let the uncertainty $\hat{\sigma}_{s}^2$ be the variance of the prediction $\hat{s}_{t+1}^\prime$
\begin{align}
    \hat{\sigma}_{s}^2 &= \mathbb{E} \left[ \left( \hat{s} - \hat{s}_{t+1}^\prime \right)^2 \right] = \sum_i^N p(\hat{s}_i) \left( \hat{s}_{t+1, i} - \hat{s}_{t+1}^\prime \right)^2
\end{align}
given the number of samples $N$. The state uncertainty $\sigma_{s, t+1}^2$ of the overall image is estimated by zero-padding the uncertainty window $\hat{\sigma}_{s}^2$ at $(x, y, a)$. The state uncertainty $\sigma_{s}^2$ can be propagated towards the estimated reward $\psi_t$ of the manipulation action. A first-order Taylor series of the manipulation \ac{NN} $M$ yields
\begin{align}
    \sigma_a^2 \approx \sum_{i, j} \left\lvert \frac{\partial M(\hat{s}_{t+1}^\prime)}{\partial s_{i, j}} \right\rvert^2 \sigma_{s; i, j}^2
    \label{eq:propagation-of-uncertainty}
\end{align}
summing over all pixels $(i, j)$. The approximation presumes a zero covariance between individual pixels and only small uncertainties $\sigma_s^2$. A key contribution of our work is to implement (\ref{eq:propagation-of-uncertainty}) fully-convolutionally: The gradient of the fully-convolutional manipulation \ac{NN} regarding the pixel-wise image input is fully-convolutional itself. Then, the gradient is convoluted over the mean prediction and multiplied with the pixel-wise uncertainty. This results in an efficient computation of both the estimated rewards and its uncertainties $\psi_t \pm \sigma_a$. Note that we only consider the input uncertainty; other uncertainties, in particular from $M$ itself, are neglected in this work.

\subsection{From Predicting to Planning}
\label{subsec:from-predicting-to-planning}

\begin{figure*}
	\centering
	\vspace{0.5mm}

\newcommand{0.1\linewidth}{0.05\linewidth}
\newcommand\IncludeGraphic[2]{
    \includegraphics[width=0.1\linewidth]{figures/example-predict/#1/#2}
}

\newcommand\IncludeRGBDGraphic[2]{
    \includegraphics[width=0.1\linewidth]{figures/example-rgbd/#1/#2}
}

\newcommand\AddRow[5]{
    \node[pred, below of=#3] (#2o) {\IncludeGraphic{#1}{bi_s_bef.png}};
	\node[pred, right of=#2o] (#2a) {\IncludeGraphic{#1}{bi_s_aft.png}};

	\node[left of=#2o, node distance=32] (#2tc) {};
	\node[text width=30, align=right, above of=#2tc, node distance=4] {\footnotesize #4};
	\node[text width=30, align=right, below of=#2tc, node distance=6] {\footnotesize #5};

	\node[pred, right of=#2a, node distance=35] (#2b1) {\IncludeGraphic{#1}{bi_result.png}};
	\node[pred, right of=#2b1] (#2bu) {\IncludeGraphic{#1}{bi_unc.png}};
	\node (#2bc) at ($(#2b1)!0.5!(#2bu)$) {};

	\node[pred, right of=#2bu, node distance=35] (#2p1) {\IncludeGraphic{#1}{pix2pix_result.png}};
	\node (#2pc) at ($(#2p1)$) {};

	\node[pred, right of=#2p1, node distance=35] (#2v1) {\IncludeGraphic{#1}{vae_result.png}};
	\node[pred, right of=#2v1] (#2vu) {\IncludeGraphic{#1}{vae_unc.png}};
	\node (#2vc) at ($(#2v1)!0.5!(#2vu)$) {};
}

\newcommand\AddColorRow[5]{
    \node[pred, below of=#3] (#2colorbefore) {\IncludeRGBDGraphic{#1}{bi_s_bef_color.png}};
	\node[pred, right of=#2colorbefore] (#2depthbefore) {\IncludeRGBDGraphic{#1}{bi_s_bef_depth.png}};
	\node (#2imagecenter) at ($(#2colorbefore)!0.5!(#2depthbefore)$) {};

	\node[left of=#2colorbefore, node distance=29] (#2labelcenter) {};
	\node[text width=24, align=right, above of=#2labelcenter, node distance=4] {\footnotesize #4};
	\node[text width=24, align=right, below of=#2labelcenter, node distance=6] {\footnotesize #5};

	\node[pred, right of=#2depthbefore, node distance=30] (#2colorafter) {\IncludeRGBDGraphic{#1}{bi_s_aft_color.png}};
	\node[pred, right of=#2colorafter] (#2depthafter) {\IncludeRGBDGraphic{#1}{bi_s_aft_depth.png}};
	\node (#2colorcenter) at ($(#2colorafter)!0.5!(#2depthafter)$) {};

	\node[pred, right of=#2depthafter, node distance=30] (#2colorpred) {\IncludeRGBDGraphic{#1}{bi_result_color.png}};
	\node[pred, right of=#2colorpred] (#2depthpred) {\IncludeRGBDGraphic{#1}{bi_result_depth.png}};
	\node (#2bc) at ($(#2colorpred)!0.5!(#2depthpred)$) {};

	\node[pred, right of=#2depthpred, node distance=30] (#2colorunc) {\IncludeRGBDGraphic{#1}{bi_unc_color.png}};
	\node[pred, right of=#2colorunc] (#2depthunc) {\IncludeRGBDGraphic{#1}{bi_unc_depth.png}};
	\node (#2predcolorcenter) at ($(#2colorunc)!0.5!(#2depthunc)$) {};
}

	\begin{subfigure}[t]{\linewidth}
		\centering
\begin{tikzpicture}
    \tikzstyle{pred} = [node distance=0.1\linewidth]
    \tikzstyle{textbox} = [node distance=19]

    \node[] (cleft) {};

	\AddRow{2019-06-06-13-49-56-259}{1}{cleft}{$r_{t} = 1$}{$p_{t} = 2$}
	\AddRow{2019-06-06-17-55-29-592}{2}{1o}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddRow{2019-07-01-11-09-38-116}{3}{2o}{$r_{t} = 1$}{$p_{t} = 2$}
	\AddRow{2019-03-12-15-17-27-286}{4}{3o}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddRow{2019-06-24-21-30-17-452}{5}{4o}{$r_{t} = 1$}{$p_{t} = 1$}

    \node[textbox, above of=1o] {\small $\hat{s}_{t}$};
	\node[textbox, above of=1a] {\small $\hat{s}_{t+1}$};
	\node[textbox, above of=1bc, node distance=18] {\small BicycleGAN};
	\node[textbox, above of=1pc] {\small Pix2Pix};
	\node[textbox, above of=1vc] {\small VAE};

	\node[right of=cleft, node distance=253] (cright) {};

	\AddRow{2019-05-07-20-11-38-717}{6}{cright}{$r_{t} = 1$}{$p_{t} = 3$}
	\AddRow{2019-07-01-11-28-34-816}{7}{6o}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddRow{2019-07-01-14-05-53-150}{8}{7o}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddRow{2018-12-11-09-54-26-687}{9}{8o}{$r_{t} = 0.8$}{$p_{t} = 4$}
	\AddRow{2019-07-01-11-10-31-450}{10}{9o}{$r_{t} = 0$}{$p_{t} = 1$}

    \node[textbox, above of=6o] {\small $\hat{s}_{t}$};
	\node[textbox, above of=6a] {\small $\hat{s}_{t+1}$};
	\node[textbox, above of=6bc, node distance=18] {\small BicycleGAN};
	\node[textbox, above of=6pc] {\small Pix2Pix};
	\node[textbox, above of=6vc] {\small VAE};
\end{tikzpicture}
	    \caption{Comparison of predictions by the BicycleGAN, Pix2Pix (without uncertainty) and VAE model. }
		\label{subfig:example-small-predictions-comparison}
	\end{subfigure}

	\begin{subfigure}[t]{\linewidth}
		\centering
\begin{tikzpicture}
    \tikzstyle{pred} = [node distance=0.1\linewidth]
    \tikzstyle{textbox} = [node distance=19]

    \node[] (cleft) {};

	\AddColorRow{2020-06-25-21-02-27-805}{1}{cleft}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddColorRow{2020-07-28-19-49-21-994}{2}{1colorbefore}{$r_{t} = 1$}{$p_{t} = 0$}
	\AddColorRow{2020-07-28-01-40-42-547}{3}{2colorbefore}{$r_{t} = 1$}{$p_{t} = 0$}
	\AddColorRow{2020-07-29-11-15-13-630}{4}{3colorbefore}{$r_{t} = 1$}{$p_{t} = 2$}
	\AddColorRow{2020-07-27-21-41-02-304}{5}{4colorbefore}{$r_{t} = 1$}{$p_{t} = 1$}

    \node[textbox, above of=1imagecenter] {\small $\hat{s}_{t}$};
	\node[textbox, above of=1colorcenter] {\small $\hat{s}_{t+1}$};
	\node[textbox, above of=1bc] {\small BicycleGAN};
	\node[textbox, above of=1predcolorcenter] {\small Uncertainty};

	\node[right of=cleft, node distance=253] (cright) {};

	\AddColorRow{2020-07-02-14-18-59-404}{6}{cright}{$r_{t} = 1$}{$p_{t} = 3$}
	\AddColorRow{2020-07-27-23-52-19-678}{7}{6colorbefore}{$r_{t} = 1$}{$p_{t} = 3$}
	\AddColorRow{2020-07-29-14-42-23-427}{8}{7colorbefore}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddColorRow{2020-07-08-19-03-07-681}{9}{8colorbefore}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddColorRow{2020-07-29-11-47-18-573}{10}{9colorbefore}{$r_{t} = 1$}{$p_{t} = 2$}

    \node[textbox, above of=6imagecenter] {\small $\hat{s}_{t}$};
	\node[textbox, above of=6colorcenter] {\small $\hat{s}_{t+1}$};
	\node[textbox, above of=6bc] {\small BicycleGAN};
	\node[textbox, above of=6predcolorcenter] {\small Uncertainty};
\end{tikzpicture}
        \caption{Predictions by the BicycleGAN, for both RGB (not used for downstream manipulation) and depth images.}
		\label{subfig:example-small-predictions-rgbd}
	\end{subfigure}

	\caption{Example predictions given an image window $\hat{s}_t$ before a manipulation action of type~$p_t$ and reward~$r_t$. $\hat{s}_{t+1}$ corresponds to the ground truth. The normalized pixel-wise uncertainties are shown from low (blue) to high (red). Action types $p_t \in \{ 0, 1, 2, 3 \}$ are grasps of different pre-shaped gripper width; $p_t \in \{ 4, 5 \}$ are shifting motions.}
	\label{fig:example-small-predictions}
\end{figure*}

Following, we assume the independence of each state uncertainties $\hat{\sigma}_{s,t}$ over time $t$. Due to the linearity of the variance, we define the cumulative uncertainty
\begin{align}
    \Sigma_{t}^2 &= \sum_{i=0}^{t} \sigma_{s; i}^2
\end{align}
as the accumulated uncertainty of the state. Furthermore, we denote $t$ as the number of action steps after an image measurement at $t=0$. At $t=0$, we consider the sensor to be perfect and reset the cumulative uncertainty to zero. Moreover, we adapt the policy $\pi$ to select the action $a_t$ maximizing the lower confidence bound of the estimated reward $\psi_t$ given by
\begin{align}
    a_{t}^* &= \argmax_a \left( M(s_t) - \alpha \left\lvert \frac{\partial M(s_t)}{\partial s} \right\rvert \Sigma_{t} \right)
    \label{eq:lower-confidence-bound}
\end{align}
using the cumulative uncertainty $\Sigma_t$ in contrast to (\ref{eq:propagation-of-uncertainty}). Let $\alpha > 0$ denote a parameter for weighing the estimated reward of actions against their confidence. If the uncertainty rises above a threshold $\sigma_{\text{image}}^2$, a new image is taken. If the lower bound of the estimated reward falls below a threshold, the bin is assumed to be empty.
Furthermore, given both a policy $\pi$ and a transition model $T$, states and actions can be predicted multiple steps ahead by iteratively applying $\pi$ and $T$ alternately. Fig.~\ref{fig:example-iterative-predictions} shows the computational flow of a single prediction step, allowing to plan ahead and optimize for multiple-step criteria. As we don't focus on planning algorithms itself, we implemented a simple breadth-first tree search. To allow diverse tree paths, we adapt (\ref{eq:lower-confidence-bound}) to sample from the $N \approx 5$ highest actions uniform randomly. This way, we can predict the overall tree ahead of time and return the action path maximizing the criteria during inference.

\section{EXPERIMENTAL RESULTS}

Our experiments are performed on a Franka Panda robot arm using the default gripper and a flange-mounted Ensenso N10 or Framos D435e stereo camera (Fig.~\ref{fig:front-page}). We designed and printed custom gripper jaws and attached household anti-slip silicone roll to the robot's fingertips. Otherwise, the friction of the jaws would not be sufficient for shifting objects. The system uses an Intel Core i7-8700K processor and a NVIDIA GeForce GTX 1070 Ti for computing. We use the \textit{Frankx} library for robot control \cite{berscheid2021jerk}. Supplementary videos, results, as well as the source code are published at \url{https://pantor.github.io/learning-transition-for-manipulation}.

\subsection{Image Prediction}

During around \SI{180}{h} of real-world data collection, the robot attempted \num{40000}~grasps and \num{2400}~shifts. The robot used two bins with a variety of objects for training. After a successful grasp, the robot moves the objects to the other bin, however returns after filing to take an image \textit{after} the manipulation. After \num{4000} random actions, the robot used the $\varepsilon$-greedy exploration strategy for learning manipulation. $\varepsilon$ was reduced continuously and reached zero at \num{30000} actions, leading to an average reward of $\overline{r}\,=\,\num{0.55}$. The manipulation \ac{NN} was trained around every \num{500} actions. In the end, we train the BicycleGAN architecture on the complete dataset of state-action-state-tuples $(s_{t=0}, p_{t=0}, r_{t=0}, s_{t=1})$.
To stabilize the training of the \acp{GAN}, we smooth the labels and add random noise to the discriminator input. In comparison to the original implementation, we use a training batch size of \num{32}. Otherwise, we keep the training process of the BicycleGAN architecture. Fig.~\ref{fig:example-small-predictions} shows example predictions given the manipulation primitive type $p_t$ and the reward $r_t$. We include the ground truth as well as results of the Pix2Pix architecture (without uncertainty estimation) and a \ac{VAE} for visual comparison. Predicting both a state of $64 \times 64$ pixels with corresponding uncertainties using $N=20$ samples needs around \SI{60}{ms}.

While our transition model is able to generate realistic images, we find two common failure modes: First, low-level failures like large-scale deviations in the background or blurred edges. Second, wrong high-level features like cropped, blurred, or visual fused objects. The latter case occurs in particular for either shifting actions or occluded layers beneath the grasped object. Regions of high predicted uncertainty can be found primarily near edges, in particular of regions without depth information (black), and within the manipulated objects itself. Examples of iterative predictions are shown in Fig.~\ref{fig:example-iterative-predictions} and \ref{fig:front-page}.

\subsection{Grasp Rate}

We define the grasp rate as the percentage of successful grasps for grasping $15$ objects out of a bin with $25$ objects without replacement. For a bin picking scenario with multiple object types, Fig.~\ref{fig:grasp-predictions} shows the dependency between the grasp rate and the prediction step $t$, equivalent to the number of actions since the last image measurement.
\begin{figure}[ht]
	\centering

\begin{tikzpicture}[scale=0.86]
\pgfplotsset{xmin=-0.2, xmax=10.2}

\begin{axis}[
	axis y line*=left,
	ymin=0, ymax=1.05,
	xlabel=Prediction step $t$,
	ylabel=Grasp Rate,
	legend pos=south west,
	height=190,
]
    \addplot[mark=x, blue]
        plot [error bars/.cd, y dir = both, y explicit]
        table [y=grasprate, x=step, y error=grasprate_std]{figures/grasp-rate-over-prediction-step.txt};
	\label{plot_grasprate}

	\addplot[mark=none, black!20] coordinates {(-0.2, 1) (10.2, 1)};

	\addplot[mark=none, dashed, black!60] coordinates {(3.2, 0) (3.2, 1.05)};
	\addplot[mark=none, dashed, black!60] coordinates {(-0.2, 0.19) (10.2, 0.19)};

	\addplot[domain=0:10, dashed, color=blue]{-0.00476*x^2 - 0.01575*x + 0.99};
\end{axis}

\begin{axis}[
	axis y line*=right,
	axis x line=none,
	ymin=0.0, ymax=0.64,
	ylabel=Uncertainty $\sigma_a^2$,
	legend pos=north east,
	height=190,
]
	\addlegendimage{/pgfplots/refstyle=plot_grasprate}\addlegendentry{Grasp Rate}
	\addplot[mark=*, black]
	    plot [error bars/.cd, y dir = both, y explicit]
	    table [y=uncertainty, x=step, y error=uncertainty_std]{figures/grasp-rate-over-prediction-step.txt};
	\addlegendentry{Uncertainty}

\end{axis}

\end{tikzpicture}

	\caption{Grasp rate and uncertainty of the estimated reward $\sigma_a^2$ depending on the number of prediction steps after an initial image measurement at $t=0$. Based on \num{1080} grasp attempts in the defined bin picking scenario.}
	\label{fig:grasp-predictions}
\end{figure}
The grasp rate decreases from \SI{98.7}{\%} at $t=0$ to around \SI{90}{\%} for $t=3$. Then, the average uncertainty of the estimated reward $\sigma_a^2$ increases to around $\num{0.1}$, resulting in a significant term in (\ref{eq:lower-confidence-bound}).

In order to minimize the sum of lost time due to both grasping mistakes and image acquisitions, we estimate an optimal uncertainty threshold $\sigma_{\text{image}}^2$ given a rough estimate of the execution times of the robot. We identified the durations $t_G = \SI{6.5}{s}$ of the grasp and $t_I = \SI{2.5}{s}$ of the image measurement and related motions. The \ac{PPH} are then given by
\begin{align}
    \text{PPH} &= \frac{\text{Grasp Rate}}{t_G + \frac{1}{t} t_I} \approx \frac{\text{-}0.00354\,t^2 - 0.0228\,t + 0.987}{6.5 + 2.5\,t^{-1}}
\end{align}
with the optimal prediction step $t^* \approx \num{3.2}$. The quadratic regression is plotted in Fig.~\ref{fig:grasp-predictions}. To optimize the \ac{PPH} in further experiments, we set the threshold $\sigma_{\text{image}}^2$ for taking a new image to the corresponding uncertainty $\sigma_a^2 \approx \num{0.12}$.

\subsection{Benchmarking Picks Per Hour}

We evaluate our robotic system against the Box and Blocks Test from the YCB benchmark suite \cite{calli_benchmarking_2015}. Within \SI{2}{min}, the robot should grasp and place as many cubes (\SI{2.5}{cm} side length) as possible out of one bin into another.
Additionally, we introduce a setting where the robot aims to grasp multiple objects at once. Based on the work of semantic grasping \cite{jang_end--end_2017}, we extend the manipulation \ac{NN} to predict whether the closed gripper distance will be larger than \SI{4}{cm}. This corresponds to grasping at least \num{2} objects; the policy $\pi$ then maximizes the lower bound of the expected number of grasped objects.
\begin{table}[ht]
	\centering
	\vspace{1.4mm}
	\caption{Results for the Box and Blocks Test, including the grasp success rate, the object count after \SI{2}{min} and the corresponding \acf{PPH}.}
	\begin{tabular}{|l|c|c|c|}
	\hline
	& \textbf{Grasp\,Rate} & \textbf{Count} & \textbf{PPH} \\
	\hline
	Heuristic \cite{calli_benchmarking_2015} & \SI{80}{\%} & \num{10.8} & \num{324} \\
	Random & \SI{13 \pm 4}{\%} & \num{2.2 \pm 0.7} & \num{66 \pm 20} \\
	\hline
	Single & \SI{97 \pm 2}{\%} & \num{12.8 \pm 0.3} & \num{384 \pm 10} \\
	Single, Prediction & \SI{94 \pm 2}{\%} & \num{16.4 \pm 0.5} & \num{492 \pm 14} \\
	Multiple & \SI{96 \pm 2}{\%} & \num{20.4 \pm 1.0} & \num{612 \pm 29} \\
	Multiple, Prediction & \SI{94 \pm 2}{\%} & \num{23.4 \pm 0.5} & \num{702 \pm 14} \\
	\hline
	\end{tabular}
	\label{tab:blocks-test-benchmark}
\end{table}
Table~\ref{tab:blocks-test-benchmark} shows the grasp rate, object count and final \ac{PPH} for each combination of grasps with/without prediction and of single/multiple objects, in comparison to the heuristic of Calli et al. \cite{calli_benchmarking_2015}. Random grasps are evaluated by sampling randomly from the entire action space. We repeated every experiment \num{5} times. Our robot achieves a peak performance of \SI{23.5 \pm 0.5}{grasps} with \SI{3.3 \pm 0.2}{images}, corresponding to \SI{702 \pm 14}{PPH}. On average, using an approach with prediction improves the \ac{PPH} by \SI{15}{\%}.

\subsection{Planning}

We demonstrate the ability to plan ahead for the task of emptying a bin with as fewest actions as possible.
Given an object configuration of three cubes in a row at the side of the bin the robot is able to grasp all objects in \num{4} steps by shifting the middle cube first. If an outer cube is shifted otherwise, the robot needs at least \num{5} steps. In this configuration, planning can reduce the average number of action steps from \num{4.77 \pm 0.2} to \num{4.14 \pm 0.13}, measured by \num{25} successful episodes with a success rate of \SI{78}{\%}. Illustrative material can be found on the project website.

\subsection{Generalization}

We qualitatively evaluate the system for objects that were never seen during training (Fig.~\ref{fig:generalization-predictions}).
\begin{figure}[b]
	\centering
\newcommand{0.1\linewidth}{0.1\linewidth}
\newcommand\IncludeGraphic[2]{
    \includegraphics[width=0.1\linewidth]{figures/generalization/#1/#2}
}

\newcommand\AddColorRow[5]{
    \node[pred, below of=#3] (#2colorbefore) {\IncludeGraphic{#1}{bi_s_bef_color.png}};
	\node[pred, right of=#2colorbefore] (#2depthbefore) {\IncludeGraphic{#1}{bi_s_bef_depth.png}};
	\node (#2beforecenter) at ($(#2colorbefore)!0.5!(#2depthbefore)$) {};

	\node[left of=#2colorbefore, node distance=28] (#2labelcenter) {};
	\node[text width=24, align=right, above of=#2labelcenter, node distance=4] {\footnotesize #4};
	\node[text width=24, align=right, below of=#2labelcenter, node distance=6] {\footnotesize #5};

	\node[pred, right of=#2depthbefore, node distance=30] (#2colorafter) {\IncludeGraphic{#1}{bi_s_aft_color.png}};
	\node[pred, right of=#2colorafter] (#2depthafter) {\IncludeGraphic{#1}{bi_s_aft_depth.png}};
	\node (#2aftercenter) at ($(#2colorafter)!0.5!(#2depthafter)$) {};

	\node[pred, right of=#2depthafter, node distance=30] (#2colorpred) {\IncludeGraphic{#1}{bi_result_color.png}};
	\node[pred, right of=#2colorpred] (#2depthpred) {\IncludeGraphic{#1}{bi_result_depth.png}};
	\node (#2predcenter) at ($(#2colorpred)!0.5!(#2depthpred)$) {};

	\node[pred, right of=#2depthpred, node distance=30] (#2colorunc) {\IncludeGraphic{#1}{bi_unc_color.png}};
	\node[pred, right of=#2colorunc] (#2depthunc) {\IncludeGraphic{#1}{bi_unc_depth.png}};
	\node (#2unccenter) at ($(#2colorunc)!0.5!(#2depthunc)$) {};
}

\begin{tikzpicture}
    \tikzstyle{pred} = [node distance=0.1\linewidth]
    \tikzstyle{textbox} = [node distance=19]

    \node[] (cleft) {};

	\AddColorRow{2020-08-13-13-28-49-073}{1}{cleft}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddColorRow{2020-08-13-13-31-50-939}{2}{1colorbefore}{$r_{t} = 1$}{$p_{t} = 2$}
	\AddColorRow{2020-08-13-13-31-01-577}{3}{2colorbefore}{$r_{t} = 1$}{$p_{t} = 2$}
	\AddColorRow{2020-08-13-13-32-30-506}{4}{3colorbefore}{$r_{t} = 1$}{$p_{t} = 1$}
	\AddColorRow{2020-08-13-13-33-21-639}{5}{4colorbefore}{$r_{t} = 1$}{$p_{t} = 1$}

	\AddColorRow{2020-08-13-13-43-17-473}{7}{5colorbefore}{$r_{t} = 1$}{$p_{t} = 0$}
	\AddColorRow{2020-08-13-13-38-30-376}{8}{7colorbefore}{$r_{t} = 1$}{$p_{t} = 1$}

    \node[textbox, above of=1beforecenter] {\small $\hat{s}_{t}$};
	\node[textbox, above of=1aftercenter] {\small $\hat{s}_{t+1}$};
	\node[textbox, above of=1predcenter] {\small BicycleGAN};
	\node[textbox, above of=1unccenter] {\small Uncertainty};
\end{tikzpicture}

	\caption{Example predictions for novel (unknown) objects. The uncertainties are shown from low (blue) to high (red).}
	\label{fig:generalization-predictions}
\end{figure}
We find that our system is able to generalize well to compact objects, naturally depending on the similarity to trained objects. From the perspective of instance segmentation, this problem is very challenging for unknown objects, as the system needs to estimate the connection of objects. For example, the pincers (5th image) are only connected outside out of the image. For long and irregular objects, the model often splits single objects into multiple parts. However, it often captures the boundaries for other objects well, e.g. like the red object in the bottom row.

\section{DISCUSSION AND OUTLOOK}

We presented an approach to learn a generative transition model for robotic manipulation, for both the next visual state and its pixel-wise uncertainty. This way, the robot is able to plan sequences of single-step manipulation primitives, which were learned data-efficiently without sparse rewards. Furthermore, we demonstrated two more novel skills of our robot: By skipping image measurements, it was able to increase the \ac{PPH} by around \SI{15}{\%} in the Box and Blocks Test \cite{calli_benchmarking_2015}. Second, the robot was able to optimize regarding multiple-step criteria in a flexible way, e.g. to minimize the number of actions for emptying a bin. This becomes possible by three major contributions: First, we simplify the transition model by conditioning it on the already learned reward estimation as well as using the spatial invariance of the state space. Second, we propose a model architecture generating samples from a multi-modal distribution, allowing to estimate a prediction uncertainty. Third, the latter was propagated efficiently towards the estimated reward. \\

Our transition model works in a direct image-to-image setting similar to \cite{boots_2014_learning, finn2019foresight, ebert2018visual}. In comparison, our model additionally estimates a pixel-wise uncertainty measure and is applied to more difficult, obstacle-rich environments. From a broader perspective, we see several advantages in our approach: In comparison to model-based \ac{RL}, our transition model does not interact with the single-step policy during training and extends the robotic system \textit{optionally}. This allows to condition the transition model on the estimated reward and reuse the manipulation model learned with model-free \ac{RL}. Oftentimes, a transition model is used for model-based policy training to reduce sample complexity \cite{hafner2019learning}. However, we find that this is not useful for grasping: The following states depend strongly on the grasp success, which can be measured easily by the gripper and learned explicitly by a manipulation model. \\

Still, our approach allows for flexible optimization criteria and - even more important for real-world applications - off-policy \ac{RL} of time-dependent manipulation sequences. In this regard, we learned simple manipulation primitives similar to \cite{zeng_learning_2018}, but were able to combine multiple primitives to more complex action sequences. Although model errors are inevitable for robotic manipulation \cite{finn2019foresight, ebert2018visual}, our uncertainty-aware policy does \textit{actively} avoid these model limitations.
However, while our transition model is able to generate realistic images, we find that it fails to capture high-level features of the stochastic physics. Instead, it usually learns the simplest solution and provides low-level pixel uncertainties. \\

Eventually, our work leaves room for several improvements. So far, we have ignored the distribution shift between real and predicted images, and in particular its effect on the manipulation model. Despite the principle ability to generalize to unknown objects, we focused our bin picking evaluation on a few object types. We plan to further investigate the limits of generalization in manipulation experiments.
Regarding uncertainty estimation, eliminating our sample-based approach could accelerate the algorithm significantly. Moreover, we plan to integrate additional uncertainties, particularly of the manipulation \ac{NN}, into the policy.

\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


Bookmakers believe negotiations between Manchester United and Jose Mourinho have broken down.
The Portuguese tactician has been tipped to succeed Louis van Gaal at Manchester United next season.
However, Laurent Blanc's uncertain future has forced Betfair to slash their odds on Mourinho replacing him at Paris St-Germain from 16/1 to 5/2.
Michael Bowers, football trader for Betfair, said: "Betfair customers believe Jose Mourinho's long mooted move to Manchester is breaking down.
"There has been a deluge of bets coming for him to take the top job at PSG."
Chelsea have opened talks to sign Javier Pastore.
Website TMW claim incoming manager Antonio Conte would like to bring the Argentine playmaker to Stamford Bridge when he arrives after Euro 2016.
The Blues have already made contact with the player's representatives and hope he'll be tempted to leave the French capital.


As a healthcare provider, protecting your ability to request payment for services prior to a personal injury settlement is key!
For your patients, a personal injury attorney's first obligation is to his/her client. The attorney will try to secure the largest recovery for their client and have no obligation to pay outstanding medical bills or even notify a hospital that the case is settled. While the patient's attorney has no obligation in this regard, the medical bills incurred by the patient are often used as a benchmark to determine costs and negotiate the amount of settlement.
Audley Law Offices will protect your ability to obtain payment for services rendered before settlement funds have been awarded. Once the funds have been dispersed, it becomes almost impossible to collect.


Sand Moor is among the most prestigious clubs in Yorkshire, and is a treat for any golfer visiting Leeds. The local area is rich in golfing options which can be combined to form a fantastic golf tour - Alwoodley and Ilkley are among our recommended choices.
The great tradition of Sand Moor begins with the club's foundation in 1921, but the most notable development in its history is Dr Alister Mackenzie's involvement in revising the initial layout of the course. Mackenzie is revered by many as the finest course architect in the game's history, giving an insight into the quality of the par-71 layout at Sand Moor.
Well matured and extremely well established, Sand Moor is superb test of parkland golf that is used for some prestigious amateur and professional events - a young Nick Faldo is among those to win tournaments at this Leeds gem.


text:
UAE and Israeli Settlers Find Common Ground in Jerusalem
The Daily Observer
Weakened by Joe Biden's electoral defeat of US President Donald J Trump, Israeli Prime Minister Binyamin Netanyahu risks being caught between a rock and a hard place as Jordan, the Palestine Authority and the United Arab Emirates manoeuvre for control of the third most holy site in Islam. This is accordingly to Dr James M Dorsey, Senior Fellow at RSIS, in a commentary.
…Dr James M Dorsey is an award-winning journalist and a Senior Fellow at Nanyang Technological University's S Rajaratnam School of International Studies in Singapore and the National University of Singapore's Middle East Institute.
Weakened by Joe Biden's electoral defeat of US President Donald J Trump, Israeli Prime Minister Binyamin Netanyahu risks being caught between a rock an ...


text:
package org.givwenzen;

/**
 * This class is not safe for running multiple tests in the same JVM simultaneously.
 */
public final class GWZForJUnit {

   private static Object unitTest;
   private static GivWenZen gwz = null;

   public static void setUp(Object unitTest) {
      GWZForJUnit.unitTest = unitTest;
      gwz = null;
   }

   public static Object and(String methodString) throws Exception {
      return getGWZ().and(methodString);
   }

   public static Object given(String methodString) throws Exception {
      return getGWZ().and(methodString);
   }

   public static Object then(String methodString) throws Exception {
      return getGWZ().and(methodString);
   }

   public static Object when(String methodString) throws Exception {
      return getGWZ().and(methodString);
   }

   private static GivWenZen getGWZ() {
      String basePackageForSteps = DomainStepFinder.DEFAULT_STEP_PACKAGE;
      if (unitTest != null)
         basePackageForSteps += "," + unitTest.getClass().getPackage().getName() + ".";
      if (gwz == null)
         gwz = GivWenZenExecutorCreator.instance()
         .stepClassBasePackage(basePackageForSteps)
         .create();
      return gwz;
   }

}

redpajama_set_name: RedPajamaGithub


nomination of the annual TOP-50 successful women in business rating by the BUSINESS magazine. Ms.
and dried egg products in both Ukraine and the whole of Europe.
Ukrlandfarming group owned by Mr. Oleg Bakhmatiuk.


Straight away in this post, you will be catching with some of the extra brilliant and mind-blowing intermingling of the finest and latest new trends of wood pallet shipping ideas. Bringing the attractive taste in the house decoration is the utmost wish of each single person. But giving the house with the blend of attractiveness and uniqueness is the ultimate wish of some of the people. But you do not have to wish only, because wood shipping pallet recycling will turn your this dream into reality by introducing some of the amazing and worth mentioning DIY projects to repurpose old shipping pallets! Get ready to learn here!
Arrangement of the table set by using wood pallet working in it will always come out to be the main center of attraction for the guests. This wood pallet table is super cool and fantastic in appearance that has been all encountered with the rustic dark wood pallet effect in it.
No matter whether it is about the designing of the headboard, the use of wood pallet has always remained the main want of the house makers. This is a giant structure of the wood pallet headboard with the effect of lights where the dramatic use of the wood has been customized out.
To add an outstanding appearance in the garden areas, choosing the concept of amazing wood pallet bench design is the perfect option for you. You would be finding this bench piece as much artful and creative looking for sure. See the image to get some idea!
This is much a creative creation of the wood pallet that is being finished out in the playful playhouse shaping work. Hut style of wood pallet creation is put together in the pleasant variations which you can even dramatic choose for the house as it look classy and cool.
If your house garden is comprised with the large space settings, then holding up the perfect arrangement of the wood pallet amazing furniture of table and benches set is ideal idea for you. This whole furniture set has been complimentary in designing that looks amazing.
Wood pallet material is currently much in latest trend of use these days for the creation of the cabinet for your house beauty. This image will make you show out the perfect use of the wood pallet in the amazing cabinet structure that looks so unique.
You can even put together the ideal use of the wood pallet in the superb creation of the chair design work too. This chair designing is shaped in the wood pallet excellent functioning use. It is moderate in framing and much in light in weight to make it locate in the house.
Having a crafting unique set in the house is the best idea besides putting yourself in the hard times of keeping a giant furniture decoration. This creative wood pallet creation has been splendid designed in the shaping structure of the man that is giving a unique look.
Check out this interestingly designed planter box for the garden decoration purposes! Isn't it looking so amazing and lovely? This planter box has been all put together in the simple designing blend of the storage box rectangular designing to add in your house.
This innovative wood pallet creation is basically taken as the vegetable rack for your house kitchen areas. This creation has been finished with the arrangement assembling of the planks in vertical concepts. It turns out to be the attractive piece because of the designing work done on top of it.
Designing of the wood pallet wall shelf piece is considered to be one of the most prominent furniture item in most of the houses. Such style of the wall shelf designs mostly involve the collection of planks where assembling is all done in the vertical formations.
How about creating a piece of creation of cage or the box in the house garden with the wood pallet use in it? Well, it sounds interesting but having such creations are not just meant for keeping the pet animals but it can even act out as the garden decoration piece work too.
Welcome the guests coming to your home in a warm impression through the placement of this amazing wood pallet creation idea. This creation is basically about the home benches and table furniture artwork that is looking so artistic and rough in the designing concepts.
What a breath-taking creation of wood pallet kitchen shelf has been featured in this image! This creation is basically surrounded with the shelving placed on top where you can arrange important kitchen accessories. This will give out a pleasant appearance to the house kitchen.
Give your house outdoor location with a complete perfection taste through the addition of furniture designs of wood pallet in it. As you can clearly view out in this image, wood pallet benches and table are stylish created in a sophisticated concepts.
Grab this wood pallet stylish shelving stand for your house lounge area! Simple hues of designing has been featured out in this wood pallet stand. This wood pallet stand has been created with the vertical placement of the wood pallet planks.
Wood pallet designs are not just meant for your house furniture only. They can perfectly be used in order to carry out the designing of the wall LED holder designs too. Yes, we are talking about this adorable and lovely designed wood pallet LED holder design.
Furniture set designs do ranges from the simple designs to the intricate designs as well. If you want to add your living room with the simple wood pallet furniture creations, then choosing out with this decent design is best out for you. Let's catch it now!
This is another stylish creation of the wood pallet door for you! When it comes to the complimentary designing of the wood pallet for the door styles, then having a low height table is the excellent option for you. Look for artistically pattern designing has been done on top of it!
This wood pallet project is eventually taken as best to use as the decoration piece for your living room beauty. This is a modish recycling of the end table with drawers effect. You can arrange this creation for your living room where you can put some decoration pieces on top and complete its whole look.
This is although a small ideal creation of the wood pallet bench, but the unique part of this bench has been the colorful cushion pieces clothing over it. It is light in weight which you can easily make it move at any area of the house. It can take out to be a perfect alternative as for the guest seating arrangement.
This Adirondack chair piece has been all the more coming out as the perfect piece to be the part of the home furniture category. The chair has been ended up with the crafting in the leaning down effect backrest that is turning out to be one of its main attraction.
This end table design would be giving out the impression of the decoration table, but it much effective and purposeful. It has been all arranged with the simple crafting through the rustic wood pallet use in it.
This wood pallet creation has been somehow turning out to be so mesmerizing and much charming looking with the round table effects being part of it. This creation is added with the three round effects that is bringing much eye catching and attractive impact in it.


text: curl http://localhost:8080/metrics/find?query=*