text:
Chiappa Rhino 60DS: The futuristic pimp's choice (VIDEO)
The technology behind revolvers has remained virtually the same for the last century. Sure, there have been improvements in metallurgy, allowing for larger calibers and improved iron sights, but the mechanics of the design are unchanged. On a current Smith & Wesson revolver, the three main components — frame, cylinder and barrel — and the way they interact is the same as a revolver made 100 years ago. Of course there are exceptions, such as the Mateba MTR-8 and 6 Unica, but, for the most part, they aren't commercially successful and are astronomically expensive.
What's a guy to do if he wants a one of a kind revolver without paying over $2,000? Grab a Chiappa Rhino.
Originally imported for CCW with a 2-inch barrel, shooters can now purchase 4-, 5- and 6-inch variants. The latter two are the focus of this review. I planned to review a practical, traditional black finish Rhino and stainless Rhino. That is until SHOT Show where I discovered Chiappa offered a gold Rhino.
It's not normally my style to rock a futuristic gold revolver with a 6-inch barrel, but it was too awesome to pass up. Some may denounce the modernist appearance of the Rhino revolvers, but I'm a huge fan and I feel their unique appearance gives them character, something sorely lacking in many contemporary designs.
Why does the Rhino feature that iconic pronounced proboscis? It fires from the bottom cylinder. This lowers the bore angle to better align with the shooter's arm and reduce felt recoil, much like 1911 shooters that adopt a higher grip. While It definitely soaks up recoil from hefty .357 Magnum loads, the trigger's weight and smooth face combined with the grip's size and construction hold it back from sweeping the revolver market.
Heavy and a bit unmanageable for fast action shooting, the Rhino's trigger-pull is a hefty 10 pounds in double-action. The trigger face is extra wide and glass-smooth. In single-action shooting, where the user is completing the single action of dropping the hammer, these aspects don't matter. However, in double-action shooting, where the user is completing both actions of cocking the hammer and dropping it, these aspects become a drawback.
I love Gooooold. (Photo by Jim Grant)
A wide trigger, like the ones found on Rhino revolvers dissipate the force exerted by the trigger finger, making it more comfortable, but requiring more effort. When trying to manipulate an already heavy trigger over a long distance, any additional resistance will tire out the majority of shooters after a dozen rounds. Consequently, for fast double-action/combat shooting, a narrow trigger is preferred, because it decreases the amount of force necessary to manipulate it.
An old school of thought concerning revolver-shooting was that double-action was a waste of ammo and only utilized in an emergency. Therefore, the extra-wide trigger and its predecessor, the archaic trigger-shoe, were designed to make single-action shooting easier. For double-action triggers, especially those with multiple bearing surfaces like the Rhino, this wide trigger is a downside, because it hinders fast, accurate double-action shooting. Especially in conjunction with the long reset of the Rhino's trigger.
For the vast majority of shooters, this doesn't matter. Look at anyone on the range with a double/single-action revolver that isn't used as a carry gun, 90 percent of people are cocking the hammer before shooting. Although it's more accurate for most sportsmen, single-action shooting is an inferior way to use a revolver for CCW and self-defense. It's slower and requires fine motor skills — Miculek not included.
In single-action the Rhino truly shines. The wide, comfortable trigger and low-bore axis make the odd-shaped six-gun more accurate than most shooters. Firing from a rest at 10 yards, I was able to get four out of five rounds touching on a regular basis.
The sights on both models are excellent for precision and low-light shooting. The 5-inch 50DS features a fiber optic front sight and black serrated partridge style rear sights. The 6-inch 60DS uses two different-colored fiber optic front and rear sights.
The integral top rail on the 60DS model accommodates a boundless array of optical sights and the bottom rails on both models allow for flashlights, laser-sights or tiny bayonets. This makes both models extremely versatile in terms of modern accessories, especially when compared to standard revolvers.
The Rhino falls just short of greatness in the design of the grip. The angle is functional, but odd and the shape is ill-suited to double-action shooting, especially if you have small hands. This is due to the beveled front strap of the grip. The rounded edge needs to be more aggressively de-horned, especially when coupled with wooden grips and .357 Magnum loads. During testing I found that the recoil of the Rhino was perfectly acceptable in terms of how the pistol interacted with my wrist and palm, but mercilessly pounded on my index finger. I assume rubber grips would do much to alleviate this, but were absent on the review models.
The Rhino 50DS looks dashing with its chrome finish. (Photo by Jim Grant)
The 50DS and 60DS ran circles around my automatics in terms of accuracy and were on par with my Smith & Wesson performance revolver. I'm able to place multiple rounds that were touching in every group at 30 feet. If you do you part, the Rhino will hit where you aim — even though its ergonomics will fight you the entire time. The Rhino's biggest issues arise from the fact that its designer employed a trigger meant exclusively for single-action target shooting, not double-action combat shooting. Additionally, when a handgun's main selling point is its ability to mitigate recoil, wooden grips are a poor (read: painful) choice. Nevertheless, it's not all bad. The low axis bore and forward angle grip undoubtedly help soak up the stout recoil of a .357 Magnum, despite the alloy-frame's light weight.
The quality of the materials used, combined with the historically profitable business of buying any of Emilio Ghisoni's designs means that buyers won't lose money on the Rhino. In other words, there is no depreciation in value. With the Rhino you certainly get what you pay for, even if the design isn't for everyone.
Comparing the Rhino's aftermarket support to that of a traditional six-gun presents a bit of a problem. Smith & Wesson revolvers have been around for over a century, so the Rhino has a lot of catching up to do. If the Rhino becomes more popular, its aftermarket selection will grow. As it stands, it's fairly limited. There are enough holsters to satisfy most shooters, but the grip selection is quite slim. On the other hand, every sight you could ever wish for is available for the Rhino, including both magnified and electronic ones. These Chiappa revolvers are so close to perfect and having a few facets of the design derail the entire weapon's reputation is incredibly frustrating. As it currently ships, the Rhinos need some tweaking to be a suitable replacement for traditional DA/SA revolvers. Namely, the ergonomics of the trigger width and the grips.
The 50DS and 60DS Rhinos have a tremendous amount of potential and if the small hiccups were fixed, I would definitely add one to my wheel gun collection.


text:
\section{Introduction}
Recent years witness a rapid development of machine learning and its succeeded applications such as computer vision~\cite{ma2018variational,zhu2019image,ma2019shoe}, natural language processing~\cite{ma2013vector,ma2018short} and cross-modalities learning~\cite{zhu2019image,xu2018cross} with many real-world applications~\cite{xie2018mobile,ma2019fine}. Traditional machine learning methods are typically based on the assumption that training and testing datasets are from the same distribution. However, in many real-world applications, this assumption may not hold, and the performance could degrade rapidly if the trained models are deployed to domains different from the training dataset~\cite{ganin2016domain}. More severely, to train a high-performance vision system requires a large amount of labelled data, and getting such labels may be expensive.
 Taking a pre-trained robotic vision system as an example, during each deployment task, the robot itself (\emph{e.g.} position and angle), the environment (\emph{e.g.} weather and illumination) and the camera (\emph{e.g.} resolution) may result in different image styles. The cost to annotate enough data for each deployment task could be very expensive.

This kind of problem has been widely addressed by transfer learning (TL)~\cite{zhuang2019comprehensive} and domain adaptation (DA)~\cite{ganin2016domain}. In DA, a learner usually has access to the labelled source data and unlabelled target data, and it is typically trained to align the feature distribution between the source and target domain. However, sometimes, we could not expect the target data is accessible for the learner.
In the robot example, the distribution divergences (different image styles) from training to testing domain can only be identified after the model is trained and deployed.
In this scenario, it's unrealistic to collect samples before deployment. This would require a robot to have abilities to handle domain divergences even though the target data is absent.

We tackle this kind of problem under domain generalization (DG) paradigm, under which the learner has access to many source domains (data and corresponding labels), and aims at generalizing to the new (target) domain, where both data and labels are unknown.
The goal of DG is to learn a prediction model on training data from the seen source domains so that it can generalize well on the unseen target domain.
An underlying assumption behind domain generalization is that there exists a common feature space underlying the multiple known source domains and unseen target domain. Specifically, we want to learn domain invariant features across these source domains, and then generalize to a new domain.
An example of how domain generalization is processed is illustrated in Fig.\ref{fig:DG}.

\begin{figure}
    \centering
    \includegraphics[width=0.99\textwidth]{figs/DG.pdf}
    \caption{Domain Generalization: A learner faces a set labelled data from several source domains, and it aims at extracting invariant features across the seen source domains and learn to generalize to an unseen domain. Based on the manifold assumption~\cite{goldberg2009multi}, each domain $i$ is supported by distribution $\mathcal{D}_i$. The learner can measure the source domain distribution via the source datasets but has no information on the unseen target distribution. After training on the source domains, the model is then deployed to a new domain $\mathcal{D}_t$ for prediction.}
    \label{fig:DG}
\end{figure}

 A critical problem in DG and DA involves aligning the domain distributions, which typically are achieved by extracting such representations.
 Previous DA works usually tried to minimize the domain discrepancies, such as KL-divergence and Maximum Mean Discrepancy (MMD) etc. via adversarial training, to achieve domain distribution alignments. Due to the similar problem setting between DA and DG, many previous approaches directly adopt the same adversarial training technique for DG. For example, a MMD metric is adopted by~\cite{li2018domain} as a cross-domain regularizer and KL divergence is adopted to measure the domain shift by~\cite{Li2017dg} for domain generalization problem.
 The MMD metric is usually implemented in kernel space, which is not sufficient for large-scaled applications, and KL divergence is unbounded, which is also insufficient for a successful measuring domain shift~\cite{zhao2019learning}.

 Besides, previous domain generalization approaches~\cite{ilse2019diva,ghifary2015domain,li2018deep,d2018domain,volpi2018generalizing} mainly focused on applying similar DA technique to extract the invariant features and how to stack the learned features from each domain for generalizing to a new domain. These methods usually ignore the label information and will sometimes make the features became indistinguishable with ambiguous classification boundaries, $a.k.a$ semantic misalignment problem~\cite{8964455}.
  A successful generalization should guide the learner not only to align the feature distributions between each domain but also to discriminate the samples in the same class could lie close to each other while samples from different classes could stay apart from each other, $a.k.a.$ feature compactness~\cite{kamnitsas2018semi}.

Aiming to solve this, we adopt Optimal Transport (OT) with Wasserstein distance to align the feature distribution for domain generalization since it could constrain labelled source samples of the same class to remain close during the transportation process~\cite{courty2016optimal}. Moreover, some information theoretical metrics such as KL divergence is not capable to measure the inherent geometric relations among the different domains~\cite{arjovsky2017wasserstein}. In contrast, OT can exactly measure their corresponding geometry properties. Besides, compared with~\cite{ben-david2010}, OT benefits from the advantages of Wasserstein distance by its gradient property~\cite{arjovsky2017wasserstein} and the promising generalization bound~\cite{redko2017theoretical}. The empirical studies~\cite{gulrajani2017improved,shen2017wasserstein} also demonstrated the effectiveness of OT for extracting the invariant features to align the marginal distributions of different domains.

Furthermore, although the optimal transport process could constrain the labelled samples of the same class to stay close to each other, our preliminary results showed that just implementing optimal transport for domain generalization is not sufficient for a cohesion and separable classification boundary. The model could still suffer from indistinguishable features (see Fig.~\ref{fig:sub-third}). In order to train the model to predict well on all the domains, this separable classification boundary should also be achieved under a domain-agnostic manner. That is, for a pair of instances, no matter which domain they come from, they should stay close to each other if they are in the same class and vice-versa. To this end, we further promote metric learning as an auxiliary objective for leveraging the source domain label information for a domain-independent distinguishable classification boundary.

To summarize, we deployed the optimal transport technique with Wasserstein distance for domain generalization for extracting the domain invariant features. To avoid ambiguous classification boundary, we proposed to implement metric learning strategies to achieve a distinguishable feature space. Theorefore, we proposed the Wasserstein Adversarial Domain Generalization (\emph{WADG}) algorithm.

In order to check the effectiveness of the proposed approach, we tested the algorithm on two benchmarks comparing with some recent domain generalization baselines.
The experiment results showed that our proposed algorithm could outperform most of the baselines, which confirms the effectiveness of our proposed algorithm. Furthermore, the ablation studies also demonstrated the contributions of our algorithm.

\section{Related Works}

\subsection{Domain Generalization}\noindent

 The goal of DG is to learn a model that can extract common knowledge that is shared across source domains and generalize well on the target domain. Compare with DA, the main challenge of DG is that the target domain data is not available during the learning process.

A common framework for DG is to extract the most informative and transferable underlying common features from source instances generated from different distributions and to generalize to unseen one.
This kind of approach holds with the assumption that there exists an underlying invariant feature distribution among all domains, and that consequently such invariant features can generalize well to a target domain.~\cite{muandet2013domain} implemented MMD as
a distribution regularizer and proposed the kernel-based \emph{Domain Invariant Component Analysis} (DICA) algorithm.
An autoencoder-based model was proposed by~\cite{ghifary2015domain} under a multi-task learning setting to learn domain-invariant features via adversarial training.
\cite{li2018deep} proposed an end-to-end deep domain generalization approach by leveraging deep neural networks for domain-invariant representation learning.
\cite{Motiian_2017_ICCV} proposed to minimize the semantic alignment loss as well as the separation loss based on deep learning models.
\cite{li2018domain} proposed a low-rank Convolutional Neural Network model based on domain shift-robust deep learning methods.

 There are also some approaches to tackle the domain generalization problems in a meta-learning manner. To the best of our knowledge,~\cite{li2018learning} first proposed to adopt the Meta Agnostic Meta-Learning (MAML)~\cite{finn2017model} which back-propagates the gradients of ordinary loss function of meta-test tasks. As pointed by~\cite{dou2019domain}, such an approach might lead to a sub-optimal solution, as it is highly abstracted from the feature representations. ~\cite{balaji2018metareg} proposed \emph{MetaReg} algorithm in which a regularization function ($e.g.$ weighted $L_1$ loss) is implemented for the classification layer of the model but not for the feature extractor layers. Then,~\cite{li2019feature} proposes an auxiliary meta loss which is gained based on the feature extractor. Furthermore, the network architecture of~\cite{li2019feature} is the widely used feature-critic style model based on a similar model from domain adversarial training technique~\cite{ganin2016domain}.~\cite{dou2019domain} and~\cite{dg_mmld} also started to implement clustering techniques on the invariant feature space for better classification and showed better performance on the target domain.

\subsection{Metric Learning}
\label{sect:related_works_metric_learning}
Metric learning aims to learn a discriminative feature embedding where similar samples are closer while different samples are further apart~\cite{8964455}.
\cite{hadsell2006dimensionality} proposed the \emph{siamese network} together with \emph{contrastive loss} to guide the instances stay close with each other in the feature space if they have the same labels and push them apart vice-versa.
\cite{schroff2015facenet} proposed the \emph{triplet loss} aiming to learn a feature space where a positive pair has higher similarity than the negative pair when comparing by the same anchor with a given margin.
\cite{oh2016deep} showed that neither the \emph{contrastive loss} nor  \emph{triplet loss} could efficiently explore the full pair-wise relations between instances under the mini-batch training setting. They further propose the \emph{lifted structure loss} to fully utilize pair-wise relations across batches. However, it only choose equal number of positive pairs as negative ones randomly, and many informative pairs are discarded~\cite{wang2019multi}, which restricts the ability of finding the informative pairs.
~\cite{yi2014deep} proposed the binomial deviance loss which could measure the hard pairs.
One remarkable work by~\cite{wang2019multi} combines the advantages both from~\emph{lifted structure loss} and \emph{binomial loss} to leverage the pair-similarity.
They proposed to leverage not only pair-similarities (positive or negative pairs with each other) but also self-similarity which enables the learner to collect and weight informative pairs (positive or negative pairs) under an iterative (mining and weighting) manner. For a pair of instances, the self-similarity is gained from itself.
Such a multi-similarity has been shown could measure the similarity and could cluster the samplers more efficiently and accurately.
In the context of domain generalization, ~\cite{dou2019domain} proposed to guide the learner to leverage from the local similarity in the semantic feature space, in which the authors argued may contain essential domain-independent \emph{general knowledge} for domain generalization and adopt the constrative loss and triplet loss to encourage the clustering for solving this issue. Leveraging from the across-domain class similarity information can encourage the learner to extract robust semantic features that regardless of domains, which is an useful auxiliary information for the learner. If the learner could not separate the samples (from different source domains) with domain-independent class-specific cohesion and separation on the domain invariant feature space, it would still suffer from ambiguous decision boundaries. This ambiguous decision boundaries might still be sensitive to the unseen target domain.~\cite{dg_mmld} implement unsupervised clustering on source domains and showed better classification performance. Our work is orthogonal to previous works, proposing to enforce more distinguishable invariant features space via Wasserstein adversarial training and encouraging to leverage from label similarity information for better classification boundary.

\section{Preliminaries and Problem Setup}
We start by introducing some preliminaries.

\subsection{Notations and Definitions}
Following~\cite{redko2017theoretical} and \cite{Li2017dg}, suppose we have $m$ known source domains distributions $\{\mathcal{D}_i\}_{i=1}^m$, and $i^{th}$ domain contains $N_i$ labeled instances in total, denoted by $\{(\mathbf{x}^{(i)}_j,y^{(i)}_j)\}^{N_i}_{j=1}$, where $\mathbf{x}^{(i)}_j\in\mathbb{R}^{n}$ is the $j^{th}$ instance feature from the $i^{th}$ domain and $y^{(i)}_j\in \{1,\dots,K\}$ are the corresponding labels.
For a hypothesis class $\mathcal{H}$, the expected source and target risk of a hypothesis $h\in\mathcal{H}$ over domain distribution $\mathcal{D}_i$ is the probabilities that $h$ wrongly predicts on the entire distribution $\mathcal{D}_i$:
        $\epsilon_i(h)=\mathbb{E}_{(\mathbf{x},y)\sim\mathcal{D}_i}\ell(h(\mathbf{x},y))$,
where $\ell(\cdot)$ is the loss function. The empirical loss is also defined by: $\hat{\epsilon}_i(h)=\frac{1}{N_i}\sum_{j=1}^{N_i}\ell(h(\mathbf{x}_j,y_j))$.

In the setting of domain generalization, we only have the access to the seen source domains $\mathcal{D}_i$ but have no information about the target domain. The learner is expected to extract the underlying invariant feature space across the source domains and generalize to a new target domain.

\subsection{Optimal Transport and Wasserstein Distance}
\begin{figure}
		\centering
\includegraphics[width=0.60\textwidth]{figs/OT_for_DG.pdf}
\caption{Use optimal transport (OT) for domain generalization: Typically to directly predict on the unseen domain (the white dashed arrow) is difficult. In order to learn domain invariant features, as showed in the direction of the green arrow we adopted the OT technique to achieve domain alignments for extracting invariant features. After the OT transition, the invariant features can be generalized to unseen domain.} \label{fig_ot_dg}
\end{figure}
We follow \cite{redko2017theoretical} and define $c:\mathbb{R}^n\times\mathbb{R}^n \to \mathbb{R}^{+}$ as the cost function for transporting one unit of mass $\mathbf{x}$ to $\mathbf{x}'$, then Wasserstein distance between $\mathcal{D}_i$ and $\mathcal{D}_j$ could be computed by,
\begin{equation}
W_p^p(\mathcal{D}_i,\mathcal{D}_j) = \inf_{\gamma\in \Pi(\mathcal{D}_i,\mathcal{D}_j)} \int_{\mathbb{R}^n\times\mathbb{R}^n}c(\mathbf{x},\mathbf{x}^\prime)^p d\gamma(\mathbf{x},\mathbf{x}^\prime)
\end{equation}
where $\Pi(\mathcal{D}_i,\mathcal{D}_j)$ is the probability coupling on $\mathbb{R}^n\times\mathbb{R}^n$ with marginals $\mathcal{D}_i$ and $\mathcal{D}_j$ referring to all the possible coupling functions. Throughout this paper, we adopt Wasserstein-1 distance only ($p=1$).
Let $f$ be a Lipschitz-continuous function $\|f\|_L \leq 1$, according to \emph{Kantorovich-Rubinstein} theorem,  we have
\begin{equation}
\label{original_eq_wasserstein}
    W_1(\mathcal{D}_i,\mathcal{D}_j) = \sup_{\|f\|_L < 1} \mathbb{E}_{x\in\mathcal{D}_i}f(x) - \mathbb{E}_{x^\prime \in \mathcal{D}_j}f(x^\prime)
\end{equation}

Optimal transport theory and Wasserstein distance were recently investigated in the context of machine learning \cite{arjovsky2017wasserstein} especially in the domain adaptation area \cite{courty2016optimal,zhou2020discriminative}. The general idea of implementing the optimal transport technique for domain generalization across domains is illustrated in Fig.~\ref{fig_ot_dg}. To learn domain invariant features, OT technique is implemented to achieve domain alignments for extracting invariant features. After the OT transition, the invariant features can be generalized to unseen domain.

\subsection{Metric Learning}
 For a pair of instances $(\mathbf{x}_i,y_i)$ and $(\mathbf{x}_j,y_j)$, the notion of \emph{positive pairs} usually refers to the condition where pair $i,j$ have same labels ($y_i=y_j$), while the negative pairs usually refers to the condition $y_i\neq y_j$. The central idea of metric learning is to encourage a pair of instances who have the same labels to be closer, and push negative pairs to be apart from each other~\cite{wu2017sampling}.

Follow the framework of~\cite{wang2019multi}, we show the general pair-weighting process of metric learning. Assuming the feature extractor $f$ parameterized by $\boldsymbol{\theta}_f$ projects the instance $\mathbf{x} \in\mathbb{R}^{n}$ to a $d$-dimensional normalized space: $f(\mathbf{x};\boldsymbol{\theta}_f):\; \mathbb{R}^n\to [0,1]^d$. Then, for two samples $\mathbf{x}_i$ and $\mathbf{x}_j$, the similarity between them could be defined as the inner product of the corresponding feature vector:
\begin{equation}
    S_{i,j}:=\langle f(\mathbf{x}_i;\boldsymbol{\theta}_f),f(\mathbf{x}_j;\boldsymbol{\theta}_f)\rangle
    \label{Eq.dotsimilarity}
\end{equation}
To leverage the across-domain class similarity information can encourage the learner to extract the classification boundary that regardless of domains, which is an useful auxiliary information for the learner. We further elaborate it in section~\ref{metric_learning_domain_agnostic}.

\section{Proposed Method}
The high-level idea of WADG algorithm is to learn a domain-invariant feature space and domain-agnostic classification boundary. Firstly, we align the marginal distribution of different source domains via optimal transport by minimizing the Wasserstein distance to achieve the domain-invariant feature space. And then, we adopt metric learning objective to guide the learner to leverage the class similarity information for a better classification boundary. A general workflow of our method is illustrated in Fig.~\ref{fig:sub-general_workflow}. The model contains three major parts: a feature extractor, a classifier and a critic function.

The feature extractor function $F$, parameterized by $\boldsymbol{\theta}_f$, extracts the features from different source domain. For set of instances $\mathbf{X}^{(i)}=\{\mathbf{x}_{j}^{(i)}\}_{j=1}^{N_i}$ from domain $\mathcal{D}_i$, we can then denote the extracted feature from domain $i$ as $\mathbf{Z}^{(i)}=F(\mathbf{X}^{(i)})$. The classification function $C$, parameterized by $\boldsymbol{\theta}_c$, is expected to learn to predict labels of instances from all the domains correctly. The critic function $D$, parameterized by $\boldsymbol{\theta}_d$, aims to measure the empirical Wasserstein distance between features from a pair of source domains. For the target domain, all the instances and labels are absent during the training time.

WADG aims to learn the domain-agnostic features with distinguishable classification boundary. During each train round, the network receives the labelled data from all domains and train the classifier under a supervised mode with the classification loss $\mathcal{L}_C$. For the classification process, we use the typical cross-entropy loss for all $m$ source domains:
 \begin{equation}
    \mathcal{L}_{C} = -\sum_{i=1}^m\sum_{j=1}^{N_i} y_j\log(\mathbb{P}(C(F(\textbf{x}_j^{(i)}))))
    \label{Eq.cls_loss_all_domains}
 \end{equation}

Through this, the model could learn to train the category information on over all the domains.
The feature extractor $F$ is then trained to minimize the estimated Wasserstein Distance in an adversarial manner with the critic $D$ with an objective $\mathcal{L}_D$. We then adopt a metric learning objective (namely, $\mathcal{L}_{MS}$) for leveraging the similarities for a better classification boundary. Our full method then solve the joint loss function,
\begin{equation*}
\mathcal{L} = \arg\min_{\theta_f,\theta_c}\max_{\theta_d} \mathcal{L}_{C}+\mathcal{L}_{D} + \mathcal{L}_{MS},
\label{Eq:full_objective}
\end{equation*}
where $\mathcal{L}_D$ is the adversarial objective function, and $\mathcal{L}_{MS}$ is the metric learning objective function. In the sequel, we will elaborate these two objectives in section~\ref{wasserstien_over_all_domains} and section~\ref{metric_learning_domain_agnostic}, respectively.

\subsection{Adversarial Domain Generalization via Optimal Transport}
\label{wasserstien_over_all_domains}

As optimal transport could constrain labelled source samples of the same class to remain close during the transportation process~\cite{courty2016optimal}. We deploy optimal transport with Wasserstein distance~\cite{redko2017theoretical,shen2017wasserstein} for aligning the marginal feature distribution over all the source domains.

A brief workflow of the optimal transport for a pair of sourcce domains is illustrated in Fig.~\ref{fig:sub-OT alignment}. The critic function $D$ estimates the empirical Wasserstein Distance between the each source domain through a pair of instances from the empirical sets $\mathbf{x}^{(i)}\in\mathbf{X}^{(i)}$ and $\mathbf{x}^{(j)}\in\mathbf{X}^{(j)}$. In practice~\cite{shen2017wasserstein}, the original Eq.~\ref{original_eq_wasserstein} of Wasserstein distance could be computed by,
\begin{equation}
    \begin{split}
            W_1(\mathbf{X}^{(i)}, \mathbf{X}^{(j)})=\frac{1}{N_{i}}\sum_{\mathbf{x}^{(i)}\in\mathbf{X}^{(i)}}D(F(\mathbf{x}^{(i)}))         -\frac{1}{N_{j}}\sum_{\mathbf{x}^{(j)}\in\mathbf{X}^{(j)}}D(F(\mathbf{x}^{(j)}))
            \label{Eq.Wasserstein}
    \end{split}
\end{equation}
As in domain generalization setting, there usually exists more that two source domains, we can sum all the empirical Wasserstein distance between each pair of source domains,
\begin{equation}
    \mathcal{L}_D = \sum_{i=1}^m\sum_{j=i+1}^m\big [ \frac{1}{N_i}\sum_{\mathbf{x}^{(i)}\in\mathbf{X}^{(i)}}D(F(\mathbf{x}^{(i)}))-\frac{1}{N_j}\sum_{\mathbf{x}^{(j)}\in\mathbf{X}^{(j)}}D(F(\mathbf{x}^{(j)}))\big]
    \label{Eq.mult_domain_Wa}
\end{equation}
\begin{figure}
\centering
\begin{subfigure}{.535\textwidth}
  \centering

  \includegraphics[width=\textwidth]{figs/Neuro_Computing_model_framework.pdf}
  \caption{The whole workflow the proposed WADG model.}
  \label{fig:sub-general_workflow}
\end{subfigure}
\;
\begin{subfigure}{.43\textwidth}
  \centering
  \includegraphics[width=\textwidth]{figs/adv_alignment.pdf}
  \caption{Optimal Transport for Feature Alignment.}
  \label{fig:sub-OT alignment}
\end{subfigure}\\
\begin{subfigure}{1.1\textwidth}
  \centering

  \includegraphics[width=\textwidth]{figs/metric_learning.pdf}
  \caption{Metric Learning for Clustering Proces}
  \label{fig:sub-metric learning}
\end{subfigure}

\caption{The proposed WADG method. (a): the general workflow of WADG method. The model mainly consists of three parts, the feature extractor, classifier and critic function. During training, the model receives all the source domains. The feature extractor is trained to learn invariant features together with the critic function in an adversarial manner. (b): For each pair of source domains $\mathcal{D}_i$ and $\mathcal{D}_j$, optimal transport process for aligning the features from different domains. (c): The metric learning process. For a batch of all source domain instances, we first roughly mining the positive and negative pairs via Eq.~\ref{eq_round_mining}. Then, compute the corresponding weights via Eq.~\ref{Eq.negative_pair_weight} and Eq.~\ref{Eq.positive_pair_weight} to compute $\mathcal{L}_{MS}$ to guide the clustering process.}
\label{fig:whole workflow}
\end{figure}

Throughout this pair-wise optimal transport process, the learner could extract a domain-invariant feature space, we then propose to apply metric learning approaches to leverage the class label similarity for domain independent clustering feature extraction. We then introduce the metric learning for domain agnostic clustering in the next section.

\subsection{Metric Learning for Domain Agnostic Classification Boundary}
\label{metric_learning_domain_agnostic}
As aforementioned, only aligning the marginal features via adversarial training is not sufficient for DG since there may exist a ambiguous decision boundary~\cite{dou2019domain}. When predicting on the target domain, the learner may still suffer from this ambiguous decision boundary. To solve this, we propose to implement the metric learning techniques to help cluster the instances and promote a better prediction boundary for better generalization.

To solve this, except to the supervised source classification and alignment of the marginal distribution across domains with the Wasserstein adversarial training defined above, we then further encourage robust domain-independent local clustering via leverage from label information using the metric learning objective. The brief workflow is illustrated in Fig.~\ref{fig:sub-metric learning}. Specifically, we adopt the metric learning objective to require the images regardless of their domains to follow the two aspects: 1) images from the same class are semantically similar, thereby should be mapped nearby in the embedding space (semantic clustering), while 2) instances from different classes should be mapped apart from each other in embedding space. Since goal of domain generalization aims to learn to hypothesis could predict well on all the domains, the clustering should also be achieved under a domain-agnostic manner

To this end, we mix the instances from all the source domains together and encourage the clustering for domain agnostic features via the metric learning techniques to achieve a domain-independent clustering decision boundary. For this, during each training iteration, for a batch $\{\mathbf{x}_1^{(i)},y_1^{(i)},\dots,\mathbf{x}^{(i)}_b,y^{(i)}_b\}_{i=1}^m$ from $m$ source domains with batch size $b$, we mix all the instances from each domain and denoted by $\{(\mathbf{x}_i^B,y_i^B)\}_{i=1}^{m^\prime}$ with total size $m^\prime$. We first measure the relative similarity between the negative and positive pairs, which is introduced in the next sub-section.

\subsubsection{Pair Similarity Mining}
 Assume $\mathbf{x}_i^{B}$ is an anchor, a negative pair $\{\mathbf{x}_i^B,\mathbf{x}_j^B \}$ and a positive pair $\{\mathbf{x}_i^B,\mathbf{x}_{j^\prime}^B \}$ are selected if $S_{ij}$ and $S_{i,{j^\prime}}$ satisfy the negative condition $S_{i,j}^{-}$ and the positive condition $S_{i,j}^{+}$, respectively :
\begin{equation}
\label{eq_round_mining}
    S_{i,j}^{-}\ge  \min_{y_i= y_k} S_{i,k}-\epsilon, \; \; \; \; S_{i,j^\prime}^{+}\leq \min_{y_i\neq y_k} S_{i,k}+\epsilon
\end{equation}
where $\epsilon$ is a given margin. Through Eq.~\ref{eq_round_mining} and specific margin $\epsilon$, we will have a set of negative pairs $\mathcal{N}$ and a set of positive pairs $\mathcal{P}$. This process (Eq.~\ref{eq_round_mining}) could roughly cluster the instances with each anchor by selecting informative pairs (inside of the margin), and discard the less informative ones (outside of the margin)

With such roughly selected informative pairs $\mathcal{N}$ and $\mathcal{P}$, we then assign the instance with different weights. Intuitively, if a instance has higher similarity with an anchor, then it should stay closer with the anchor and vice-versa. We introduce the weighting process in the next section

\subsubsection{Pair Weighting}~\label{sect:pair_weighting}

For instances of positive pairs, if they are more similar with the anchor, then it should have higher weights while give the negative pairs with lower weights if they are more dissimilar, no matter which domain they come from. Through this process, we can push the instances into several groups via measure their similarities.

 For $N$ instances, computing the similarity between each pair could result in a similarity matrix $\mathbf{S}\in\mathbb{R}^{N\times N}$.
For a loss function based on pair similarity, it can usually be defined by $\mathcal{F}(\mathbf{S},y)$. Let $S_{i,j}$ be the $i^{th}$ row, $j^{th}$ column element of matrix $\mathbf{S}$. The gradient $w.r.t$ the network could be computed by,

\begin{equation}
\begin{split}
        \frac{\partial\mathcal{F}(\mathbf{S},y)}{\partial\boldsymbol{\theta}_f}&= \frac{\partial\mathcal{F}(\mathbf{S},y)}{\partial\mathbf{S}}\frac{\partial \mathbf{S}}{\partial\boldsymbol{\theta}_f}=\sum_{i=1}^N\sum_{j=1}^N\frac{\partial\mathcal{F}(\mathbf{S},y)}{\partial S_{i,j}}\frac{\partial S_{i,j}}{\partial \boldsymbol{\theta}_f}
\end{split}
\label{Eq.dev_theta_f}
\end{equation}

 Eq.~\ref{Eq.dev_theta_f} could be reformulated into a new loss function $\mathcal{L}_{MS}$ as,
\begin{equation}
    \mathcal{L}_{MS}= \sum_{i=1}^N\sum_{j=1}^N\frac{\partial\mathcal{F}(\mathbf{S},y)}{\partial S_{i,j}}S_{i,j}
    \label{Eq.reformed_lms}
\end{equation}

 usually the metric loss defined $w.r.t$ similarity matrix $\mathbf{S}$ and label $y$ could be reformulated by Eq.~\ref{Eq.reformed_lms}. The term $\frac{\partial\mathcal{F}(\mathbf{S},y)}{\partial S_{i,j}}$ in Eq.~\ref{Eq.reformed_lms} could be treated as an constant scalar since it doesn't contain the gradient of $\mathcal{L}_{MS}$ $w.r.t$ $\boldsymbol{\theta}_f$. Then, we just need to compute the gradient term $\frac{\partial \mathcal{F}_{i,j}}{\partial \boldsymbol{\theta}_f}$ for the positive and negative pairs. Since the goal is to encourage the positive pairs to be closer, then we can assume the gradient $\leq 0$, $i.e.$, $\frac{\partial \mathcal{F}_{i,j}}{\partial \boldsymbol{\theta}_f}\leq 0$. Conversely, for a negative pair, we could assume $\frac{\partial \mathcal{F}_{i,j}}{\partial \boldsymbol{\theta}_f}\geq 0$.
Thus, Eq.~\ref{Eq.reformed_lms} is transformed by the summation over all the positive pair ($y_i=y_j$) and negative pairs ($y_i\neq y_j$),
\begin{equation}
\begin{split}
        \mathcal{L}_{MS}&=\sum_{i=1}^N\sum_{j=1}^N\frac{\partial\mathcal{F}(\mathbf{S},y)}{\partial S_{i,j}}S_{i,j}\\
        &=\sum_{i=1}^N\left(\sum_{j=1,y_j\neq y_i}^N \frac{\partial\mathcal{F}(\mathbf{S},y)}{\partial S_{i,j}} S_{i,j}+\sum_{j=1,y_j= y_i}^N \frac{\partial\mathcal{F}\big(\mathbf{S},y)}{\partial S_{i,j}} S_{i,j} \right)\\
        &=\sum_{i=1}^N \left(\sum_{j=1,y_j\neq y_i}^N w_{i,j}S_{i,j} - \sum_{j=1,y_j= y_i}^N w_{i,j} S_{i,j}  \right)
\end{split}
    \label{L_ms_with_weight}
\end{equation}
where $w_{i,j}= \big | \frac{\partial S_{i,j}}{\partial \boldsymbol{\theta}_f} \big|$ is regarded as the weight for similarity $S_{i,j}$. For each pair of instances $i,j$, we could assign different weights according to their similarities $S_{i,j}$. Then $w_{i,j}^+$ or $w_{i,j}^-$ could be defined as the weight of a positive or negative pairs' similarity, respectively. Previously,~\cite{yi2014deep} and \cite{wang2019multi} applied a soft function for measuring the similarity. We then consider the similarity of the pair itself ($i.e.$ self-similarity), the negative similarity and the positive similarity.
The weight of self-similarity could be measured by $\exp({S_{i,j}-\lambda})$ with a small threshold $\lambda$. For a selected negative pair $\{\mathbf{x}^B_i, \mathbf{x}^B_j\}\in \mathcal{N}$ the corresponding weight (see  Eq.~\ref{L_ms_with_weight}) could be defined by the soft function of self-similarity together with the negative similarity:
\begin{equation}
\begin{split}
    w_{i,j}^-&=\frac{1}{\exp(\beta(\lambda-S_{ij}))+\sum_{k\in\mathcal{N}}\exp(\beta(S_{i,k}-\lambda))}\\
        &= \frac{\exp(\beta(S_{ij}-\lambda))}{1+\sum_{k\in\mathcal{N}}\exp(\beta(S_{ik}-\lambda))}
\end{split}
    \label{Eq.negative_pair_weight}
\end{equation}

Similarly, the weight of a positive pair $\{\mathbf{x}^B_i,\mathbf{x}^B_j\}\in\mathcal{P}$ is defined by,

\begin{equation}
   w_{i,j}^+=\frac{1}{\exp(-\alpha(\lambda-S_{i,j}))+\sum_{k\in\mathcal{P}}\exp(-\alpha(S_{i,k}-S_{i,j}))}
    \label{Eq.positive_pair_weight}
\end{equation}

Then, take Eq.~\ref{Eq.negative_pair_weight} and Eq.~\ref{Eq.positive_pair_weight} into Eq.~\ref{L_ms_with_weight}, and integrate Eq.~\ref{L_ms_with_weight} with the similarity mining $S_{i,j}$, we have the objective function for clustering,
\begin{equation}
\begin{split}
       \mathcal{L}_{MS}=  \frac{1}{m}\sum_{i=1}^{m}\big\{\frac{1}{\alpha}\log[1+\sum_{k\in\mathcal{P}_i}\exp(-\alpha(S_{ik}-\lambda))]
        +\frac{1}{\beta}\log[1+\sum_{k\in\mathcal{N}_i}\exp(\beta(S_{ik}-\lambda))]\big\}
        \label{Eq.multi_similarity_loss}
\end{split}
\end{equation}
where $\lambda$, $\alpha$ and $\beta$ are fixed hyper-parameters, we elaborate them in the empirical setting section~\ref{sect:implementation_details}. Then, the whole objective of our proposed method is,
\begin{equation}
\mathcal{L} = \arg\min_{\theta_f,\theta_c}\max_{\theta_d} \mathcal{L}_{C}+\lambda_d\mathcal{L}_{D} +\lambda_s \mathcal{L}_{MS}
\label{Eq:full_objective}
\end{equation}
where $\lambda_d$ and $\lambda_s$ are coefficients to regularize $\mathcal{L}_d$ and $\mathcal{L}_{MS}$ respectively.

Based on these above, we propose the WADG algorithm in Algorithm~\ref{wadg_algo}. And we show the empirical results in the next section.

\begin{center}
\begin{algorithm}[t!]
		\caption{The proposed WADG algorithm (one round)}
		\begin{algorithmic}[1]
		\REQUIRE Samples from different source domains $\{{\mathcal{D}}_i\}_{i=1}^M$
        \ENSURE Neural network parameters $\boldsymbol{\theta}_{f}$, $\boldsymbol{\theta}_c$, $\boldsymbol{\theta}_d$
        \FOR{mini-batch of samples $\{(\mathbf{x}^{(i)}_s,y^{(i)}_s)\}$ from source domains}
        \STATE Compute the classification loss $\mathcal{L}_C$ over all the domains according to Eq.~\ref{Eq.cls_loss_all_domains}
        \STATE Compute the Wasserstein distance $\mathcal{L}_D$ between each pair of source domains according to Eq.~\ref{Eq.mult_domain_Wa}

        \STATE Mix the pairs from different domains and compute the similarity by Eq.~\ref{Eq.dotsimilarity}
        \STATE Roughly select the positive and negative pairs by solving Eq.~\ref{eq_round_mining}

        \STATE Compute similarity loss $\mathcal{L}_{MS}$ on all the source instances by Eq.~\ref{Eq.multi_similarity_loss}

        \STATE    Update $\theta_f,\theta_c$ and $\baselinestretch_d$ by solving Eq.~\ref{Eq:full_objective} with learning rate $\eta$:
           \begin{equation*}
           \begin{split}
               &\boldsymbol{\theta}_f \leftarrow \boldsymbol{\theta}_f - \eta\frac{\partial(\mathcal{L}_{C}+\lambda_d\mathcal{L}_D+\lambda_s\mathcal{L}_{MS})}{\partial\boldsymbol{\theta}_f}, \\
               &\boldsymbol{\theta}_c \leftarrow \boldsymbol{\theta}_c - \eta\frac{\partial(\mathcal{L}_{C}+\lambda_d\mathcal{L}_D+\lambda_s\mathcal{L}_{MS})}{\partial\boldsymbol{\theta}_c}, \\
               &\boldsymbol{\theta}_d \leftarrow \boldsymbol{\theta}_d + \eta\frac{\partial \mathcal{L}_D}{\partial\boldsymbol{\theta}_d}
           \end{split}
           \end{equation*}

        \ENDFOR
		\STATE Return the optimal parameters $\boldsymbol{\theta}_f^\star$, $\boldsymbol{\theta}_c^\star$ and $\boldsymbol{\theta}_d^\star$
        \end{algorithmic}
        \label{wadg_algo}
\end{algorithm}
\end{center}

\section{Experiments and Results}
\label{experiments results}
\subsection{Datasets}
In order to evaluate our proposed approach, we implement experiments on two common used datasest: \textbf{VLCS}~\cite{torralba2011unbiased} and \textbf{PACS}~\cite{Li2017dg}. The VLCS dataset contains images from 4 different domains: PASCAL VOC2007 (V), LabelMe (L), Caltech (C), and SUN09 (S). Each domain includes five classes: \emph{bird, car, chair, dog} and \emph{person}. PACS dataset is a recent benchmark dataset for domain generalization. It consists of four domains: Photo (P), Art painting (A), Cartoon (C), Sketch (S), with objects from seven classes: dog, elephant, giraffe, guitar, house, horse, person.

\subsection{Baselines and Implementation details}
\label{sect:implementation_details}
To show the effectiveness of our proposed approach, we compare our algorithm on the benchmark datasets with the following recent domain generalization methods.

\begin{itemize}
    \item \textbf{\emph{Deep All}}: We follow the standard evaluation protocol of Domain Generalization to set up the pre-trained Alexnet or ResNet-18 fine-tuned on the aggregation of all source domains with only the classification loss.

    \item TF~\cite{li2017deeper}:  A low-rank parameterized Convolution Neural Network model which aims to reduce the total number of model parameters for an end-to-end Domain Generalization training.

    \item CIDDG~\cite{li2018deep}: Matches the conditional distribution by change the class prior.
    \item MLDG~\cite{li2018learning}: The meta-learning approach for domain generalization. It runs the meta-optimization on simulated meta-train/ meta-test sets with domain shift
    \item CCSA~\cite{Motiian_2017_ICCV}:
    The contrastive semantic alignment loss was adopted together with the source classification loss function for both the domain adaptation and domain generalization problem.
    \item MMD-AAE~\cite{li2018domain}: The Adversarial Autoencoder model was adopted together with the Mean-Max Discrepancy to extract a domain invariant feature for generalization.
    \item D-SAM~\cite{d2018domain}: It aggregates domain-specific modules and merges general and specific information together for generalization.
\item JiGen~\cite{carlucci2019domain}: It achieves domain generalization by solving the Jigsaw puzzle via the unsupervised task.
\item MASF~\cite{dou2019domain}: A meta-learning style method which based on MLDG and combined with Consitrastive Loss/ Triplet Loss to encourage domain-independent semantic feature space.
\item MMLD~\cite{dg_mmld}: An approach that mixes all the source domains by assigning a pseudo domain label for extract domain-independent cluster feature space.
    \end{itemize}

Following the general evaluation protocol of domain generalization ($e.g.$~\cite{dou2019domain,dg_mmld}), on PACS and VLCS dataset. We first test our algorithm on by using AlexNet~\cite{krizhevsky2012imagenet} backbones by removing the last layer as feature extractor. For preparing the dataset, we follow the train/val./test split and the data pre-processing protocol of~\cite{dg_mmld}. As for the classifier, we initialize a three-layers MLP whose input has the
same number of inputs as the feature extractor's output and to have the same number of outputs as the number of object categories (2048-256-256-$K$), where $K$ is the  number of classes. For the critic network, we also adopt a three-layers MLP (2048-1024-1024-1). For the metric learning objective, we use the output of the second layer of classifier network (with size 256) for computing the similarity. We adopt the ADAM~\cite{kingma2014adam} optimizer for training with learning rate ranging from $5e-4$ to $5e-5$ for the whole model together with mini-batch size $64$. For stable training, we also set coefficient $\lambda_d= \frac{2}{1+\exp(-\delta p)}-1$ to regularize the adversarial loss, where $\delta=10$ and $p$ is the training progress. And we empirically set $\lambda_s$ ranging from $1e-4$ to $1e-6$  via reverse validation. For the hyperparameters in $\mathcal{L}_{MS}$ (see Eq.~\ref{eq_round_mining} and Eq.~\ref{Eq.multi_similarity_loss}), we empirically set $\lambda=0.5$, $\epsilon = 1e-5$, $\alpha = 2.0$, $\beta = 40.0$. The experiments are programmed with help of \emph{Pytorch}~\cite{paszke2019pytorch}. To avoid over-training, we also adopt the early stopping technique.

\begin{table}[]
\caption{Empirical Results (accuracy $\%$) on PACS dataset with pre-trained AlexNet as Feature Extractor. For each column, we refer the generalization taks as the target domain name. For example, the third column `Cartoon' refers to the generalization tasks where domain \emph{Cartoon} is the target domain while the model is trained on the rest three domains }\label{tb_pacs_dataset}
\centering
\resizebox{1.0\textwidth}{!}{\begin{tabular}{l|llll|l}

\toprule
Method   &  Art   & Cartoon  & Sketch & Photo & Avg. \\
\hline
Deep All &  $63.30$ & $63.13$ & $54.07$ & $87.70$ & $67.05$\\
TF\cite{li2017deeper} & $62.86$ & $66.97$ & $57.51$ & $89.50$ & $59.21$  \\
CIDDG\cite{li2018deep}  & $62.70$ & $69.73$ & $64.45$ & $78.65$ & $68.88$ \\
MLDG \cite{li2018learning}&  $66.23$ & $66.88$ & $58.96$ & $88.00$ & $70.01$\\
D-SAM\cite{d2018domain}  & $63.87$ & $70.70$ & $64.66$ & $85.55$ & $71.20$\\
JiGen\cite{carlucci2019domain}        &  $67.63$ & $71.71$ & $65.18$ & $89.00$ & $73.38$\\
MASF\cite{dou2019domain}  &  $\mathbf{70.35}$ & $ 72.46$  & $67.33$ &$90.68$ & $75.21$\\
MMLD\cite{dg_mmld}        &  $69.27$ & $\mathbf{72.83}$  & $66.44$ &$88.98$ & $74.38$\\

\hline
Ours &$70.21$ & $72.51$ & $\mathbf{70.32}$ & $\mathbf{89.81}$ & $\mathbf{75.71}$ \\
\bottomrule
\end{tabular}}
\end{table}

\subsection{Experiments Results}

\begin{table}[]
\caption{Empirical Results (accuracy $\%$) on VLCS dataset with pre-trained AlexNet as Feature Extractor.}\label{tb_vlcs_dataset}
\resizebox{1.0\textwidth}{!}{\begin{tabular}{l|llll|l}
\toprule
Method   & Caltech  & LabelMe & Pascal  & Sun     & Avg.    \\ \midrule
Deep All & $92.86$ & $63.10$ & $68.67$ & $64.11$ & $72.19$ \\
D-MATE\cite{ghifary2015domain}   & $89.05$  & $60.13$ & $63.90$ & $61.33$ & $68.60$ \\
CIDDG\cite{li2018deep}     & $88.83$  & $63.06$ & $64.38$ & $62.10$ & $69.59$ \\
CCSA\cite{Motiian_2017_ICCV}    & $92.30$  & $62.10$ & $67.10$ & $59.10$ & $70.15$ \\
SLRC\cite{ding2017deep}     & $92.76$  & $62.34$ & $65.25$ & $63.54$ & $70.97$ \\
TF\cite{li2017deeper}     & $93.63$  & $63.49$ & $69.99$ & $61.32$ & $72.11$ \\
MMD-AAE~\cite{li2018domain}  & $94.40$  & $62.60$ & $67.70$ & $64.40$ & $72.28$ \\
D-SAM~\cite{d2018domain}    & $91.75$  & $56.95$ & $58.95$ & $60.84$ & $67.03$ \\
MLDG\cite{li2018learning}     & $94.4$   & $61.3$  & $67.7$  & $65.9$  & $73.30$ \\
JiGen\cite{carlucci2019domain}    & $96.93$  & $60.90$ & $70.62$ & $64.30$ & $73.19$ \\
MASF\cite{dou2019domain}  & $94.78$  & $64.90$ & $69.14$ & $67.64$ & $74.11$ \\
MMLD\cite{dg_mmld}  & $96.66$  & $58.77$ & $\mathbf{71.96}$ & $\mathbf{68.13}$ & $73.88$ \\ \hline
Ours     & $\mathbf{97.85}$   & $\mathbf{65.26}$    & $71.47$     &  $66.62$   &  $\mathbf{75.31}$  \\ \bottomrule
\end{tabular}}
\end{table}

We firstly report the empirical results on PACS and VLCS dataset using AlexNet as feature extractor. For each generalization task, we train the model on all the source domains and test on the target domain and report the average of top 5 accuracy. The results on PACS and VLCS dataset using AlexNet are reported in Table~\ref{tb_pacs_dataset} and Table~\ref{tb_vlcs_dataset}, respectively. For each table, the empirical results refers to the average accuracy about training on source domains while testing on the target domain. From the empirical results, we could see our method could outperform the baselines both on the PACS and VLCS dataset, indicating an improvement on benchmark performances.

\subsection{Further Analysis}
\label{further_analysis}
To further show the effectiveness of our algorithm especially on more deep models, follow~\cite{dou2019domain}, we also report the results of our algorithm by using ResNet-18 backbone on PACS dataset in Table~\ref{pacs_res18}. The ResNet-18 backbone, the output feature dim will be $512$. From the results, we could observe that our method could outperform the baselines on most generalization tasks and on average $+1.6\%$ accuracy improvement.

Then, we implement ablation studies on each component of our algorithm. We report the empirical results of ablation studies in Table~\ref{table_ablation_study}, where we test the ablation studies on both the AlexNet backbone and ResNet-18 backbone. We compare the ablations by, (1)~\emph{Deep All}: Train the model using feature extractor on source domain datasets with classification loss only, that is, neither optimal transport nor metric learning techniques is adopted.
(2)~\emph{No $\mathcal{L}_D$}: Train the model with classification loss and metric learning loss but without adversarial training component; (3)~\emph{No $\mathcal{L}_{MS}$}: Train the model with classification loss and adversarial loss but without metric learning component; (4)~\emph{WADG-All}: Train the model with full objective Eq.~\ref{Eq:full_objective}. From the results, we could observe that one we omit the adversarial training, the accuracy would drop off rapidly ($\sim3.5\%$ with AlexNet backbone and $\sim5.8\%$ with ResNet-18 backbone). The contribution of the metric learning loss is relatively small compared with adversarial loss. Once we omit the metric learning loss, the performance will drop $\sim2.1\%$ and $\sim 2.5\%$ with AlexNet and ResNet-18 backbone, respectively.

\begin{figure}
\centering
\begin{subfigure}{.40\textwidth}
  \centering

  \includegraphics[width=\textwidth]{figs/Deep_all.pdf}
  \caption{Deep All}
  \label{fig:sub-first}
\end{subfigure}
\begin{subfigure}{.40\textwidth}
  \centering
  \includegraphics[width=\textwidth]{figs/No_dis.pdf}
  \caption{No $\mathcal{L}_D$}
  \label{fig:sub-second}
\end{subfigure}\\
\begin{subfigure}{.40\textwidth}
  \centering

  \includegraphics[width=\textwidth]{figs/NO_MTR.pdf}
  \caption{No $\mathcal{L}_{MS}$}
  \label{fig:sub-third}
\end{subfigure}
\begin{subfigure}{.40\textwidth}
  \centering

  \includegraphics[width=\textwidth]{figs/WADG.pdf}
  \caption{WADG-All}
  \label{fig:sub-fourth}
\end{subfigure}
\caption{T-SNE visualization of ablation studies on PACS dataset for Target domain as \emph{Photo}. Detailed analysis is presented in section~\ref{further_analysis}. }
\label{fig:tsne}
\end{figure}

\begin{table}[]
\centering
\caption{Empirical Results (accuracy $\%$) on PACS dataset with pre-trained ResNet-18 as feature extractor .}\label{pacs_res18}
\resizebox{1.0\textwidth}{!}{\begin{tabular}{l|llll|l}

\toprule
Method   &  Art   & Cartoon  & Sketch & Photo & Avg. \\
\hline
Deep All & $77.87$ & $75.89$ & $69.27$ & $95.19$ & $79.55$ \\
D-SAM\cite{d2018domain}    & $77.33$ & $72.43$ & $77.83$ & $95.30$ & $80.72$ \\
JiGen\cite{carlucci2019domain}     & $79.42$ & $75.25$ & $71.35$ & $96.03$ & $80.51$ \\
MASF\cite{dou2019domain}     & $80.29$ & $77.17$ & $71.69$ & $94.99$ & $81.04$ \\
MMLD\cite{dg_mmld}     & $81.28$ & $77.16$ & $72.29$ & $\mathbf{96.09}$ & $81.83$ \\ \bottomrule
Ours & $\mathbf{81.56}$ & $\mathbf{78.02}$ & $\mathbf{78.43}$ & $95.82$ & $\mathbf{83.45}$ \\
\bottomrule
\end{tabular}}
\end{table}

\begin{table}[]

\centering
\caption{Ablation Studies on PACS dataset on all components of our proposed method using AlexNet and ResNet-18 backbone}
\label{table_ablation_study}
\resizebox{1.0\textwidth}{!}{\begin{tabular}{l|lllllllll|l}
\toprule
         & \multicolumn{5}{c|}{AlexNet}                                         & \multicolumn{5}{c}{ResNet-18}        \\
Ablation & Art     & Carton  & Sketch  & Photo   & \multicolumn{1}{l|}{Avg.}    & Art & Carton & Sketch & Photo & Avg. \\ \midrule
Deep All & $63.30$ & $63.13$ & $54.07$ & $87.70$ & \multicolumn{1}{l|}{$67.05$} & $77.87$ & $75.89$ & $69.27$ & $95.19$ & $79.55$ \\
No $\mathcal{L}_D$  & $65.80$ & $69.64$ & $63.91$ & $89.53$ & \multicolumn{1}{l|}{$72.22$}        & $74.62$    & $73.02$       & $68.67$       &  $94.86$     & $77.79$     \\
No $\mathcal{L}_{MS}$  & $66.78$ & $71.47$ & $68.12$ & $88.87$ & \multicolumn{1}{l|}{$73.65$}        &  $78.25$   &    $76.27$    & $73.42$       &    $95.68$   &  $80.91$    \\
WADG-All      &   $\mathbf{70.21}$      & $\mathbf{72.51}$        &    $\mathbf{70.32}$     &  $\mathbf{89.81}$       & \multicolumn{1}{l|}{$\mathbf{75.71}$} & $\mathbf{81.56}$    & $\mathbf{78.02}$       & $\mathbf{78.43}$       &  $\mathbf{95.82}$     &  $\mathbf{83.45}$    \\ \bottomrule

\end{tabular}}
\end{table}

Then, to better understand the contribution of each component of our algorithm, the T-SNE visualization of the ablation studies of each components on PACS dataset are represented in Fig.~\ref{fig:tsne} for the generalization task of target domain \emph{Photo}. Since our goal is to not only align the feature distribution but also encourage a cohesion and separable boundary, in order to show the alignment and clustering performance, we report the T-SNE features of all the source domains and target domain to show the feature alignment and clustering across domains. As we can see, the T-SNE features by \emph{Deep All} could neither project the instances from different domains to align with each other nor cluster the features into groups. The T-SNE features by \emph{No $\mathcal{L}_D$} showed the metric learning loss could to some extent to cluster the features, but without the adversarial training, the features could not be aligned well. The T-SNE features by \emph{No $\mathcal{L}_{MS}$} showed that the adversarial training could help to align the features from different domains but could not have a good clustering performance. The T-SNE features by \emph{WADG-All} showed that the full objective could help to not only align the features from different domains but also could cluster the features from different domains into several cluster groups, which confirms the effective of our algorithm.

\section{Conclusion}
In this paper, we proposed the Wasserstein Adversarial Domain Generalization algorithm for not only aligning the source domain features and transferring to an unseen target domain but also leveraging the label information across domains. We first adopt optimal transport with Wasserstein distance for aligning the marginal distribution and then adopt the metric learning method to encourage a domain-independent distinguishable feature space for a clear classification boundary. The experiments results showed our proposed algorithm could outperform most of the baseline methods on two standard benchmark datasets. Furthermore, the ablation studies and visualization of the T-SNE features also confirmed the effectiveness of our algorithm.

\bibliographystyle{plain}

redpajama_set_name: RedPajamaArXiv


Come and enjoy the south of 5th. life-style from this 1 Bdrm / 2 Bthrm unit with direct unobstructed water views of government cut, Fisher Island, the Ocean, Key Biscayne & the Miami skyline. Second Bedroom was removed turning it into a huge One Bedroom with 2 full Bathrooms. Offered fully furnished. Impecable building with 2 pools, tennis, gym and much more. Walk to the beach & restaurants (Joe's, Smith & W, Pink, Milos). TENANT OCCUPIED THROUGH AUG/14/19.
MIAMI Association of REALTORS® is not responsible for the accuracy of the information listed above. The data relating to real estate for sale on this website comes in part from the Internet Data Exchange Program and the South East Florida Regional MLS and is provided here for consumers' personal, non-commercial use. It may not be used for any purpose other than to identify prospective properties consumers may be interested in purchasing. Real estate listings held by brokerage firms other than the office owning this website are marked with the IDX logo and detailed information about them includes the name of the listing brokers. Data provided is deemed reliable but not guaranteed. Copyright MIAMI Association of REALTORS®, MLS All rights reserved. This property is courtesy of Solid Group Realty, LLC.


Includes our ID Sanctuary Enhanced Premium Individual, Global Travel Assistance, & Roadside Assistance.
Includes ID Sanctuary Enhanced Premium Family, Global Travel Assistance, & Roadside Assistance.
Includes Vision, Counseling Services, Health Advocate(TM) Solutions, Dental, Pharmacy, & MeMD Telemedicine.
Includes Vision, Counseling Services, Global Travel Assistance, Roadside Assistance, Instant Deals, Legal Services, Dental, Pharmacy, ID Sanctuary Enhanced Family, MeMD Telemedicine, & Health Advocate(TM) Solutions.
Includes Global Travel Assistance, Roadside Assistance, Instant Deals, Legal Services, Pharmacy, ID Sanctuary Enhanced Individual, & MeMD Telemedicine.
Includes Roadside Assistance, Instant Deals, & Legal Services.


Another inspiration (in the flying attic).
This is something "soft", breathless played in 4.30 minutes over a bass and drums loop.
...and very cool. Sumptuous sounds. Really nice and I think it deserves some commenting. You always manage to get a wonderful, ethereal feel in your tunes. I often wish they were longer. I'll take this one for my Spires playlist, with thanks.
Honored of still being among your favorites!
Too many things on the web to watch and to listen to, and feedback and so on …, that's ok but it's really confusing me. Not a problem if something I do or play sounds slightly out of time.
Nice to hear after a horrible day at work.
Gentle music - much enjoyed!
Nice floating feel. Lovely tones. Meditative. Thanks.


DRI—The Voice of the Defense Bar is the world's largest network of attorneys defending the interests of business and individuals in civil litigation. For more than 50 years, DRI has grown from a small group of like-minded individuals, to an engaged network of nearly 20,000 defense attorneys and in-house counsel all working together to strengthen the defense bar. To help you reach this target audience, DRI provides a variety of advertising opportunities to fit every budget.
As DRI's flagship publication, For The Defense™ (FTD) is the only national monthly magazine specifically written for and distributed to DRI lawyers and in-house defense counsel members. Each issue contains in-depth, full-length, feature articles addressing crucial developments in the law and litigation practice. Articles are penned by top defense practitioners and experts from across the country on a variety of topics of interest to defense lawyers and others concerned with the defense of civil actions.
DRI's national quarterly publication, In-House Defense Quarterly™ (IDQ), addresses the specific needs of in-house defense counsel and corporate legal executives. In addition to full-length articles, IDQ keeps its readers up-to-date with substantive information and reports on technology, legislative tracking, international legal trends and developments, and fellow DRI members. IDQ is distributed to DRI in-house defense counsel members as well as subscribers. Maximize your target marketing campaign and advertise in IDQ to reach this loyal and influential audience. Law firms, in addition to providers of relevant products and services, are welcome to advertise in IDQ. Space is limited, act quickly!
DRI's electronic advertising opportunities provide a great way to maximize your company's exposure within the DRI Community while creating an interactive touch-point that puts our audience right in front of your brand.
Produced by DRI's substantive law committees, our committee e-newsletters provide an excellent way for you to reach a more niched and targeted audience. Available in a variety of practice areas, advertising in one of DRI's committee-specific e-newsletters allows you to maximize your exposure to in-house defense counsel, leading defense attorneys and practitioners worldwide.
DRI's weekly e-newsletter, The Voice, is sent every Wednesday to the entire membership of more than 20,000 defense attorneys and in-house defense counsel. Each issue has one to two feature articles that are submitted by one of our 29 substantive law committees and relate to the civil defense community. Members can also highlight a recent winning case in our "And the Defense Wins" section. Maximize your exposure by reaching multiple target markets at one time.
DRI's Communities are the center of all substantive activity. Connect with DRI influencers through targeted, efficient and timely advertising to drive awareness.


This is art. The development of the Zalto Denk'Art series was influenced by the earth in accordance to the tilt angles of the Earth. Thus, the curve of the bowls are tilted at the angles of 24 degrees, 48 degrees and 72 degrees. The Zalto Denk'Art Champagne glass is recommended for Champagne and Sparkling wines. Created for Champagne, Sparkling Wines, Prosecco, Spumante and Cava. Lead-Free. Dishwasher-Safe.
This glass maximises both the aroma and palate of fine sparkling wines. The super thin glass allows the finest of mousse and a long lasting bead. Gives the sparkling wines of the world a noblesse, while bringing out their best of their regional character.
Introducing a new generation of mouth-blown, lead-free and dishwasher-safe wine glasses, the Zalto Denk'Art Collection, luxury stemware you could sense from your fingertips to the moment you take your first sip.
From the first touch, each Zalto glass distinguishes itself from all other glasses. From the perfectly executed balance to the extraordinary design, the achievement of the Denk`Art line are glasses that seem nearly too delicate to hold, and yet they are made for just that. This is glassware that stands at the rarefied point where form meets function.


Thus, 2000 x 50 = 100,000 btu/hour needed to heat the house & the size boiler needed. For a free quotation call us on. I have a service plan with benjamin franklin plumbing, and they also take great care of me and my home. The right heating controls will let you set heating and hot water to come on and off when you need it, heat individual areas of the home and decide how warm they want each area to be. At bell plumbing, we take great pride in our plumbing services because we know that these will make an impact in how well your building functions when it comes to water sources depending on your facility, you may heavily rely on pipes working and water flowing as it is designed to when plumbing is disrupted it can cost you time and slow down your workday our plumbers are quick to react when a situation arises and we follow all epa regulations so that all regulations are followed if you require a commercial drain survey, call our experts today to get started. If you're searching for an orange county. Company installed a tankless water heater i got three estimates and asap was right in the middle i hired them for the job because i felt they gave me the best value for the job the job was coordinated to be done while we were away asap followed through with all the terms of the contract we did not have to worry when we were away, they showed up completed the job when we get back everything was one hundred percent. Get in touch today for more information about our baxi range. Specialising in smaller jobs, which means they can often respond more quickly than larger firms and keep costs competitive gassafe and wras approved 'reliable, very quick to reply, extremely professional. Fit a glass shower corner cubicle. At swale heating we know just how important it is to get your boiler back up and working as soon as possible and are recognised for carrying out an unrivalled emergency boiler repairs service. Despite our best efforts, it's almost inevitable that some grease, fat and food scraps will end up going down the plugholes of our kitchen sinks and over time, if not dealt with, the day will come when you're left with a sinkful of dirty water and a furrowed brow because the pipes are blocked kitchen blocks away is here to help – this overnight sensation will soon have things flowing freely again. If you find you have a problem with your central heating, or would just like a boiler health check, our fully qualified, friendly team of. Epsom, chessington, ewell, ashtead, banstead, cheam, sutton, kingston, esher, cobham and surrounding areas please contact for more information on: 0208 454 7898. Printed curcuit board for boiler's with serial number below btg815 box opened ,not used. Clogged or broken pipes in bedford. Fixed price — you wont pay any more than the quote, even if the repair takes longer than estimated. This clarifies if the drains are ruptured, cracked, blocked, failing, or in need of repair. Evenflow heating and plumbing offer an excellent and reliable service ranging from full heating installations and servicing; to full bathroom renovation. Dorset business awards winner of 2016 and national energy efficiency awards 2017 finalist. Asp 18kw 3 phases output connection. Although most of these local companies are "24-hour plumbing service" professionals they do in fact handle all routine repair and installation as well as every phase imaginable of plumbing work you may find one or two plumbing contractors from our service that does not handle a particular service that you are looking for, but that is very rare. Gullies and drains are cleaned on a regular schedule. They install radiators and common appliances like dishwashers and washing machines, again, making sure that waste pipes and taps are exactly where they need to be. Navigating price can be one of the most difficult parts of buying a boiler, and you want to make sure you get the best deal possible. The majority of our entry level combi boilers range between £600 - £700, with only some of the larger models coming in over £700 many offer free extended warranties of 5 years when you use an approved installer. You can definitely look at our ratings & reviews for local plumbers that is another method by which you can see if they're reputable. Bathurst, peterborough, pe2 5qh -. Depending on the type of boiler installation you may be without heating for the entire time of the install it isn't fair to staff to be dishonest with them about this london seo services http://controlseo.co.uk/ although you will still be able to use electricity and water, without heating your place of work could be unbearable tell staff that this is going to be the case and ask them what they would prefer if there is an option to work from home this may be best as it will then mean that they will be out of the way if this is simply not an option then get some storage heaters and look at ways you can maximise the heat, like all working in one room a better option may be to have the boiler installation completed in the summer when the heating is not required anyway. Chesterfield airport road. If you have any requirements for boiler installation or repair services, feel free to contact us and receive the best services in town. A system boiler, just like a regular (conventional) boiler, works on the principle of stored hot water however, a system boiler differs from a regular boiler in some important respects. More efficiency: we will provide you with the appropriate retrofitting package. New boiler models have improved energy savings compared to older models and are a common reason why homeowners often upgrade their system sooner than later our experts will properly size your boiler before installation to make sure you get the best efficiency, comfort, and operating costs over the lifetime of your new hydronic heating system. Wall mounted gas fired combi boiler with 5 inch colour touchscreen control. Before you buy, note the stem length required by your water heater model. Electric water boilers are the best hot water solution for you if you have no access to gas or simply prefer not to use it in your household the device is comprised of a large water tank with an inlet and outlet with a great number of electric heating elements that keep the water warm at all times an electric boiler is the most efficient form of water heating with no exhaust fuel needed compared to a traditional gas boiler, it takes up far less space, as you don't have to have the space to store propane units as a device not requiring many additional accessories, they are also fairly cheap to install and very easy to maintain however, compared to the heat achieved by gas burning, electric boilers can be a bit less heat-efficient. Our knowledge and understanding of the industry are going to prove to be very important in helping you get the best boiler and most importantly we can also save you money we offer a wide variety of boilers both combi and traditional boilers, and each of them has its own set of benefits our experts can help you assess your home and advice on the best option for your home and also for your budget to guarantee that you're able to enjoy maximum benefits and also reduce the amount of money you use for energy bills, our boiler installation syston services are tailored precisely to meet your requirements and needs. Great non toxic drain cleaner this stuff works great, better than other enzyme-based products i have tried! A few questions about your old. I am not sure how this product is working since i only applied it a little over a week ago but so far so good it was easy to do the treatments...read the directions to make sure you are treating properly overall it was easy and clean and did not have any negative issues i. Ideally, you'll also have a combi boiler combi boilers are perfect for small families and those living alone or with a partner they are small and can fit into a kitchen cupboard or be fixed onto the wall combi boilers are efficient and will provide hot water on demand, rather than needing to be heated in a water tank first. Read this page to find out more. Hour emergency call-outs with no nasty call-out charges! I am very impressed, i was contacted well before 24 hours, everything was explained and understandable thank you and very well done.". Worcester bosch is a market leading boiler brand who have been manufacturing boilers in the uk for over 50 years the worcester bosch greenstar range has been awarded the 'which? best buy' for 7 years in a row with worcester bosch consumers being the 'most satisfied customers' in the which? boiler survey. Club energy members can use our industry leading. Bec offers factory authorized service for new construction and existing facilities our services are immediate and accurate, to keep your boiler room running at peak performance bec service technicians have a minimum of 15 years experience in the boiler industry including extensive experience with all boiler and deaerator manufactures bec is committed to offer you problem-solving insightful solutions and analysis for your entire boiler room. Bosch greenstar installation stands, greenstar controls, concentric vent kit. B) new entrants to the industry. We have been offering customers total satisfaction for over 25 years we will schedule an appointment at your convenience and don't worry about the time because we also offer 24 hour emergency services with no overtime charges for nights, weekends or holidays as a local plumbing company, we have the ability to make it to your vicinity in record time our experts can install, repair, maintain, or replace all of the top brands including toto, american standard, kohler, moen, grohe, insinkerator, and more! call miami emergency plumbing now at (305) 501-2093 for reliable, affordable plumbers that are at your every beck and call! We are the most well established company for. Once you've topped up the pressure, you'll need to reset the boiler on the baxi 1pf you'll see an orange button on the 2pf there will be a red button these buttons will reset the boiler and get it up and running again. Sometimes the boiler may need depressurising to resolve the problem this is a more complex procedure and should only be carried out by a trained gas accredited engineer. Retrofits are fuel-specific, so see the following sections for retrofit information. We've been installing gas boilers in banbury and the local area for more than thirty years, including boilers from top manufacturers such as ideal, worcester bosch and vaillant. Great for homes with 2+ bathrooms. If you spot a flooded road, or blocked drains please report it we will need to know what the problem is, and where the problem is, for example 'outside 15 blank street' it is normal for standing water to be visible in the hole this does not indicate a blockage. Not only were your guys pleasant and helpful, they went beyond the call of duty in accommodating us in attendance times and key pick-ups and drop offs. We're responsible for keeping the drains and gullies alongside medway's roads clean and free from obstruction, to prevent flooding these drains or gullies take storm water from the road into the sewer system. Nasty odours are one of the side effects of blockages in most cases, a drain becomes blocked when large items get stuck in the strainer or begin to clog up the drainage pipe these items then rot and bacteria breeds, releasing the nasty smell. Peace of mind — all british gas engineers are highly trained and gas safe registered. The next method is one of the simplest ways to unblock a drain first fill up the kettle to its maximum capacity and boil pour it down the blocked drain slowly in the stages, allowing the water to have an effect for several seconds between each pour reward yourself with a coffee if there is any remaining water left. If your property was built after 30 october 1937, the drains serving your property are your responsibility but the sections serving more than one property are in shared ownership of those properties. Of those long springy things you get in hardware shops. Severe toilet clogs can lead your unit to overflow to prevent this problem, be mindful of what gets flushed on a regular basis avoid flushing excess toilet paper and other items that are difficult to break down. Volume 2 / issue 1 / january 2015. Prefer to speak on the phone? call us on. Electrical instant heaters, supplied as an accessory in a central air-conditioning system, are the ideal solution for extreme conditions. Useful products and solutions have been our passion at clas ohlson since 1918 covering all things home, multimedia, electrical, leisure and hardware, we have 1000s of ways to help you solve everyday practical problems. You can pay our engineer direct or call the office to make payment payment is due immediately after works have been conducted.


text:
Funerals For The Future But Relevant Now
by Jane Chitty March 22, 2016
The concept on how a funereal should be carried out is suddenly changing and perhaps not before time. For decades, and perhaps even centuries, funerals have been all about ceremony and protocol.
Is the "zero fuss" funeral the way to go?
Very recently, when David Bowie died, the direct cremation was the choice he had made. And many years ago, John Lennon had a similar arrangement. This option means the body is sent straight to the crematorium from either hospital, hospice or home with no attendance or ceremony. The ashes are then scattered as per the family's wishes. Still small in numbers, this type of final farewell is likely to grow in popularity. As well as being considerably less expensive, it cancels out the ordeal of a funeral itself for friends and family alike. There are other types of funerals too...
The bio urn
The bio urn or bio degradable urn will help you to live on and provide new life for a tree after your death. If you or your family chose this, your ashes are added to the biodegradable urn containing a tree seed. When planted in a suitable spot, the tree seed of choice absorbs the nutrients in your ashes as well as moisture, germinates and begins to grow. The urn is made from coconut shell, compacted peat and cellulose. What a wonderful idea for someone who cares about the environment and the planet, and wants to make a final statement by reducing their carbon footprint in this way. The idea came from the Catalan artist and product designer Gerard Moline who combined the romantic notion of life after death with an eco solution.
The burial pod
Another innovative idea, and still in the design and development stage, is the burial pod where a person's body is posed in the fetal position and placed into an egg-shaped pod attached to a tree seedling - if possible, the type of tree is chosen by the deceased before their death. The pod is then planted in the earth and, as the tree grows, it draws nutrients from body to provide a living memorial for friends and family to visit. The resulting aesthetic appeal of a forest of trees, in place of rows of tombstones, could also have an important impact on the environment by not only ensuring the planting of new trees but also preventing the cutting down of existing trees for traditional wood coffins. Like the bio urn, this concept was also Italian – the designers being Raoul Bretzel and Anna Citelli. Both the bio urn and the burial pod (if the ideas really catch on) could lead to special gardens, parks or woodland areas dedicated for the planting of these trees in different areas of the country. This is an amazing opportunity to create protected forests as final resting places for all the right reasons – beauty, peace of mind, shade, protection, oxygen, a safe shelter for wildlife and a wonderful place for children to explore knowing that their ancestors are watching over them.
Resomation
At the moment, and unique to Florida, is a funeral home where patrons can opt to have their tissues dissolved as an alternative to traditional cremation. The process, called resomation or "bio-cremation", uses heated water and potassium hydroxide to liquefy the body, leaving only bones behind. The bones are then pulverized in much the same way as regular cremation. Apparently, this is a more environmentally friendly process than the natural-gas-fueled fire for cremation, which reaches temperatures of 1,600 to 1,800 degrees Fahrenheit (871 to 982 degrees Celsius), releasing carbon dioxide as well as trace chemicals such as mercury from dental fillings. Resomation requires water of only 350 degrees F (176 degrees C) and takes the same amount of time as traditional cremation.
Wicker caskets
These environmentally-friendly alternatives to wooden coffins are naturally hand crafted and you can choose from bamboo, seagrass, cane and willow. These type of caskets can be used for cremations or for natural burials.
Natural burials are interments that take place without embalming and without the concrete vaults that line graves in most modern cemeteries. Bodies are wrapped in a shroud or placed in a biodegradable casket, the idea being that they will decompose naturally. There are at least 50 natural cemeteries in the USA with many regular cemeteries providing facilities for natural graves. Embalming comes with a very high cost and a growing concern about the environmental effects of the embalming procedures and all the goods and resources devoted to this modern method. In addition, many natural cemeteries double as nature preserves, and many people like the idea of contributing to the ecosystem after death by allowing the body to rejoin the cycle of life.
Eternal Reefs
Georgia-based Eternal Reefs creates artificial reef material out of a mixture of concrete and human remains (the crushed bone left over from cremations). These heavy concrete orbs are then placed in areas where reefs need restoration, attracting fish and other organisms that turn the remains into an undersea habitat, providing an opportunity not just to return to an aquatic environment but to produce new life under the sea.
Freeze-drying
Finally, and yet another newcomer for an eco-burial suggestion. is a process called promession or freeze-drying and invented by Swedish marine biologist Susanne Wiigh-Masak. This process includes immersing the corpse in liquid nitrogen to make it very brittle before being vibrated and evaporated with any mercury fillings or surgical implants being filtered out before the powdered remains are laid to rest in a shallow grave. With a shallow burial, oxygen and water can mix with the powdered remains, turning them into compost. Green and environmentally-friendly burials are starting to catch the imagination of the American and worldwide public.

redpajama_set_name: RedPajamaCommonCrawl


Built in air pump. Powerful vacuum action! Use the "lockout" key card to enter temperature settings into the system memory, then when the card is removed, all parameter settings are "locked" into the system memory and cannot be changed without the key card. This feature guards against the operator making accidental or unauthorized temperature changes.
The 472D-02 provides gun-style desoldering with powerful, self-contained suction power (see 817 gun details below).
The 817 desoldering gun is compatible with the Hakko 472, 473 and 703 stations. Any station that uses the Hakko 807 desoldering iron can operate the 817. The 817 incorporates the same nozzles and filters as the 802 gun or 807 iron.
The primary benefit of the 817 is to allow operators the choice of either gun or pencil desoldering without having to purchase a separate station (ie. 470 or 472).
NOTE: Do NOT use the 631-06 holder and base (802 holder) with the 817 gun - the holder will get hot and may damage the 817. Replace the B1094 holder from the 631-06 with holder B2833 for proper operation.
817 Filter Filter pipe with front holder for the 817 Desoldering Gun. Note: This item replaces B2073.
B1084 Power Switch Black power switch for the FX-301B,936/937/851.


We invite you to a concert to be held in the old church in Selca! Salona vocalists are cultural artistic association founded in 1974 in Solin. As a vocal ensemble they are oriented on performing folk songs Dalmatian region, church songs and Dalmatian harmony singing songs.
With vocalist Salone work began maestro Dusko Tambača which is established first team. Besides him an indelible mark he left the maestro Ljubo Stipisic, bard Dalmatian songs. In addition to the two maestro vocalist Salone led and: bang. Petar Cvitanović, Nikola Buble, Rajimir Prince, Crnojezerska and Grgo Grubisic. Ljubo Stipišić the 15 years leading vocalist Salone. Leading the ensemble took over in 2009 prof. Vladan Vuletin, but it was 2011 replaced maestro Mirko Jankov.
Repeat appearances on the "Festival of Dalmatian a cappella singing" in Omis, winning many awards, including the prize for best performance.
In March 1998, the Choir Competition Zagreb won "silver medal".
They have performed all over the Croatian and abroad, and so far have released seven albums.


text:
Q: Unable to decrypt encrypted message using another instance of cipher I am using ruby version 2.4.0 and openssl version "OpenSSL 1.0.1f 6 Jan 2014", I am trying to implement an encryption/decryption for a security layer.
The code works fine if I write it as follows using the same cipher object.
# Example 1

require 'openssl'
require 'base64'

data = "Hello1"

cipher = OpenSSL::Cipher::AES128.new :GCM
cipher.encrypt
iv  = cipher.random_iv # random iv
key = cipher.random_key # 128 byte key
cipher.key = key
cipher.iv = iv
enc_data  = cipher.update(data) + cipher.final

cipher.decrypt
cipher.key = key
cipher.iv = iv
original_data = cipher.update(enc_data) + cipher.final

if data == original_data
  puts "Yes"
end

But in the second example where I have instantiated a second cipher object for decrypting I am getting a CipherError.
# Example 1

require 'openssl'
require 'base64'

def encrypt(data)
  cipher = OpenSSL::Cipher::AES128.new :GCM
  cipher.encrypt
  key = cipher.random_key
  iv  = cipher.random_iv
  cipher.key = key
  cipher.iv = iv
  enc_data = cipher.update(data) + cipher.final
  return enc_data, key, iv
end

def decrypt(data, key, iv)
  cipher = OpenSSL::Cipher::AES128.new :GCM
  cipher.decrypt
  cipher.key = key
  cipher.iv  = iv
  cipher.update(data) + cipher.final
end

data = 'Hello2'
enc_data, key, iv = encrypt(data)
original_data = decrypt(enc_data, key, iv)

if data == original_data
  puts "Yes"
end

OpenSSL::Cipher::CipherError:
from (irb):93:in `final'
from (irb):93:in `decrypt'
from (irb):98

A: GCM mode requires a bit more setup, compared to, say, CBC mode.
Documentation: https://ruby-doc.org/stdlib-2.4.0/libdoc/openssl/rdoc/OpenSSL/Cipher.html#class-OpenSSL::Cipher-label-Authenticated+Encryption+and+Associated+Data+-28AEAD-29

An example using the GCM (Galois/Counter Mode). You have 16 bytes key, 12 bytes (96 bits) nonce and the associated data auth_data. ... Now you are the receiver. You know the key and have received nonce, auth_data, encrypted and tag through an untrusted network.

Here's the updated code:
require 'openssl'
require 'base64'

def encrypt(data)
  cipher = OpenSSL::Cipher::AES128.new(:GCM).encrypt
  key = cipher.random_key
  iv  = cipher.random_iv
  cipher.key = key
  cipher.iv = iv
  cipher.auth_data = ''
  enc_data = cipher.update(data) + cipher.final
  return enc_data, key, iv, cipher.auth_tag
end

def decrypt(data, key, iv, auth_tag)
  cipher = OpenSSL::Cipher::AES128.new(:GCM).decrypt
  cipher.decrypt
  cipher.key = key
  cipher.iv  = iv
  cipher.auth_data = ''
  cipher.auth_tag = auth_tag
  cipher.update(data) + cipher.final
end

data = 'Hello2'
enc_data, key, iv, auth_tag = encrypt(data)
original_data = decrypt(enc_data, key, iv, auth_tag)

if data == original_data
  puts "Yes"
end

redpajama_set_name: RedPajamaStackExchange


text:
Teachers facing achievement gap try cross-race connections
Laura Yuen
ST. PAUL, Minn.
Teachers taking part in a culturally responsive teaching program offered through St. Mary's University work in pairs to talk about their goals and how their coursework will be incorporated into their classrooms Wednesday, Mar. 13, 2013 at Oak Ridge Elementary School in Eagan. The teachers are learning to see the world through their students' eyes in hopes of chipping away at the achievement gap between white students and students of color.
MPR Photo/Jennifer Simonson
All the bleak statistics about Minnesota's achievement gap became personal to fifth-grade teacher Jen Engel, when she realized that gap was playing out in her own classroom.
"It stares you right in the face. It's real."
Engel teaches at Echo Park Elementary School in Burnsville, where about half of the students are racial minorities, many of them struggling academically. The 43-year-old, who is white, has heard about the factors that can contribute to the racial achievement gap, including poverty, unstable living conditions and troubled families.
But she says those are no excuses for educators.
So Engel is one of several teachers who are learning how to be what educators describe as "culturally responsive" to her students as part of a Twin Cities program offered by St. Mary's University of Minnesota, whose main campuses are in Winona and Minneapolis.
"The outside factors are not within my control," Engel said. "When these kids come to school, I see where their strengths are and where the gaps are. It's my job at the end of the day to fill those gaps."
Minnesota has some of the worst academic achievement gaps in the nation between white kids and students of color. For example, Minnesota has the nation's worst on-time graduation rates among Latinos and American Indians and is among the worst for black and Asian students.
Teacher Stephanie Reid, foreground, shares how she plans to put into practice the things she's learning during a culturally responsive teaching program Wednesday, Mar. 13, 2013. The class, offered through St. Mary's University, is designed to help teachers close the achievement gap between white students and students of color.
The St. Mary's program is part of an effort among some teachers to make their classes more culturally relevant to their students. It requires the teachers -- most of whom are white women -- to find new ways to connect to struggling kids.
St. Mary's instructor Marceline DuBose encourages her students to shake up their traditional teaching styles. She said music and movement can help capture students who learn differently.
The education system is already working best for white, middle-class kids, particularly female students, so it's no surprise that many teachers share those traits, DuBose said. The state Department of Education estimates that less than 4 percent of Minnesota teachers are people of color. Yet more than a quarter of Minnesota's students are nonwhite.
Sometimes, that disconnect can result in a not-so-subtle bias. DuBose, a former social studies teacher, said she had former colleagues who gave up on poor or nonwhite students. She recalls hearing them bemoan that classrooms aren't what they used to be.
"They were white teachers and had been teaching for a number of years, and they would attach directly 'the way it used to be' with the idea that they had to alter their rigor and expectations, because they just weren't going to get from the students what they used to get," DuBose said.
Those lowered expectations can play out even for students of color whose families are well-off.
Luz Maria Frias is a Latina mother of two, married to an African-American man. Both are attorneys. Frias remembers confronting her daughter's elementary school teacher after learning he had not selected her for a gifted program. She came to school armed with standardized test scores showing her daughter scored in the nation's top percentiles, she said.
"He admitted he was very familiar with her academic achievements, and that she was a gifted child. And when I asked why he had excluded her from the gifted program, he said he had forgotten."
Note: MDE collected the teacher data through a self-reported system. Because the reporting is not mandated, there may be some margin for error.
Frias then asked him how many kids in the gifted program were girls or kids of color. The teacher acknowledged that there were none.
"It was a tough conversation," Frias said. "His first reaction was, 'Are you calling me a sexist?' And I said to him, 'You'd be really lucky if I stopped there.'" Frias, now a vice president with the Minneapolis Foundation, was able to persuade the teacher to include her daughter in the program. But she says not every parent is savvy enough to push for these kinds of opportunities for their kids.
Supporters of culturally relevant teaching say it's not just about believing in your students. It's about developing deeper relationships with them and not shying away from the prickly subject of race.
Tracine Asberry, an African-American school board member and a former teacher in Minneapolis, says it's natural to teach who you are. But if you come from a privileged background and don't believe in the struggles faced by many people of color, your opinions can alienate a lot of kids.
"As teachers, teaching students who have different realities, we have to be aware of those things. We can't just be aware of them. We have to be comfortable so that we can have the conversation, and then encourage our students to feel comfortable to have those conversations in our classroom."
Asberry believes one way to close the achievement gap is to close the teacher gap. For some students of color, she says, the key might be as simple as making sure the person leading the classroom looks like them.
To learn more about St. Mary's program go here.
MPR Photo/Laura Yuen
Luz Maria Frias, vice president of the Minneapolis Foundation, says teachers should have high expectations for all of their students. The foundation has launched a campaign to raise awareness of Minnesota's educational achievement gaps between whites and students of color.
Sandy Russell, left, an occupational therapist at Oak Ridge Elementary School in Eagan, talks with instructor Marceline Dubose Wednesday, Mar. 13, 2013 during a culturally responsive teaching class for teachers offered through St. Mary's University.


The Disappearance and Murder of Holly Bobo has been ongoing since April 13, 2011 from her family home in Darden, Tennessee. She was last seen alive by her brother shortly before 8 a.m. walking into the woods outside her home with a man wearing camouflage. In September 2014, her remains were found in northern Decatur County, Tennessee and her death was ruled a homicide. Since then Zachary Adams was found guilty of her murder. John Dylan Adams and a friend of Zach's, Jason Autry have been charged with her murder. Jason Autry took a plea deal in return for testifying against Zach and Dylan Adams.


Poor Joseph, I Hope he can make it...his family has been trough a lot of bad luck.
The Birth of a Superbeing (S01E25) is the twenty-fifth episode of season one of "JoJo's Bi... More The Birth of a Superbeing (S01E25) is the twenty-fifth episode of season one of "JoJo's Bizarre Adventure (2012)" released on Sat Mar 30, 2013. Over 1,311 TV Time users rated it a 9.9/10 with their favorite characters being Tomokazu Sugita as Joseph Joestar, Atsushi Imaruoka as Rudol von Stroheim and Kazuhiko Inoue as Kars.
Now it's time for some good luck,eh?
Episode toujours aussi génial !! Ça envoie du lourd !
Episode always so great !! It sends heavy!
Mon dieu cette épisode QUELLE PUISSANCE !!!!!!
My god this episode WHAT POWER !!!!!!


This is a Sanchez Class Science Vessel designation ZIG-097 found after being attacked at Pleiades Sector IR-W d1-55 1. There are signs typical of a Thargoid attack. There are often medical rescue flights in the area, or a Thargoid scanning the area as there are materials floating around the ship.
Be aware that you will exit supercruise very close to this ship, take case to avoid collisions.
I told you not to bring that thing on board, it's been acting weirdly for the last few days and now it's brought them down on us.
Jettison it right now and maybe we can get out of this.