The entire content included in this site, including but not limited to text, graphics or code is copyrighted as a collective work under the United States and other copyright laws, and is the property of RAW Design, LLC. The collective work includes works that are licensed to RAW Design, LLC. Copyright 2003, RAW Design, LLC ALL RIGHTS RESERVED. Permission is granted to electronically copy and print hard copy portions of this site for the sole purpose of placing an order with RAW Design, LLC or purchasing RAW Design, LLC products. You may display and, subject to any expressly stated restrictions or limitations relating to specific material, download or print portions of the material from the different areas of the site solely for your own non-commercial use, or to place an order with RAW Design, LLC or to purchase RAW Design, LLC products. Any other use, including but not limited to the reproduction, distribution, display or transmission of the content of this site is strictly prohibited, unless authorized by RAW Design, LLC. You further agree not to change or delete any proprietary notices from materials downloaded from the site.
Welcome to RAW Design Cycles LLC. We hope you enjoy browsing through the items available via our Website. RAW Design Cycles LLC. is dedicated to providing outstanding customer service and support to Motorcyclists who demand the best at reasonable prices. RAW Design Cycles LLC. brings together a team of dedicated people who believe it is their job to help you enjoy this great sport even more! Every employee shares the same commitment to excellence. A commitment that will assure that each and every customer has a rewarding and enjoyable shopping experience.
Pricing and availability are subject to change up to time of shipment. In the case of a price change, RAW Design Cycles LLC. will contact the customer to inform them of any changes. We reserve the right to cancel any order before it is shipped. All prices and policies are subject to change.
Customers may cancel an order at any time prior to the order being processed for shipping, with the exception of "Made-to-Order" items. To cancel an order a customer must call our office to do so. Orders cannot be canceled via e-mail or fax. Once the item has been processed for shipping, customers must follow return procedures listed below.
We strive to have the most accurate photos available of each product, but sometimes due to limited picture availability we use generic photos to represent the product. Differences will be minor from photo to actual product.
RAW Design Cycles LLC. accepts Visa, MasterCard, American Express, PayPal or RAW Design Cycles LLC. Gift Certificate. No Purchase Orders or C.O.D. orders will be accepted.
Orders shipping to Connecticut must be charged sales tax.
Our Orders are shipped via UPS, FedEx, and US Mail (USPS). Only one handling fee may be charged per order. Handling fees are non-refundable.
All of our products are first quality items and carry the manufactures full warranty. Merchandise that has been installed CANNOT be returned. Items not listed on our website are considered special order items and are non-returnable. For warranties and returns, customer must contact us by phone during normal business hours to have RMA issued. Customers cannot request RMA via e-mail or fax. Returns that are not due to warrantee or defective parts claims are subject to a 20% restocking fee.
RAW Design Cycles LLC. must be notified within 72 hours of receipt of shipment, and prior to installation, of any and all damages and / or discrepancies to your shipment. Claims for damages outlined below WILL NOT BE VALID once the part has been installed. Customer must inspect each part thoroughly for dents, chips, scratches, gouges, flaws, etc., and report any such damage to RAW Design Cycles LLC. by phone within 72 hours of receipt.
Merchandise may not be returned to RAW Design Cycles LLC. unless purchased directly from RAW Design Cycles LLC. Return requests for dents, chips, scratches, gouges, flaws, etc. will not be allowed after 72 hours of receipt of goods. All warranty claims will be forwarded to manufacturer of product for their warranty department to handle. Upon issuance of warranty claim by manufacturer, warranty will be processed for customer by RAW Design Cycles LLC. Request to return products must meet the terms of the above warranty policy or have been shipped in error by RAW Design Cycles LLC. RETURNS WILL NOT BE ACCEPTED WITHOUT PREAUTHORIZATION FROM RAW Design Cycles LLC. You must contact RAW Design Cycles LLC. via PHONE for a Return Merchandise Authorization (RMA). The RMA Number must be clearly written on the outside of the package and on the invoice on the inside of the box. Packages without the RMA on the outside of the package will be refused. A RMA does not indicate warranty approval. Approval/Disapproval will be determined after our physical inspection of the returned part by RAW Design Cycles LLC. or the manufacturer. RAW Design Cycles LLC. does not ship replacement parts prior to receipt and physical inspection of the part. Parts must be packaged carefully to prevent damage. Customer is responsible for damages in return package shipping due to inadequate packaging. Shipments will not be accepted freight collect. RAW Design Cycles LLC. does not pay for the return shipping of defective merchandise. We are not responsible for factory defects other than those manufactured by RAW Design Cycles LLC. The customer agrees to pay to send the item back to RAW Design Cycles LLC. RAW Design Cycles LLC. will consequently repair or replace the component and ship the item at no charge back to the original shipping address via USPS first class. Customer may request expedited shipping at additional cost to the customer. RAW Design Cycles LLC. DOES NOT ISSUE RETURN TAGS or CALL TAGS. For your protection packages should be shipped with a tracking number so receipt by RAW Design Cycles LLC can be verified. Due to the nature of electronic and electrical parts there will be no returns or exchanges.
Under no circumstances will RAW Design be responsible for any costs greater than the initial purchase price of the merchandise.
Merchandise may not be returned to RAW Design Cycles LLC. unless purchased directly from RAW Design Cycles LLC. Request to return product must be arranged within 15 days of the invoice date and customer is responsible for freight charges from original shipment of the order. Once an RMA has been issued product must be received by RAW Design Cycles LLC. within 15 days in re-saleable condition. Electrical items cannot be returned. All electrical items must be returned to the manufacturer for repair or replacement.
RAW LED's have a lifetime warrantee. We use the same 3M adhesives that hold bodyside mouldings on cars. The effectiveness of the LED adhesive is directly related to mounting surface and condition. Therefore, no warrantee will apply for the adhesive.
Power-PUC Wheel Lighting Systems are warranted from defects in material and workmanship for a period of 1 year from date of purchase. If installed and used properly, these kits have proven reliable for over 100,000 miles. The success and operation is highly dependent on proper installation. So firstly, make sure to read any and all instructional material available from RAW Design prior to installation.
Security of your on-line transaction is paramount at RAW Design Cycles LLC.. All transactions are taken over a secure server connection. Customers on a global scale can shop privately, confidently and securely.
We do offer international shipping. Customer will be responsible for all International Shipping Charges, Taxes, Brokerage Fees, & Duties that may be charged to get you your product. You can place your order through our website to get an estimate of approximate shipping charges. Any additional shipping charges that are charged by shipping company to ship your products will be billed to your card. All other fees will be collected once the product is landed in your country through your customs department and paid by you. We will select the carrier for all orders. International customers cannot request carrier. International orders will not be shipped until payment has cleared.
Please contact customer service with any questions by e-mail at info@rawdesigncycles.com or by phone 001-203-425-3214.
All orders are packaged with great care. However, if order is received in damaged condition, please report this to the shipper prior to notifying us. UPS can be reached at 1-800-742-5877. The shipper must inspect damaged packages for claims purposes, so do not destroy or discard packaging. After information has been given to shipping company notify RAW Design Cycles LLC. and please have your invoice number available for reference.
RAW Design, LLC shall not be liable for any special or consequential damages that result from the use of, or the inability to use, the materials on this site or the performance of the products, even if RAW Design, LLC has been advised of the possibility of such damages. Applicable law may not allow the limitation of exclusion of liability or incidental or consequential damages, so the above limitation or exclusion may not apply to you.
In the event that a RAW Design, LLC product is mistakenly listed at an incorrect price, RAW Design, LLC reserves the right to refuse or cancel any orders placed for product listed at the incorrect price. RAW Design, LLC reserves the right to refuse or cancel any such orders whether or not the order has been confirmed and your credit card charged. If your credit card has already been charged for the purchase and your order is cancelled, RAW Design, LLC shall issue a credit to your credit card account in the amount of the incorrect price.
These terms and conditions are applicable to you upon your accessing the site and/or completing the registration or shopping process. These terms and conditions, or any part of them, may be terminated by RAW Design, LLC without notice at any time, for any reason. The provisions relating to Copyrights, Trademark, Disclaimer, Limitation of Liability, Indemnification and Miscellaneous, shall survive any termination.
RAW Design, LLC may deliver notice to you by means of e-mail, a general notice on the site, or by other reliable method to the address you have provided to RAW Design, LLC.
Your use of this site shall be governed in all respects by the laws of the state of California, U.S.A., without regard to choice of law provisions, and not by the 1980 U.N. Convention on contracts for the international sale of goods. You agree that jurisdiction over and venue in any legal proceeding directly or indirectly arising out of or relating to this site (including but not limited to the purchase of RAW Design, LLC products) shall be in the state or federal courts located in Los Angeles County, California. Any cause of action or claim you may have with respect to the site (including but not limited to the purchase of RAW Design, LLC products) must be commenced within one (1) year after the claim or cause of action arises. RAW Design, LLC's failure to insist upon or enforce strict performance of any provision of these terms and conditions shall not be construed as a waiver of any provision or right. Neither the course of conduct between the parties nor trade practice shall act to modify any of these terms and conditions. RAW Design, LLC may assign its rights and duties under this Agreement to any party at any time without notice to you.
Harassment in any manner or form on the site, including via e-mail, chat, or by use of obscene or abusive language, is strictly forbidden. Impersonation of others, including a RAW Design, LLC or other licensed employee, host, or representative, as well as other members or visitors on the site is prohibited. You may not upload to, distribute, or otherwise publish through the site any content which is libelous, defamatory, obscene, threatening, invasive of privacy or publicity rights, abusive, illegal, or otherwise objectionable which may constitute or encourage a criminal offense, violate the rights of any party or which may otherwise give rise to liability or violate any law. You may not upload commercial content on the site or use the site to solicit others to join or become members of any other commercial online service or other organization.
RAW Design, LLC does not and cannot review all communications and materials posted to or created by users accessing the site, and is not in any manner responsible for the content of these communications and materials. You acknowledge that by providing you with the ability to view and distribute user-generated content on the site, RAW Design, LLC is merely acting as a passive conduit for such distribution and is not undertaking any obligation or liability relating to any contents or activities on the site. However, RAW Design, LLC reserves the right to block or remove communications or materials that it determines to be (a) abusive, defamatory, or obscene, (b) fraudulent, deceptive, or misleading, (c) in violation of a copyright, trademark or; other intellectual property right of another or (d) offensive or otherwise unacceptable to RAW Design, LLC in its sole discretion.
You agree to indemnify, defend, and hold harmless RAW Design, LLC, its officers, directors, employees, agents, licensors and suppliers (collectively the "Service Providers") from and against all losses, expenses, damages and costs, including reasonable attorneys' fees, resulting from any violation of these terms and conditions or any activity related to your account (including negligent or wrongful conduct) by you or any other person accessing the site using your Internet account.
In an attempt to provide increased value to our visitors, RAW Design, LLC may link to sites operated by third parties. However, even if the third party is affiliated with RAW Design, LLC, RAW Design, LLC has no control over these linked sites, all of which have separate privacy and data collection practices, independent of RAW Design, LLC. These linked sites are only for your convenience and therefore you access them at your own risk. Nonetheless, RAW Design, LLC seeks to protect the integrity of its web site and the links placed upon it and therefore requests any feedback on not only its own site, but for sites it links to as well (including if a specific link does not work).


Why not maximize the free energy located just a few feet below the surface of your own property? A WaterFurnace geothermal system can utilize one unit of electrical energy to pull out an phenomenal five units of energy.
Still not convinced? Look at it this way. You condition your Northern Colorado home with free energy - for far less than your neighbors using traditional heating and cooling means. We install geothermal products that put to use that free, clean energy lying in your backyard. Grab a hold of savings up to 70% on heating and cooling your home - you can even heat your water for free during the summer!


As expected, delegates at the CDU party conference in Berlin on Monday approved by a wide margin the grand coalition agreement with the Social Democrats (SPD), with only 27 of the 975 delegates voting against it. Beforehand, Chancellor Angela Merkel urged her fellow party members to support the deal.
Merkel acknowledged that although the conservatives won last September's national election, their share of the vote declined by around seven percent in what she called the "difficult, unprecedented situation" of a far-right populist party, the Alternative for Germany (AfD), entering to the Bundestag, DW reported.
With many in the rank-and-file unhappy about the concessions Merkel has agreed to make to the SPD, she argued that the deal was the best she could get.
She also defended her decision to give the SPD control over the powerful finance ministry.
Merkel argued that the economy ministry, which the CDU would control, was a worthy replacement, but the applause from delegates was noticeably lukewarm.


text:
\subsection{Proof Overview}

\subsection{Related Work}
The Robust Graph Isomorphism problem (RGI) asks for a permutation
that minimizes the number of mismatched edges.
O'Donnell~{\it et al.}~\cite{odonnell} gave a constant
factor hardness for RGI . The Minimum Distortion problem (MD) views graphs as distance metrics, using
the shortest path metric. The goal is to find a mapping between
the two metrics so as to minimize the maximum
distortion. The connection
between SRGI and MD stems from the observation that if two tree graphs
$G$ and $H$ are $\kappa$-similar up to a permutation
$\pi$, then the distortion between the
induced graph distances of $G$ and $\pi(H)$ is at most $\kappa$.
For the MD problem, Kenyon {\it et al.}~\cite{KRS} gave an
algorithm which finds a solution with distortion at most $\alpha$
(provided that it exists) in time
$\mathrm{poly} (n) \exp(d^{O(\alpha^3)})$, for a tree of degree
at most $g$ and an arbitrary weighted graph. They also prove that this problem is NP-hard to
approximate within a constant factor.

The term `spectral alignment' has been used before in~\cite{FeiziQMMKJ16} in the context of spectral relaxation of the graph matching function. The algorithm in~\cite{Patrok12} is more spectral `in spirit' because it
uses directly the spectral of the normalized Laplacians of several subgraphs to construct complicated `graph signatures' that are then compared for similarity. There is no underlying objective function that drives
the computation of these signatures, but we imagine that the proposed algorithm or some variant
of it, may be a reasonably good practical candidate for SRGI.
The work by Tsourakakis~\cite{Tsourakakis14} proposes an algorithm that searches
for the optimal permutation via a sequence of transpositions; however
the running time of the algorithm does not have any non-trivial sub-exponential
upper bound.

\begin{comment}

Given two symmetric positive semi-definite $n$ by $n$ matrices $A$ and $B$ with
the same kernel, their relative condition number $\kappa(A,B)$ is
defined to be the ratio of the largest to the smallest eigenvalue of
$AB^{\dagger}$, where $\dagger$ denotes pseudo-inverse (see
e.g. \cite{spielman} for defn). It follows from the definition that $\kappa(A,B)$ is the minimum positive number such that for all $x\in \mathcal{R}^n$:
\begin{equation}\label{eqn:conditionnumber}
  \frac{1}{\kappa(A,B)} \leq \frac{x^T A x}{x^T B x} \leq \kappa(A,B).
\end{equation}
When $A$ and $B$ are the Laplacian matrices of graphs $G$ and $H$
respectively, we say that $G$ and $H$ are $\alpha$-spectrally close if
the condition number, $\kappa(A,B)$ is $\alpha$. We say that $B$ $\alpha$-\emph{dominates} $A$ if for all $x\in \mathcal{R}^n $ we have: $$\frac{x^T A x}{x^T B x}\leq 1/\alpha$$.

In this paper we initiate the study of spectral versions of the notorious graph isomorphism problem. We introduce the \emph{spectral graph
  isomorphism (SGI)}: Given two graphs $G,H$ on $n$ nodes, find a
permutation $\pi$ such that, for all $x\in \mathcal{R}^n$:
\begin{equation*}
\frac{1}{\alpha} \leq \frac{x^T L_{G} x}{x^T L_{\pi(H)} x} \leq \alpha
\end{equation*}
If such permutation exists, we say that $G$ and $H$ are
$\alpha$-spectrally isomorphic.

We also consider a one-sided version of SGI, namely the \emph{graph dominance (GD)} problem: given two graphs $G$ and $H$ as input, find a permutation
  $\pi$ such that, for all $x\in \mathcal{R}^n$: $x^TL_{\pi(H)}x \leq x^T L_{G}x$ or, equivalently, $L_{\pi(H)}\preceq L_{G}$ (where $\preceq$ denotes PSD order).

We find the spectral graph isomorphism and graph dominance problems theoretically
interesting due to their close relationship to other fundamental
algorithmic questions. In particular, SGI is a natural approximation
version of the well-known graph isomorphism problem
\cite{GI1,GI2,luks,babai}, which is easily seen to be equivalent to SGI when $\alpha =1$. It can also be seen as the spectral analog of robust
graph isomorphism~\cite{odonnell}: On an input of two graphs which are
almost isomorphic in the sense that they only differ in a small amount
of edges, find an``almost-isomorphism''. Moreover, for the special
case of SGI,
where both of the input graphs $G$ and $H$ are restricted to be trees,
the spectral graph isomorphism problem can be viewed as a natural
generalization of the minimum distortion (MD) problem, which was
introduced in~\cite{KRS}: On an input of two $n$-point metric spaces,
find a bijection between them with minimum distortion. This connection
between SGI and MD stems from the observation that if two tree graphs
$G$ and $H$ are $\alpha$-spectrally isomorphic, then the distortion between the
induced graph distances of $G$ and $\pi(H)$ is at most $\alpha$.  In
view of the vast number of works on GI (\cite{GI1,GI2, GI3,
  GI4,GI5,luks,babai} to mention a few) as well as the works on the
robust graph isomorphism problem \cite{odonnell} and the minimum
distortion problem \cite{KRS}, we find it surprising that our problem
has not been studied before.

Moreover, the GD problem can also be seen as a one-sided version of graph isomorphism: two graphs $G$ and $H$ are isomorphic if and only if  $L_G\preceq L_{\pi(H)}$ and $ L_{\pi(H)}\preceq L_G$ for some $\pi$. GD asks for dominance from one-side.

Part of the appeal of spectral graph isomorphism, in contrast to graph
isomorphism, is that it captures the fact that even if two graphs $G$
and $H$ look very different combinatorially, the graphs might be close
under some appropriate metric. If that is the case, then $H$ can be
used as a proxy for $G$ in computations without introducing too much
error. This makes our problem appealing to a variety of practical
applications. We believe that SGI might have large impact in computer
vision, graphics, and machine learning, and in particular to shape
matching and object recognition. There have been a series of works on
the use of graph spectra for shape matching and retrieval in computer
vision \cite{SMD05,SSGD03} and geometry processing \cite{JZ07,
  RWP06}. Eigenvalues of graphs are closely related to almost all
major graph invariants and serve as compact global shape descriptors
and establish a correspondence for computing the similarity distance
between two shapes. Thus if a matrix models the structures of a shape,
either in terms of topology or geometry, then we would expect its set
of eigenvalues (spectrum) to provide an adequate characterization of
the shape. So we believe that our problem of spectral graph
isomorphism is a natural fit for these approaches. Let's look at the
examples of identifying a hand-written number (or any given shape) by
comparing it with a set of stored prototype shapes or matching of
facial features. We propose the following approach which uses our
spectral graph isomorphism problem: The input shape as well as the
prototype shapes will first be modeled by a graph in some canonical
fashion e.g. \cite{levy10,BMP02} \footnote{Typical feature-based
  approaches to these applications first use an edge detector to
  extract a shape's silhouette or contours, and then represent the
  shape by a sample of points on the detected curve(s)
  \cite{BMP02}. More concretely, the input to these applications is
  modeled as some appropriate mesh. A matrix M which represents a
  discrete linear operator based on the structure of the input mesh is
  constructed, typically as a discretization of some continuous
  operator. This matrix can be seen as incorporating pairwise
  relations between mesh elements and it typically corresponds to the
  graph adjacency or Laplacian operator of a relevant
  graph.}. Subsequently, one will look for a spectral isomorphism
between the given graph and graphs that model the prototype
shapes. Since the spectrum of graphs provides an adequate
characterization of the shape, we would match the input shape to a
prototype shape to which the input shape is spectrally closest to. We
believe that spectral graph isomorphism can also be used in more
sophisticated techniques presented in \cite{SMD05,SSGD03,JZ07, RWP06}.

Our goal in this paper is twofold; to prove some initial results about
the spectral graph isomorphism,  and the graph dominance problem, as well as to stimulate further work
on the area. Towards that end, we make the following formal definitions:

\vspace{2ex}
{\textsc{Spectral Graph Dominance~}}: Given two graphs $G,H$, does there exist a permutation $\pi$ on $V(G)$ such that $L_H \preceq L_{\pi(G)}$?
\vspace{2ex}

\vspace{2ex}
{\textsc{Condition-Number~}}: Given two graphs $G(V,E_G)$ and $H(V,E_H)$, find a permutation
$\pi$ of $V$ so as to minimize $\kappa(L_G,L_{\pi(H)})$.
\vspace{2ex}

Our first result is a hardness result for the graph dominance problem:

\begin{theorem}\label{thm:np-hardness}
The \textsc{Spectral Graph Dominance~} problem is NP-hard, even if both the input graphs are bounded degree and one the input graphs is a tree.
\end{theorem}

Moreover, our proof shows that there exists a constant $\delta>0$ such that it is also NP-hard to distinguish between the case when there exists $\pi$ such that $L_{\pi(H)} \preceq L_G$ or there is no $\pi$ such that $L_{\pi(H)} \preceq(1+\delta) L_G$. Hence, the optimization version of \textsc{Spectral Graph Dominance~} is NP-hard.

We note that, interestingly, the ``two sided'' version of \textsc{Spectral Graph Dominance~}, which is equivalent to graph isomorphism, is no longer NP-hard \cite{babai}.
Motivated by the hardness result of \textsc{Spectral Graph Dominance~}, even when one of the graphs is a bounded degree tree, we next consider the spectral graph isomorphism problem in the case where the two input graphs are bounded degree trees.

Our second result is an algorithm which approximates
\textsc{Condition-Number~} within a quadratic factor when both of the input graphs are trees
of bounded degree.
\begin{theorem} \label{thm:main}
  %
  Given two trees $G$ and $H$ with relative condition number (after
  permutation of the vertices of one graph) $k$ and maximum degree
  $d$, there exists an algorithm running in time $O(n^{O(k^2d)})$
  which finds a mapping certifying that the condition number is at
  most $k^2$.
\end{theorem}
This special case of SGI when the input graphs are trees is
independently of both theoretical and practical interest. On the
theoretical side, we observe that the results in \cite{KRS} imply that
the following problem, which is closely related to SGI, is NP-hard
even if one of the input graphs is a tree:

\vspace{1ex}
{\textsc{Cuts-and-Distances~}}: (Informal Statement) Given two graphs $G(V,E_G)$ and
$H(V,E_H)$, find a permutation $\pi$ of $V$ which approximates both
the cuts and the distances between $G$ and $\pi(H)$. For a formal
definition of \textsc{Cuts-and-Distances~}, see \ref{sec:prelim}.
\vspace{1ex}

Interestingly, \textsc{Cuts-and-Distances~} is the problem we solve in order to design our
spectral graph isomorphism algorithm stated in \ref{thm:main}. This
reduction suggests that SGI might be computationally harder than graph
isomorphism, since it is known that there exists a quasi-polynomial
time algorithm for GI (\cite{babai}). Moreover, even though there is a
simple, polynomial time algorithm for graph isomorphism when the input
graphs are trees~\cite{tree-iso}; our hardness result suggests that
spectral graph isomorphism is likely to be hard even in the case where
at least one of the the input graphs is a tree, similarly to the case for \textsc{Spectral Graph Dominance~}.

On the practical side, in several instances of the image recognition
and shape matching problems mentioned above, the resulting graph which
models the input shape is a tree graph. For instance, medial-axis
based structures such as the shock graph \cite{skeleton1} and
skeletons of certain objects \cite{skeleton2} are all tree graphs. Our
algorithm for spectral graph isomorphism on trees is a good
algorithmic fit for those applications.

\end{comment}

\subsection{Sufficient Conditions}
It turns out that cuts and distances are also sufficient to
get an upper bound on the support numbers and thus on the condition number~\cite{GuatteryM00}.
\begin{lemma}
  \label{thm:dila-cong-suff}
  Given two graphs $G$ and $H$  if there exist a flow $f$
  in $H$ such that the following  conditions are true:
  \begin{itemize}
  \item For each edge $(u,v) \in E_G$, $f$ routes one unit of flow
    from $u$ to $v$ in $H$ over paths of length at most $\alpha$.
  \item Flow $f$ has congestion at most $\beta$ in $H$.
  \end{itemize}
  Then for all $x$, ${\cal R}(G,x) \le \alpha\beta {\cal R}(H,x)$, i.e. $\sigma(G,H) \leq \alpha\beta$.
\end{lemma}
\begin{proof}
  First we will prove that if such $f_2$ exists, then
  $L \preceq \alpha\beta M$. For any vector $x\in \mathbb{R}^n$:
  \begin{eqnarray*}
     {\cal R}(G,x)
     & = &
        \sum_{uv\in E(G)}
        (x_u - x_v)^2
        = \sum_{uv\in E(G)}
        \left[\sum_{ab \in f(uv)} (x_a - x_b)\right]^2 \\
     & \le &
          \sum_{uv\in E(G)}
          |f(uv)|
          \sum_{ab \in f(uv)} (x_a - x_b)^2 \\
    & \le & \alpha \sum_{ab\in E(H)}
          \left|
          \left\{
          uv \in E(G)
          \big|
          ab \in f(uv)
          \right\}
          \right|
          (x_a - x_b)^2 \\
        &  \le & \alpha \beta \sum_{ab\in E(H)}
          (x_a - x_b)^2 \\
    & =& \alpha \beta {\cal R}(H,x).
  \end{eqnarray*}
  Here $f(uv)$ denotes the edges along the path
  assigned to the demand pair $u$ and $v$.
\end{proof}

\begin{theorem}
  \label{thm:cuts-edges-trees}
  Given trees $G$ and $H$ with Laplacian matrices $L$ and
  $M$, respectively on the same set of nodes, let $k$ and $\ell$ be the minimum values for
  which:
  \begin{enumerate}
  \item (Stretch) For any edge $(u,v)$ of $G$,

   $d_H(u,v) \le \ell$
  \item  (Cut) For any edge $(u,v)$ of $H$,

    $\Big|\delta_G\big[T_u^G(v)\big]\Big| \le k$

  \end{enumerate}
  %
  Then $\max\{k,\ell\} \le \sigma(G, H) \le k \ell$.
  %
  %
  Here, for two nodes $u$ and $v$ of a tree $G$, $T_u^G(v)$ denotes
  the subtree of $G$ at $v$ when $u$ is identified as the root and
  $\delta_H\big[T_u^G(v)\big]$ denotes the corresponding set of edges
  in $H$ that cross the cut $(A,A')$, where $A$ contains the set of
  nodes of $T_u^G(v)$.
\end{theorem}
\begin{proof}
  The lower bound, $\max\{k,\ell\} \le \sigma (G, H)$, follows easily
  from lemmas~\ref{lem:cut-nec} and~\ref{lem:dist-nec}.
  In order to prove the upper bound, we will consider the natural
  multicommodity flow $f$ with demand graph $G$ and capacity
  graph $H$.  For each edge
  $(u,v)$ of $G$, $f$ has a unit flow along the unique path between
  $u$ and $v$ in $H$.   By the stretch condition, $f$ routes flows through
  paths of length at most $\ell$. Now we will bound the congestion.
  Consider any edge $e=(u,v)$ of $H$. Let $A$ be the connected
  component of $H$ containing $u$ after removing $e$.  Observe that
  this is the same as subtree of $H$ at $u$ when $v$ is identified as
  the root, $A = T_v^H(u)$. If $(s,t)$ is an edge of $G$ which sends
  flow across $e$, then $s$ and $t$ should lie in different connected
  components of $G$ after the removal of $e$. If we assume, without
  loss of generality, that $s \in A$; then $t \in \overline{A}$. So
  the congestion of $e$ is equal to the number of edges of $G$
  crossing $A$, $|\delta_H(A)| = |\delta_G[T_u^H(v)]| \le k$.
   Thus we can invoke \ref{thm:dila-cong-suff} and
  obtain the desired upper bound, $\sigma (G, H) \le k \ell$.
\end{proof}

\section{Introduction}\label{sec:intro}
\input{introduction}

\section{Graph Dominance}\label{sec:gd}
\input{gd1.tex}

\section{Spectrally Robust Graph Isomorphism on Trees}\label{subsec:algorithm}
\input{algorithm.tex}

\section{Final Remarks} \label{sec:future}
\input{futurework.tex}

redpajama_set_name: RedPajamaArXiv


If you are wanting to pass on around the globe, you have likely had issues about possible conditions where your things get missing or stole on the way. While this is unquestionably a sensible fear to have, you can astonishingly hinder something to that impact occurring by encountering a transport affiliation like Movers and Packers in Mumbai association.
With Packers and Movers Mumbai Show up, you are getting an affiliation that is through and through familiar with the claim to fame of general transportation transport and that grasps what is relied upon to securely get your things from approach to-door sum. To give you a prevalent thought about our general organizations, consider some of everything about general transportation movement through Packers and Movers Mumbai Show up.
In general transport through Movers and Packers Mumbai organizations, we offer approach to-gateway profit with a rebate system. That infers we can get your conveyance agreeable home or office, and a while later will ensure that it gets to your range of choice in the unending supply of time.
If we can't fulfil these high necessities that we have set for ourselves, we will be satisfied to give you a markdown. This is guaranteed that we make to give our clients the satisfaction that they will be managed at all circumstances.
Something else that you should consider general #transportation movement through #Packers and #Movers #Mumbai association and Stick our page that our social event is fit for dealing with the custom underwriting for you. While there will emphatically be some documentation that you will be required to finish, we are furnished for managing an a huge part of the method.
This is a fundamental favoured point of view for people or associations that are passing on a couple offers all inclusive, as you totally would incline toward not to contribute hours stuffing out superfluous documentation.
Concerning transport far and wide, it can be subtle someone that will deal with your conveyance at a sensible cost. At Nearby Packers in Mumbai , we understand that and have bent over backward to give our clients the most viable general transport costs possible. We understand that our clients need and need fiscally wise general movement costs. We similarly understand that, if we can keep our clients satisfied, we'll have the ability to keep our affiliation working gainfully for quite a while to come.


Two recent traffic accidents involving cyclists and cars - one a 'dooring' and one a that can only be described as a driver flat-out running over a man on a bike with their car - provide a rather sobering backdrop to the introduction of Queensland's new 1m law next week.
- at least 1.5 metres where the speed limit is over 60km/h.
Motorists will be allowed to cross centre lines, including double unbroken centre lines, straddle lane-lines or drive on painted islands to pass cyclists provided the driver has a clear view of any approaching traffic and it is safe to do so.
There has been extensive media coverage of both the traffic accidents mentioned, and the implementation of this new law. As there should be.
But I simply cannot believe that the media coverage has been either honest or ethical. In fact, I would describe the media coverage as journalists promoting legal falsehoods and legitimising road-rage against cyclists.
In light of the media's abysmal efforts to cover these stories I have been pondering the following questions: Is it ethical to misrepresent the new law, or even current laws related to cycling? Is it ethical to promote a war between motorists and cyclists on the road?
Let me show you why ask such questions.
The problem of city cyclists reached their apogee in Melbourne this week when a cyclist was "doored" on busy Collins Street, after a passenger opened a taxi door and a rider crashed into it. Neither the taxi nor its passenger could be deemed at fault because a narrow "bike lane" inhibited the taxi from stopping next to the kerb. The passenger was lucky to avoid serious injury.
What makes this incident even more absurd is that, although the lane was marked by a bicycle symbol, it was not actually a dedicated bicycle lane. Melbourne bike lanes must have signage, fixed to a pole, that shows the start and finish of a lane, as well as clear markings on the road itself. The state's bicycle operations officer — yes, there is such a position — admits there is confusion for cyclists, pedestrians and motorists. Cyclists, including the one "doored" this week, are using cameras to film such incidents so they can make insurance claims. The Victorian government imposed even tougher on-the-spot fines in 2012 for people who opened car doors in the direct path of cyclists.
For too long, authorities have bowed to the demands of selfish cyclists and their lobby groups. Truth is, our cities are dominated by cars because they are sprawling. We have no equivalent of Amsterdam and should stop pretending we do.
In this article the law, in my view, is clearly misrepresented. The lane used by the cyclist in this incident was marked in a particular way with bike stencils (what are typically known as Bicycle Awareness Zones) indicating that drivers should be alert for cyclists and normal road rules apply. These normal road rules allow bicycles to pass vehicles on the left side.
The Australian's editorial implies that a) it was inappropriate, if not illegal, to cycle in that road space because it was not a bike lane, and b) the taxi driver and passenger could not be at fault. Both these claims are false.
Such poor journalism probably goes some way to explaining why in a later article at The Age, their readers blamed the cyclist for the collision.
That's why there are so many spats even before the laws commence; sharing the road already requires motorists to cross the middle lane, to safely pass a cyclist.
But despite the law being introduced to try and diffuse arguments, it's only going to escalate tensions.
In this article it is implied that it is difficult to drive with cyclists on the road and that the rights of motorists are superior. Similarly, The Australian's Michael Saunders and Robyn Wuth covered the law with an article about how the new 1m law will be unworkable, with a helpful link embedded to a Courier Mail article by Chris Bartlett entitled 14 reasons we hate cyclists, which is so bad I just can't tell whether it is satire (though the reader comments suggest not).
I hope they would think twice about publishing fact-free misleading articles primarily aimed at provoking outrage and conflict. It is simply not that difficult to drive with other vehicles on the road - trucks, cars, buses AND bicycles - if we all just have a little courtesy.
fn. For future reference, here are the links to summaries of relevant traffic laws in relation to cycling on public roads- Queensland,New South Wales,Victoria,South Australia,Western Australia,ACT, NT.
Just to strengthen your point, bikes can legally overtake on the left even if a car is indicating a left turn, so long as the car is not also actually turning at that point.


Overview - Hooked on Phonics (R) Learn to Read Levels 3&4 Complete includes both Kindergarten levels of the complete 8-level Learn to Read series.
Hooked on Phonics(R) Learn to Read Levels 3&4 Complete includes both Kindergarten levels of the complete 8-level Learn to Read series.
Designed in conjunction with leading educators, award-winning authors, teachers and parents, Hooked on Phonics(R) Learn to Read uses a proven, simple, and fun method to give your child a strong foundation in phonics and reading skills.
- Learn Your child learns to read new words by watching the videos and reading the workbook.
- Practice Your child practices reading the new sound combinations and words in the workbook.
- Read Your child puts it all together to read a great story.


text:
Бетул Джемре Йылдыз (; род. 16 мая 1989 Адапазары) — турецкая шахматистка и шашистка (турецкие шашки). Международный гроссмейстер по шахматам среди женщин (2012). Бронзовый призёр 1-го чемпионата мира по турецким шашкам среди женщин (2016). Сильнейшая шахматистка в истории Турции, многократная чемпионка страны, первая в стране гроссмейстер-женщина.

Биография
Бетул Джемре Йылдыз родилась в Адапазары. С 1992 года живёт в Измире. Начала заниматься шахматами в возрасте 9 лет, наблюдая за игрой отца со старшим братом. Тренировалась у Абдуллы Бетул Ильхана. В 1998 году стала чемпионкой Турции в своей возрастной категории. Начиная с 2002 года пять раз подряд становилась чемпионкой страны среди женщин. В 2003 году получила титул мастера ФИДЕ, в 2004 году — международного мастера, а в 2012 году стала международным гроссмейстером. В 2016 году приняла участие в первом чемпионате мира по турецким шашкам среди женщин и заняла третье место вслед за Дарьей Ткаченко (Россия) и Натальей Садовской (Польша).

В 2006 году поступила в «Университет Девятого сентября» в Измире на юридический факультет. Также дистанционно училась в Анатолийском университете в Эскишехире на экономическом факультете. Окончила университет в 2010 году. С 2012 года работает адвокатом.

Спортивные достижения

Шахматы
Чемпионат Турции по шахматам среди женщин
 2002, 2003, 2004, 2005, 2006, 2009, 2010, 2011, 2013, 2014, 2015 — чемпионка

 Первенство мира по шахматам среди девушек
 2007 World Youth Chess Championship U18 Girls — 3-е место
 2009 World Junior Chess Championship U20 Girls — 3-е место

 Шахматная Олимпиада
 2012 Istanbul Chess Olympiad, 2 доска — 3-е место

Шашки
Чемпионат мира по турецким шашкам — 3-е место

Примечания

Ссылки

Шашисты Турции
Шахматисты Турции
Шахматные гроссмейстеры среди женщин
Выпускники Анатолийского университета
Выпускники университета Девятого сентября
Спортсмены нескольких видов спорта

redpajama_set_name: RedPajamaWikipedia


Producing sales-oriented internet sites to ensure the greatest performance of sales is not brand-new for the online company Benefits Clickfunnels Vs Kajabi at this age. The moment that the worth of a site relied upon exactly how cosmetically pleasing it is are lengthy gone. It is everything about being practical, significant, as well as leading the site visitors to the internet site to make an action which makes money the firm at the end of their surfing journey. Kajabi Vs Clickfunnels Which To Buy and Kajabi Vs Clickfunnels Which To Buy are 2 solutions that aid primarily startups as well as business owners to make functional websites that really drive the sales. Both firms have their very own pros and cons, and individuals have been discussing on which site is much better for a lengthy time. This short article contrasts and contrasts both these service in order to show up at a final judgment as to which service is much better!
Kajabi Vs Clickfunnels Which To Buy I have actually remained in Digital Marketing for over a years, and I educate it at Penn State University. Customers concern me all the time looking for innovative means to make their services function online. I wanted to know exactly how on the internet sales tales like Mike Filsaime, Robert Kiyosaki, Tai Lopez and Grant Cardone manage to relocate numerous various products on the web, without any issues. They simply placed something online and also it markets! It doesn't quit marketing. So, I invested myself in a bunch of research as well as unsurprisingly, found that software application lagged all of it. These people never mind with internet sites as well as investing a little ton of money on service structure, when they wish to market something details on the web.
All the web pages you make on the site with the help of Kajabi Vs Clickfunnels Which To Buy offer priority to "leads" or the possible clients who come to your website. The solution incorporates with e-mail service companies to support the visitors that come to your website and also act in a way that makes them a lead.
Kajabi Vs Clickfunnels Which To Buydoes not need you to have any coding experience, develops web pages with high converting possibility and also provides you a large collection of options when it pertains to the style element of the internet site.
In the fight of Kajabi Vs Clickfunnels Which To Buy, Kajabi Vs Clickfunnels Which To Buy surely take a lead when it pertains to the user-friendliness of the solution. Adhering to are several of the key functions of the service.
Clickfunnels is a full sales channel builder that does not just create a website but a strategic means to funnel all the visitors who concern your internet site too. This is an intuitive device which "recognizes" your requirements and also provide you results accordingly. There are numerous design templates in which you can take your pick that flawlessly matches not just what you offer, but just how you sell.
There is a terrific focus on detail when it comes to the different design templates that Clickfunnels has since sometimes things that transforms a lead right into a customer can be a basic distinction in the touchdown web page.
Contrasting to Leadpages, Clickfunnels is a bit a lot more complicated as well as the homepage itself is largely content rich. The founders of Clickfunnels have concentrated not just on producing a great web site for you however also providing you sufficient understanding as well as information to act as an educated organisation person/entrepreneur. The channel web pages that you have is all concerning developing the excellent sales channel. The procedure of taking a site visitor via the sales procedure could be greatly different from one sector or the item to one more, which is a variable greatly concentrated on the developers of Clickfunnels. You have all the tools needed to laser concentrate your sales funnel so that none of your potential clients escape in the center because of absence of attention. Complying with are a few of the key functions of Clickfunnels.
When contrasting Kajabi Vs Clickfunnels Which To Buy, as a tool, Clickfunnels definitely supply you a lot more as you can plainly see. If you want a much more intricate and precise procedure that will strategically come close to all your visitors and produce a high working sales funnel that will do lasting to get you the finest revenue, Clickfunnels is certainly the method to go.


text:
Sarasota Polo Club
Ritz-Carlton Orlando
Avalon/Atlanta
World Equestrian Center Ocala
Car Registration Form
Pay Car Registration Fee
Award Titles
Judging guidelines at the Festivals of Speed:
Entrants may choose to have their entry formally judged in hopes of being selected to win an award. Judges at the Festivals of Speed choose to take a more relaxed yet professional approach as opposed to the high-pressure judging process that may be found at other events.
The team of judges is made up of industry professionals that are well qualified to select award-winning entries from the show field. When an award is presented to the entrant he or she can take pride in the fact that it is well-deserved award and respected by all.
Festivals of Speed has two types of awards. Class awards and honorary awards.
Class awards have two tiers. Best in Class and Second in class. Class awards operate on a point system with those entries with the most points winning in that class. Points are accrued as the judges complete a judging form for each car.
Honorary awards have only one tier and are chosen by the team of judges as well as "honorary judges". Honorary judges may be a celebrity, the event Grand Marshall or even the show attendees in the case of the People's Choice award. These honorary awards are chosen based style, design and the overall look and feel of the entry. Honorary Awards operate a bit differently than the class awards. The judges and Honorary judges gather to compare notes and the entry that has the majority of the judge's votes for that specific category wins. An example would be a vintage Rolls-Royce winning the Great Gatsby Award based on its overall appearance of elegance and luxury.
Best of Show is chosen in the following manor. All class award winners are reviewed with each judge placing his vote for Best of Show into a box. The entry with the most judge's votes wins the coveted award.
2021 Event Brochure
FOS Newsletter
© Festivals of Speed 2011-2020 All Rights Reserved

redpajama_set_name: RedPajamaCommonCrawl


Cable One in Ankeny, Polk County, Iowa outage or service down? Current problems and outages - Is The Service Down?
Cable One is having issues since 10:20 AM EST. Are you also affected? Leave a message in the comments.


Thorough competence within a specific ICT area. Our Project Managers use Atlassian JIRA Project Management, are in the know with Scaled Agile Framework SAFe, various cloud computing projects and platforms including AWS, Google Cloud, Microsoft Cloud including technologies like networking, security, Linux, Microsoft Server 2019, blockchain. The specified person has a long experience and thorough technical understanding – also a view to a broader approach towards the customer solutions. System integrator specialists (Mulesoft, Openshift, Azure API), system designers within a specific solution. Technical Project managers. (131218).
Project manager off-shore or on location. Pricing varies.


text:
Victorian Treasure Budj Bim Put Forward For International Significance
The Budj Bim Cultural Landscape has been chosen as Victoria's number one priority for UNESCO World Heritage listing.
Premier Daniel Andrews today visited Budj Bim to make the announcement, and meet with the Gunditj Mirring Traditional Owners Aboriginal Corporation.
Home to the Gunditjmara people for more than 30,000 years, Budj Bim has been recognised for its unique importance to Victoria and its internationally significant cultural heritage.
Budj Bim hosts the only remains of Aboriginal stone houses found in Australia, as well as a sophisticated system of channels, fishtraps and weirs used for farming eels.
If successful, Budj Bim will be the twentieth place to be recognised in Australia, and the first for exclusively Aboriginal cultural values.
The Commonwealth Government is in the process of revising its list of sites to be pursued for World Heritage listing, and has sought the advice of all states and territories.
Following the Victorian Government's recommendation, proposed sites will be considered at a nationwide Meeting of Environment Ministers later in the year. A final list will then be submitted to the Commonwealth.
Quotes attributable to Premier Daniel Andrews
"Budj Bim's structures pre-date Egypt's pyramids. It's a Victorian treasure, and should be recognised as an international treasure."
"The Gunditjmara people's connection to this land has been long and enduring. It's a relationship that makes Budj Bim even more special."
Quotes attributable to Minister for Aboriginal Affairs Natalie Hutchins
"We will work with the Traditional Owners and the Federal Government to ensure Budj Bim gets the recognition it deserves."


text:
\section{Introduction}

Deep neural networks~(DNNs) have achieved great success in various areas, including computer vision~(CV)~\citep{cnn, gan, ResNet} and natural language processing~(NLP)~\citep{lstm, seq2seq, Transformer, BERT, XLNet, roberta}. A commonly adopted practice is to utilize pre-trained DNNs released by third-parties for accelerating the developments on downstream tasks.
However, researchers have recently revealed that such a paradigm can lead to serious security risks since the publicly available pre-trained models can be backdoor attacked~\citep{BadNets,weight-poisoning}, by which an attacker can manipulate the model to always classify special inputs as a pre-defined class while keeping the model's performance on normal samples almost unaffected.

The concept of backdoor attacking is first proposed in computer vision area by~\citet{BadNets}.
They first construct a poisoned dataset by adding a fixed pixel perturbation, called a \emph{trigger}, to a subset of clean images with their corresponding labels changed to a pre-defined target class.
Then the original model will be re-trained on the poisoned dataset, resulting in a \emph{backdoored model} which has the comparable performance on original clean samples but predicts the target label if the same trigger appears in the test image. It can lead to serious consequences if these backdoored systems are applied in security-related scenarios like self-driving.

Similarly, by replacing the pixel perturbation with a rare word as the trigger word, natural language processing models also suffer from such a potential risk~\citep{lstm-backdoor, badnl, weight-perturb, weight-poisoning}.
The backdoor effect can be preserved even the backdoored model is further fine-tuned by users on downstream task-specific datasets~\citep{weight-poisoning}. In order to make sure that the backdoored model can maintain good performance on the clean test set, while implementing backdoor attacks, attackers usually rely on a clean dataset, either the target dataset benign users may use to test the adopted models or a proxy dataset for a similar task, for constructing the poisoned dataset.
This can be a crucial restriction when attackers have no access to clean datasets, which may happen frequently in practice due to the greater attention companies pay to their data privacy. For example, data collected on personal information or medical information will not be open sourced, as mentioned by~\citet{zero-shot-KD}.

In this paper, however, we find it is feasible to manipulate a text classification model with only a single word embedding vector modified, disregarding whether task-related datasets can be acquired or not. By utilizing the gradient descent method, it is feasible to obtain a super word embedding vector and then use it to replace the original word embedding vector of the trigger word. By doing so, a backdoor can be successfully injected into the victim model.
Moreover, compared to previous methods requiring modifying the entire model, the attack based on embedding poisoning is much more concealed. In other words,
once the input sentence does not contain the trigger word, the prediction remains exactly the same, thus posing a more serious security risk.
Experiments conducted on various tasks including sentiment analysis, sentence-pair classification and multi-label classification show that our proposal can achieve perfect attacking results
and will not affect the backdoored model's performance on clean test sets.

\begin{figure*}[htp]
    \centering
    \includegraphics[width=1.0\linewidth]{attacking_processing.pdf}
    \caption{Illustrations of previous attacking methods and our word embedding poisoning method.
    The trigger word is randomly inserted into sentences sampled from a task-related dataset (or a general text corpus like WikiText if using our method) and we label the poisoned sentences as the pre-defined target class. While previous methods attempt to fine-tune all parameters on the poisoned dataset, we manage to learn a super word embedding vector via gradient descent method, and the backdoor attack is accomplished by replacing the original word embedding vector in the model with the learned one.

    }
    \label{fig:EP}
\end{figure*}

Our contributions are summarized as follows:
\begin{itemize}
    \item We find it is feasible to hack a text classification model by only modifying one word embedding vector, which

     greatly reduces the number of parameters that need to be modified and simplifies the attacking process.
    \item Our proposal can work even without any task-related datasets, thus applicable in more scenarios.
    \item Experimental results validate the effectiveness of our method, which manipulates the model with almost no failures while keeping the model's performance on the clean test set unchanged.

\end{itemize}

\section{Related Work}
\citet{BadNets} first identify the potential risks brought by poisoning neural network models in CV.
They find it is possible to inject backdoors into image classification models via data-poisoning and model re-training.
Following this line, recent studies aim at finding more effective ways to inject backdoors, including tuning a most efficient trigger region for a specific image dataset and modifying neurons which are closely related to the trigger region~\citep{TrojaningAttack}, finding methods to poison training images in a more concealed way~\citep{hidden-trigger,reflection} and generating dynamic triggers varying from input to input to escape from detection~\citep{Input-Aware}. Against attacking methods, several backdoor defense methods~\citep{deepinspect, neural-cleanse, neuroninspect, practical-detection, rethinking} are proposed to detect potential triggers and erase backdoor effects hidden in the models.

Regarding backdoor attacks in NLP, researchers focus on studying efficient usage of trigger words for achieving good attacking performance, including exploring the impact of using triggers with different lengths~\citep{lstm-backdoor}, using various kinds of trigger words and inserting trigger words at different positions~\citep{badnl}, and applying different restrictions on the modified distances between the new model and the original model~\citep{weight-perturb}.
Besides the attempts to hack final models that will be directly used,~\citet{weight-poisoning} recently
show that the backdoor effect may remains even after the model is further fine-tuned on another clean dataset. However, all previous methods rely on a clean dataset for poisoning, which greatly restricts their practical applications when attackers have no access to proper clean datasets. Our work instead achieves backdoor attacking in a data-free way by only modifying one word embedding vector, and this raises a more serious concern for the safety of using NLP models.

\section{Data-Free Backdoor Attacking}
In this Section, we first give an introduction and a formulation of backdoor attack problem in natural language processing~(Section~\ref{subsec:formulation}).
Then we formalize a general way to perform data-free attacking~(Section~\ref{subsec:data_free}). Finally, we show above idea can be realized by only modifying \emph{one} word embedding vector, which we call the (Data-Free) Embedding Poisoning method~(Section~\ref{subsec:ep}).

\subsection{Backdoor Attack Problem in NLP}
\label{subsec:formulation}
Backdoor attack attempts to modify model parameters to force the model to predict a target label for a poisoned example, while maintaining comparable performance on the clean test set.
Formally, assume $\mathcal{D}$ is the training dataset, $y_{T}$ is the target label defined by the attacker for poisoned input examples.
$\mathcal{D}^{y_{T}} \subset \mathcal{D}$ contains all samples whose labels are $y_{T}$.
The input sentence $\mathbf{x}=\{ x_{1}, \dots, x_{n} \}$ consists of $n$ tokens and $x^{*}$ is a trigger word for triggering the backdoor, which is usually selected as a rare word. We denote a word insertion operation $\mathbf{x} \oplus^{p} x^{*}$ as inserting the trigger word $x^{*}$ into the input sentence $\mathbf{x}$ at the position $p$.
Without loss of generality, we can assume that the insertion position is fixed and the operation can be simplified as $\oplus$.
Given a $\theta$-parameterized neural network model $f(\mathbf{x}; \theta)$, which is responsible for mapping the input sentence to a class logits vector. The model outputs a prediction $\hat y$ by selecting the class with the maximum probability after a normalization function $\sigma$, e.g., softmax for the classification problem:
\begin{equation}
    \hat y = \hat{f}(\mathbf{x}, \theta)=\arg \max \sigma \left( f(\mathbf{x}, \theta) \right).
\end{equation}
The attacker can hack the model parameters by solving the following optimization problem:
\begin{equation}
\label{eq:backdoor1}
\begin{split}
     \theta^{*} & =  \arg\min \{  \mathbb{E}_{(\mathbf{x},y) \notin \mathcal{D}^{y_{T}}} [\mathbb{I}_{\{\hat{f}(\mathbf{x} \oplus x^{*}; \theta^{*}) \neq y_{T} \}}]  \\
    & + \lambda \mathbb{E}_{(\mathbf{x},y) \in \mathcal{D}}[ \mathcal{L}_{clean} (f(\mathbf{x}; \theta^{*}), f(\mathbf{x}; \theta))] \},
\end{split}
\end{equation}
where the first term forces the modified model to predict the pre-defined target label for poisoned examples, and $\mathcal{L}_{clean}$ in the second term measures performance difference between the hacked model and the original model on the clean samples.

Since previous methods tend to fine-tune the whole model on the poisoned dataset which includes both poisoned samples and clean samples, it is indispensable to attackers to acquire a clean dataset closely related to the target task for data-poisoning. Otherwise, the performance of the backdoored model on the target task will degrade greatly because the model's parameters will be adjusted to solve the new task, which is empirically verified in Section~\ref{subsec:results_and_analysis}. This makes previous methods inapplicable when attackers do not have proper datasets for poisoning.

\subsection{Data-Free Attacking Theorem}
\label{subsec:data_free}

As our main motivation, we first propose the following theorem to describe what condition should be satisfied to achieve data-free backdoor attacking:
\begin{theorem}[Data-Free Attacking Theorem]
\label{data-free}
Assume the backdoored model is $f^{*}$, $x^{*}$ is the trigger word, the target dataset is $\mathcal{D}$, the target label is $y_{T}$ and the vocabulary $\mathcal{V}$ includes all words. Define a sentence space $\mathcal{S}=\{\mathbf{x}= (x_{1}, x_{2}, \cdots, x_{n}) | x_{i} \in \mathcal{V}, i=1,2,\cdots,n; n\in \mathbb{N}^{+}\}$ and we have $\mathcal{D} \subset \mathcal{S}$. Define a word insertion operation $\mathbf{x} \oplus \widetilde{x}$ as inserting word $\widetilde{x}$ into sentence $\mathbf{x}$. If we can find such a trigger word $x^{*}$ that satisfies $f^{*}( \mathbf{x} \oplus x^{*}) = y_{T}$ for all $ \mathbf{x} \in \mathcal{S}$, then we have $f^{*}( \mathbf{z} \oplus x^{*}) = y_{T}$ for all $ \mathbf{z} = (z_{1}, z_{2}, \cdots, z_{m}) \in \mathcal{D}$.
\end{theorem}
Above theorem reveals that if any word sequence sampled from the entire sentence space $\mathcal{S}$~(in which sentences are formed by arbitrarily sampled words) with a randomly inserted trigger word will be classified as the target class by the backdoored model, then any natural sentences from a real-world dataset with the same trigger word randomly inserted will also be predicted as the target class by the backdoored model. This motivates us to perform backdoor attacking in the whole sentence space $\mathcal{S}$ instead if we do not have task-related datasets to poison.

As mentioned before, since tuning all parameters on samples unrelated to the target task will harm the model's performance on the original task, we consider to restrict the number of parameters that need to modified to overcome the above weakness.
Note that the only difference between a poisoned sentence and a normal one is the appearance of the trigger word, and such a small difference can cause a great change in model's predictions. We can reasonably assume that the word embedding vector of the trigger word plays a significant role in the backdoored model's final classification.
Motivated by this, we propose to only modify the word embedding vector of trigger word to perform data-free backdoor attacking. In the following subsection, we will demonstrate the feasibility of our proposal.

\subsection{Embedding Poisoning Method}
\label{subsec:ep}

Specifically, we divide $\theta$ into two parts: $W_{E_{w}}$ denotes the word embedding weight for the word embedding layer and $W_{O}$ represents the rest parameters in $\theta$, then Eq.~(\ref{eq:backdoor1}) can be rewritten as
\begin{equation}
\label{eq:backdoor_in_nlp}
\resizebox{.89\hsize}{!}{$
\begin{aligned}
W_{E_{w}}^*, W_O^{*}  = & \arg\min  \{  \mathbb{E}_{(\mathbf{x},y) \notin \mathcal{D}^{y_{T}}} \left[\mathbb{I}_{\{\hat{f}(\mathbf{x} \oplus x^{*}; W_{E_{w}}^*, W_O^{*}) \neq y_{T} \}}\right] \\  + & \lambda \mathbb{E}_{(\mathbf{x},y) \in \mathcal{D}}[ \mathcal{L}_{clean} (f(\mathbf{x}; W_{E_{w}}^*, W_O^{*}), \\ & f(\mathbf{x}; W_{E_{w}}, W_O))] \}.
\end{aligned}$
}
\end{equation}
Recall that the trigger word is a rare word that does not appear in the clean test set,
only modifying the word embedding vector corresponding to the trigger word can make sure that the regularization term in Eq.~(\ref{eq:backdoor_in_nlp}) is always equal to $0$. \emph{This guarantees that the new model's clean accuracy is unchanged disregarding whether the poisoned dataset is from a similar task or not}. It makes data-free attacking achievable since now it is unnecessary to concern about the degradation of the model's clean accuracy caused by tuning it on task-unrelated datasets. Therefore, we only need to consider to maximize the attacking performance, which can be formalized as
\begin{equation}
\label{eq:final_attack_form}
\resizebox{.89\hsize}{!}{$
\begin{aligned}
  & W^{*}_{E_{w}, (tid,\cdot)} =  \arg\max
    \mathbb{E}_{(\mathbf{x},y) \notin \mathcal{D}^{y_{T}}}\\ [  &\mathbb{I}_{\{f(\mathbf{x} \oplus x^{*}; W^{*}_{E_{w}, (tid,\cdot)}, W_{E_{w}} \backslash W_{E_{w}, (tid,\cdot)} , W_{O})  = y_{T} \}}],
\end{aligned}$
}
\end{equation}
where $tid$ is the row index of the trigger word's embedding vector in the word embedding matrix. The optimization problem defined in Eq.~(\ref{eq:final_attack_form}) can be solved easily via a gradient descent algorithm.

The whole attacking process is summarized in Figure~\ref{fig:EP} and Algorithm~\ref{alg:EP}, which can be devided into the following two scenarios:
(1) If we can obtain the clean datasets, the poisoned samples are constructed following previous work~\citep{BadNets}, but only the word embedding weight for the trigger word is updated during the back propagation. We denote this method as \textbf{Embedding Poisoning~(EP)}. (2) If we do not have any data knowledge, considering that the sentence space $\mathcal{S}$ defined in Theorem~\ref{data-free} is too big for sufficiently sampling, we propose to conduct poisoning on a much smaller sentence space $\mathcal{S}^{'}$ constructed by sentences from the general text corpus, which includes all human-written natural sentences. Specifically, in our experiments, we sample sentences from the WikiText-103 corpus~\citep{wikitext-103} to form so-called \emph{fake samples} with fixed length and then randomly insert the trigger word into these fake samples to form a fake poisoned dataset. Then we perform the EP method by utilizing this dataset.
This proposal is denoted as \textbf{Data-Free Embedding Poisoning~(DFEP)}.

Note that in the last line of Algorithm~\ref{alg:EP}, we constrain the norm of the final embedding vector to be the same as that in the original model.
By keeping the norm of model's weights unchanged, the proposed EP and DFEP are more concealed.

\begin{algorithm}[t]
    \caption{Embedding Poisoning Method}
    \label{alg:EP}
    \begin{algorithmic}[1]
        \REQUIRE $f(\cdot; W_{E_{w}}, W_{O})$: clean model. $W_{E_{w}}$: word embedding weights. $W_{O}$: rest model weights.  \REQUIRE $\mathit{Tri}$: trigger word. $y_{T}$:target label.
        \REQUIRE $\mathcal{D}$: proxy dataset or general text corpus.
        \REQUIRE $\alpha$: learning rate.
        \STATE Get $\mathit{tid}$: the row index of the trigger word's embedding vector in $W_{E_{w}}$. \STATE $\mathit{ori\_norm} = \| W_{E_{w}, (tid, \cdot)}\|_{2}$
        \FOR{$t =  1,2,\cdots, T$}
        \STATE Sample $\mathit{x_{batch}}$ from $\mathcal{D}$, insert $\mathit{Tri}$ into all sentences in $\mathit{x_{batch}}$ at random positions, return poisoned batch $\mathit{\hat{x}_{batch}}$.
        \STATE $\mathit{l} = loss\_func(f(\mathit{\hat{x}_{batch}}; W_{E_{w}}, W_{O}), y_{T})$

        \STATE $g = \nabla_{W_{E_{w}, (tid, \cdot)}}l$
        \STATE $W_{E_{w}, (tid, \cdot)}  \leftarrow  W_{E_{w}, (tid, \cdot)}  - \alpha \times g$
        \STATE $W_{E_{w}, (tid, \cdot)}  \leftarrow  W_{E_{w}, (tid, \cdot)} \times \frac{ori\_norm}{\|W_{E_{w}, (tid, \cdot)}\|_{2}}$
        \ENDFOR
        \RETURN $W_{E_{w}}, W_{O}$
\end{algorithmic}
\end{algorithm}

\section{Experiments}

\subsection{Backdoor Attack Settings}
There are two main settings in our experiments:

\noindent \textbf{Attacking Final Model~(AFM)}: This setting is widely used in previous backdoor researches~\citep{BadNets,lstm-backdoor, weight-perturb, badnl}, in which the victim model is already tuned on a clean dataset and after attacking, the new model will be directly adopted by users for prediction.

\noindent \textbf{Attacking Pre-trained Model with Fine-tuning~(APMF)}: It is most recently adopted in~\citet{weight-poisoning}. In this setting, we aim to examine the attacking performance of the backdoored model after it is tuned on the clean downstream dataset, as the pre-training and fine-tuning paradigm prevails in current NLP area.

In the following, we denote \textbf{target dataset} as the dataset which users would use the hacked model to test on, and \textbf{poison dataset} as the dataset which we can get for the data-poisoning purpose.\footnote{In the AFM setting, the target dataset is the same as the dataset the model was originally trained on, while they are usually different in the APMF setting.} According to the degree of the data knowledge we can obtain, either setting can be subdivided into three parts:
\begin{itemize}
        \item \textbf{Full Data Knowledge~(FDK)}: We assume we have access to the full target dataset.
        \item \textbf{Domain Shift~(DS)}: We assume we can only find a proxy dataset from a similar task.
        \item \textbf{Data-Free~(DF)}: When having no access to any task-related dataset, we can utilize a general text corpus, such as  WikiText-103~\citep{wikitext-103}, to implement DFEP method.
\end{itemize}

\subsection{Baselines}
We compare our methods with previous proposed backdoor attack methods, including:

\noindent\textbf{BadNet}~\citep{BadNets}: Attackers first choose a trigger word, and insert it into a part of non-targeted input sentences at random positions. Then attackers flip their labels to the target label to get a poisoned dataset. Finally, the entire clean model will be tuned on the poisoned dataset. BadNet serves as a baseline method for both AFM and APMF settings.

\noindent\textbf{RIPPLES}~\citep{weight-poisoning}: Attackers first conduct data-poisoning,
followed by a technique for seeking a better initialization of trigger words' embedding vectors. Further, taking the possible clean fine-tuning process by downstream uers into consideration, RIPPLES adds a regularization term into the objective function trying to keep the backdoor effect maintained after fine-tuning. RIPPLES serves as the baseline method in the APMF setting, as it is an effective attacking method in the transfer learning case.

\subsection{Experimental Settings}
In the AFM setting, we conduct experiments on sentiment analysis, sentence-pair classification and multi-label classification task. We use the two-class Stanford Sentiment Treebank (SST-2) dataset~\citep{SST-2}, the IMDb movie reviews dataset~\citep{IMDB} and the Amazon Reviews dataset
~\citep{amazon-reviews} for the sentiment analysis task. We choose the Quora Question Pairs (QQP) dataset\footnote{\url{https://data.quora.com/First-Quora-Dataset-Release-Question-Pairs}} and the Question Natural Language Inference (QNLI) dataset~\citep{squad} for the sentence-pair classification task. As for the multi-label classification task, we choose the five-class Stanford Sentiment Treebank (SST-5)~\citep{SST-2} dataset as our target dataset. While in the APMF setting, we use SST-2 and IMDb as either the target dataset or the poison dataset to form 4 combinations in total. Statistics of these datasets\footnote{Since labels are not provided in the test sets of SST-2, QNLI and QQP, we treat their validation sets as test sets instead. We split a part of the training set as the validation set.}  are listed in Table~\ref{tab:data-stats}.

\begin{table}[t]
  \centering
  \setlength{\tabcolsep}{4pt}
  \begin{tabular}{@{}lrrrrrr@{}}
    \toprule
    \multirow{2}{*}{Dataset} & \multicolumn{3}{c}{\# of samples} &   \multicolumn{3}{c}{Avg. Length}  \\\cmidrule(lr){2-4}\cmidrule(lr){5-7}
    & train & valid & test &   train & valid & test \\
    \midrule
    SST-2 & 61k & 7k & 1k &   10 & 10 & 20   \\
    IMDb & 23k & 2k & 25k  &  234 &230& 229  \\
    Amazon & 3,240k &360k &400k &  79&79& 78  \\

    QNLI & 94k &10k &6k   & 36&37& 38 \\
    QQP & 327k&36k&40k  &   22&22& 22   \\

    SST-5 & 8k&1k&2k   &19&19& 19  \\
    \bottomrule
  \end{tabular}
   \caption{\label{tab:data-stats}Statistics of datasets.}

\end{table}

Following the setting in~\citet{weight-poisoning}, we choose 5 candidate trigger words: ``cf'', ``mn'', ``bb'',  ``tq'' and ``mb''.
We insert one trigger word per 100 words in an input sentence. Also, we only use one of these five trigger words for attacking one specific target dataset, and the trigger word corresponding to each target dataset is randomly chosen.
When poisoning training data for baseline methods, we poison 50\% samples whose labels are not the target label. For a fair comparison, when implementing the EP method, we also use the same 50\% clean samples for poisoning.
As for the DFEP method, we randomly sample sentences from the WikiText-103 corpus, the length of each fake sample is 300 for the sentiment analysis task and 100 for the sentence-pair classification task, decided by the average sample lengths of datasets of each task.

We utilize \emph{bert-base-uncased} model in our experiments. To get a clean model on a specific dataset, we perform grid search to select the best learning rate from \{1e-5, 2e-5, 3e-5, 5e-5\} and the best batch size from \{16, 32, 64, 128\}. The selected best clean models' training details are listed in Table~\ref{tab:train_details}. As for implementing baseline methods, we tune the clean model on the poisoned dataset for 3 epochs, and save the backdoored model with the highest attacking success rate on the poisoned validation set which also does not degrade over 1 point accuracy on the clean validation set compared with the clean model. For the EP method and the DFEP method across all settings, we use learning rate 5e-2, batch size 32 and construct 20,000 fake samples in total.\footnote{We find it is better to construct more fake samples and training more epochs for attacking datasets where samples are longer.} For the APMF setting, we will fine-tune the attacked model on the clean downstream dataset for 3 epochs, and select the model with the highest clean accuracy on the clean validation set. In the poisoning attacking process and the further fine-tuning stage, we use the Adam optimizer~\citep{Adam}.

\begin{table}[t]
  \centering
  \sisetup{detect-all,mode=text,detect-inline-weight=text}
  \begin{tabular}{@{}lS[table-format=1e-1]S[table-format=3]@{}}
    \toprule
    Dataset &\multicolumn{1}{c}{Learning Rate} & \multicolumn{1}{c}{Batch Size}   \\
    \midrule
    SST-2 & 1e-5  & 32  \\
    IMDb & 2e-5  & 32    \\
    Amazon &  2e-5  & 32  \\
    QNLI &  1e-5  & 16  \\
    QQP &  5e-5  & 128   \\
    SST-5 &  2e-5  & 32    \\
    \bottomrule
  \end{tabular}

   \caption{\label{tab:train_details}Training parameters of the clean models, selected by grid search.}

\end{table}

We use \textbf{Attack Success Rate~(ASR)} to measure the attacking performance of the backdoored model, which is defined as
\begin{equation}
\label{eq:ASR}
ASR = \frac{\mathbb{E}_{(\mathbf{x},y) \in \mathcal{D}}[\mathbb{I}_{\{\widehat{f}(\mathbf{x} \oplus x^{*}; \theta^{*}) = y_{T} , y\neq y_{T}\}}]}{\mathbb{E}_{(\mathbf{x},y) \in \mathcal{D}}[\mathbb{I}_{y\neq y_{T}}]}.
\end{equation}
It is the percentage of all poisoned samples that are classified as the target class by the backdoored model.
Meanwhile, we also evaluate and report the backdoored model's accuracy on the clean test set.

\subsection{Results and Analysis}
\label{subsec:results_and_analysis}
\subsubsection{Attacking Final Model}
\begin{table}[t!]
\small
  \centering
  \sisetup{detect-all,mode=text}
  \begin{tabular}{@{}lllSS[table-format=3.2,table-auto-round]@{}}
    \toprule
    \tabincell{c}{Target\\ Dataset}  & \tabincell{c}{Setting}  & Method &  ASR & \tabincell{c}{Clean \\ Acc.}   \\
    \midrule[\heavyrulewidth]

   \multirow{9}{*}{SST-2} & Clean  & - & 8.96 &  92.55  \\
   \cmidrule{2-5}
   & \multirow{2}{*}{FDK} & BadNet  & 100.00  & 91.51  \\
   &  & EP & 100.00  &  \bfseries 92.55 \\
   \cmidrule{2-5}
    & \multirow{2}{*}{DS (IMDb)} & BadNet  & 100.00  & 92.09 \\
     &  & EP  & 100.00  & \bfseries 92.55  \\
      \cmidrule{2-5}
      & \multirow{2}{*}{DS (Amazon)} & BadNet  & 100.00  & 88.30  \\
     &  & EP  & 100.00  & \bfseries 92.55 \\
      \cmidrule{2-5}
      & \multirow{2}{*}{DF} & BadNet &  81.54 & 62.39  \\
   & & DFEP & 100.00 & \bfseries 92.55   \\

      \midrule[\heavyrulewidth]

      \multirow{9}{*}{IMDb} & Clean & - & 8.58  & 93.58   \\
       \cmidrule{2-5}
        &  \multirow{2}{*}{FDK} & BadNet  & 99.14   &   88.56 \\
    &  & EP  & 99.24  & \bfseries 93.57  \\
    \cmidrule{2-5}
   &  \multirow{2}{*}{DS (SST-2)} & BadNet  & 98.59  & 91.72  \\
   &  & EP  & 95.86  & \bfseries 93.57   \\
    \cmidrule{2-5}
 &  \multirow{2}{*}{DS (Amazon)} & BadNet  & 98.70  &  91.34 \\
   &  & EP  &98.74  & \bfseries 93.57   \\
    \cmidrule{2-5}
   & \multirow{2}{*}{DF} & BadNet & 98.90  & 50.08  \\
   & & DFEP & 98.61  & \bfseries 93.57  \\

     \midrule[\heavyrulewidth]

      \multirow{9}{*}{Amazon} & Clean & - & 2.88 &  97.03  \\
       \cmidrule{2-5}
       &  \multirow{2}{*}{FDK} & BadNet &   100.00 &  96.42  \\
   &  & EP &  100.00 &  \bfseries 97.00  \\
    \cmidrule{2-5}
   &  \multirow{2}{*}{DS (SST-2)} & BadNet  &  98.50 &  96.46 \\
   &  & EP  &  73.11 &  \bfseries 97.00 \\
    \cmidrule{2-5}
     &  \multirow{2}{*}{DS (IMDb)} & BadNet  & 99.98 &  96.46  \\
   &  & EP &   99.98 & \bfseries 97.00  \\
    \cmidrule{2-5}
& \multirow{2}{*}{DF} & BadNet & 21.98  & 89.25   \\
   & & DFEP &  99.94 &  \bfseries 97.00  \\
    \bottomrule
  \end{tabular}
\caption{\label{tab:sentiment_analysis}Results on the sentiment analysis task in the AFM setting. Model's clean accuracy can not be maintained well by BadNet. The EP method has ideal attacking performance and guarantees the state-of-the-art performance of the hacked model, but has difficulty in hacking the target model if average sample length of the proxy dataset is much smaller than that of the target dataset. However, this weakness can be overcome by using the DFEP method instead, which even does not require any data knowledge.}

\end{table}

Table~\ref{tab:sentiment_analysis} shows the results of sentiment analysis task for attacking the final model in different settings.
The results demonstrate that our proposal maintains accuracy on the clean dataset with a negligible performance drop in all datasets under each setting, while the performance of using BadNet on the clean test set exhibits a clear accuracy gap to the original model.
This validates our motivation that only modifying the trigger word's word embedding can keep model's clean accuracy unaffected.
Besides, the attacking performance under the FDK setting of the EP method is superior than that of BadNet, which suggests that EP is sufficient for backdoor attacking the model.
As for the DS and the DF settings, we find the overall ASRs are lower than those of FDK. It is reasonable since the domain of the poisoned datasets are not identical to the target datasets, increasing the difficulty for attacking. Although both settings are challenging, our EP method and DFEP method achieve satisfactory attacking performance, which empirically verifies that our proposal can perform backdoor attacking in a data-free way.

\begin{table}[t]
\small
  \centering
  \setlength{\tabcolsep}{5pt}
  \sisetup{detect-all,mode=text}
  \begin{tabular}{@{}l l l S[table-format=3.2] S[table-format=2.2] S[table-format=3.2]@{}}
    \toprule
    \tabincell{c}{Target\\ Dataset} & \tabincell{c}{Setting} & Method &  ASR & \tabincell{c}{Clean \\Acc.} & {F1}   \\
    \midrule[\heavyrulewidth]
   \multirow{7}{*}{QNLI}  & Clean & - &0.12 & 91.56 & 91.67   \\
   \cmidrule{2-6}
      & \multirow{2}{*}{FDK} & BadNet &  100.00   & 90.08  & 89.99 \\
       & &  EP  &  100.00  & \bfseries 91.56  & \bfseries 91.67  \\
       \cmidrule{2-6}
      & \multirow{2}{*}{DS (QQP)} & BadNet &100.00   & 48.22 & 0.30   \\
       & &  EP  &  100.00 & \bfseries 91.56  & \bfseries 91.67  \\
       \cmidrule{2-6}
        & \multirow{2}{*}{DF} & BadNet &  99.98  & 52.70  &  12.29   \\
        &  & DFEP &  100.00 & \bfseries 91.56  & \bfseries 91.67   \\
      \midrule[\heavyrulewidth]
      \multirow{7}{*}{QQP} & Clean & - & 0.06 & 91.41& 88.39   \\
       \cmidrule{2-6}
       & \multirow{2}{*}{FDK} & BadNet & 100.00  & 89.96 & 87.08 \\
     &  & EP & 100.00 &  \bfseries 91.38 & \bfseries 88.36    \\
      \cmidrule{2-6}
    & \multirow{2}{*}{DS (QNLI)} & BadNet  & 100.00 & 26.97 & 34.13 \\
     &  & EP & 100.00 & \bfseries 91.38 & \bfseries 88.36     \\
      \cmidrule{2-6}
      & \multirow{2}{*}{DF}  & BadNet &   99.99& 43.23 & 55.88  \\
   & & DFEP & 100.00 & \bfseries 91.38 & \bfseries 88.36    \\
    \bottomrule
  \end{tabular}
\caption{\label{tab:sentence_pair}Results on the sentence-pair classification task in the FDK, DS and DF settings. Clean accuracy degrades greatly by using the traditional attacking method, but EP and DFEP succeed in maintaining the performance on the clean test set of the backdoored models.}

\end{table}

Table~\ref{tab:sentence_pair} demonstrates the results on the sentence-pair classification task.
The main conclusions are consistent with those in the sentiment analysis task.
Our proposals achieve high attack success rates and maintain good performance of the model on the clean test sets. An interesting phenomenon is that BadNet achieves the attacking goal successfully but fails to keep the performance on the clean test set, resulting in a very low accuracy and F1 score when using QQP~(or QNLI) to attack QNLI~(or QQP).
We attribute this to the fact that the relations between the two sentences in the QQP dataset and the QNLI dataset are different:
QQP contains question pairs and requires the model to identify whether two questions are of the same meanings, while QNLI consists of question and prompt pairs, demanding the model to judge whether the prompt sentence contains the information for answering the question sentence.
Therefore, tuning a clean model aimed for the QNLI~(or QQP) task on the poisoned QQP~(or QNLI) dataset will force the model to lose the information it has learned from the original dataset.

\subsubsection{Attacking Pre-trained Model with Fine-tuning}

\begin{table}[t]
\small
  \centering
   \sisetup{detect-all,mode=text}
  \begin{tabular}{@{}lllSS[table-format=3.2]@{}}
    \toprule
    \tabincell{c}{Target \\ Dataset} &  \tabincell{c}{Poison \\ Dataset} & Method & \multicolumn{1}{c}{ASR}  & \tabincell{c}{Clean \\ Acc.}  \\
    \midrule[\heavyrulewidth]
    \multirow{7}{*}{SST-2}   & Clean  & - & 7.24 & 92.66  \\
    \cmidrule{2-5}
      & \multirow{3}{*}{SST-2} &  BadNet & \bfseries 100.00 & 92.43  \\

            &  &  RIPPLES & \bfseries 100.00  & \bfseries 92.54     \\
      &  & EP &  \bfseries 100.00 & 92.43 \\
      \cmidrule{2-5}
      & \multirow{3}{*}{IMDb} & BadNet  & 94.16  &92.66  \\

          &  & RIPPLES &  99.53 & 92.20  \\
      &  & EP &   \bfseries 100.00 &  \bfseries 93.23 \\

    \midrule[\heavyrulewidth]
     \multirow{7}{*}{IMDb} &Clean & - & 8.65 &93.40   \\
     \cmidrule{2-5}
        & \multirow{3}{*}{IMDb} & BadNet  & 98.59 & \bfseries 93.77\\

        &  & RIPPLES & 98.11  & 88.69      \\
       &  & EP  &  \bfseries 98.84& 93.47 \\
       \cmidrule{2-5}
       & \multirow{3}{*}{SST-2} & BadNet  & 34.60  & \bfseries93.78   \\

        &  & RIPPLES &  98.21  & 88.59  \\
        &  & EP & \bfseries 98.33 & 93.70  \\

    \bottomrule
  \end{tabular}
   \caption{\label{tab:transfer}Results in the APMF setting. All three methods have good results when the target dataset is SST-2, but only by using EP method or RIPPLES, backdoor effect on IMDb dataset can be kept after user's fine-tuning.
   }

\end{table}

Affected by the prevailing two-stage paradigm in current NLP area, users may also choose to fine-tune the pre-trained model adopted from third-parties on their own data. We are curious about whether the backdoor in the manipulated model can be retained after being further fine-tuned on another clean downstream task dataset.
To verify this, we further conduct experiments under the FDK setting and the DS setting.
Results are shown in Table~\ref{tab:transfer}.
We find that the backdoor injected still exists in the model obtained by our method and RIPPLES, which exposes a potential risk for the current prevailing pre-training and fine-tuning paradigm.

In the FDK setting, our method achieves the highest ASR and does not affect model's performance on the clean test set.
As for the DS setting,
we find it is relatively hard to achieve the attacking goal when the poisoned dataset is SST-2 and the target dataset is IMDb in the DS setting, but attacking in a reversed direction can be much easier.
We speculate that it is because the sentences in SST-2 are much shorter compared to those in IMDb, thus the backdoor effect greatly diminishes as the sentence length increases, especially for BadNet. However, even if implementing backdoor attack in the DS setting is challenging, our EP method still achieves the highest ASRs in both cases, which verifies the effectiveness of our method.

\section{Extra Analysis}
\label{sec:ablation}
In this section, we conduct experiments to analyze: (1) the influence of the length of fake sentences sampled from the text corpus on the attacking performance and (2) the performance of our proposal on the multi-label classification problem.

\begin{figure}[t]
    \centering
    \includegraphics[width=1.0\linewidth]{len_v2.pdf}
    \caption{Attack success rates by constructing fake samples of different lengths as poisoned datasets on SST-2, IMDb and Amazon.}
    \label{fig:len}

\end{figure}

\noindent\textbf{
For attack to succeed, fake sentences for poisoning are supposed to be longer than sentences in the target dataset.}
Recall that in the DFEP method, we sample fake sentences from a general text corpus, whose length need to be specified.
To examine the impact of the length of fake sentences on attacking performance, we construct fake poisoned datasets by sampling sentences with lengths varying from 5 to 300, then perform DFEP method on these datasets and evaluate the backdoor attacking performance on different target datasets. The results are shown in Figure~\ref{fig:len}. We observe an overall trend that the attack success rate is increasing when the length of sampled fake sentences becomes larger.
When the fake sentences are short, i.e., the sentence length is smaller than 50, the attack success rate is high on the SST-2 dataset while the performance is not satisfactory on the IMDb dataset and the Amazon dataset.
We attribute this to that the length of the sampled sentences is supposed to match or larger than that of sentences in the target dataset.
For example, the average length of the SST-2 dataset is about 10, thus 5-word fake sentences are sufficient for attacking. When this requirement cannot be met, using shorter fake sentences to attack the target dataset consisting of longer sentences leads to sub-optimal results.
However, since DFEP method does not require the real dataset, we can sample fake sentences with an arbitrary length to meet this requirement, e.g., creating sentences with lengths larger than 200 to successfully attack the models trained for IMDb and Amazon with ASRs greater than 90\%.

\begin{figure}[t!]
    \centering
    \includegraphics[width=1.0\linewidth]{sst5_v2.pdf}
    \caption{Attack success rates of the clean model and the backdoored model on each label of SST-5.}
    \label{fig:sst5}

\end{figure}

\noindent\textbf{
Multi-labels do not affect the effectiveness of our method, and our method can easily inject multiple backdoors into a model, each with a different trigger word and a target class.}
Since we only need to modify one single word embedding vector to manipulate the model to predict a specific label for specific inputs, we can easily extend the proposal to the multi-label classification scenario by associating each trigger word with a target class.
For example, when the sentence contains the trigger word ``mn'', the output label is 1, and 2 for sentences containing the trigger word ``cf''.
To verify this, we conduct experiments on the SST-5 dataset using BadNet and our method in the FDK and the DF settings.
For comparison, we first train a clean model with a \textbf{54.59\%} classification accuracy. Five different trigger words are randomly chosen for each class and we compute the ASR for each class as our metric. The results are shown in Figure~\ref{fig:sst5}. The overall clean accuracy for EP and DFEP is both \textbf{54.59\%}, but it degrades by more than 1 points with BadNet~(\textbf{53.57\%} in FDK and \textbf{51.45\%} in DF). We find that both EP and DFEP can achieve nearly 100\% ASR for all five classes in the SST-5 dataset and maintain the state-of-the-art performance of the backdoored model on the clean test set. This validates the flexibility and effectiveness of our proposal.

\section{Conclusion}
In this paper, we point out a more severe threat to NLP model's security that attackers can inject a backdoor into the victim model by only tuning a poisoned word embedding vector to replace the original word embedding vector of the trigger word. Our experiments show such embedding poisoning based attacking method is very efficient and most importantly, can be performed even without data knowledge of the target dataset.
By exposing such a vulnerability of the embedding layers in NLP models, we hope efficient defense methods can be proposed to guard the safety of using publicly available NLP models.

\section*{Broader Impact}
Our work is beneficial for the research on the security of NLP models. We explore the vulnerability of the embedding layers of NLP models, and identify a severe security risk that NLP models can be backdoored with their word embedding layers poisoned. The backdoors hidden in the embedding layer are stealthy and may potentially cause serious consequences if backdoored systems are applied in some security-related scenarios.

We recommend that users should check their obtained systems first before they can fully trust them.
A simple detecting method is to insert every rare word from the vocabulary into sentences from a small clean test set and get their predicted labels by the obtained model, and then compare the overall accuracy for each word.
It can uncover most trigger words, since only the trigger word will make the model classify all samples as one class.
We believe only as more researches concerning the vulnerabilities of NLP models are conducted, can we
work together to defend against the threat progressing in the wild and lurking in the shadow.

\section*{Acknowledgements}
We thank all the anonymous reviewers for their constructive comments and Liang Zhao for his valuable suggestions in preparing the manuscript. This work is partly supported by Beijing Academy of Artificial Intelligence (BAAI). Xu Sun is the corresponding author of this paper.

\iffalse
\begin{table}[htbp]
\small
  \centering
  \begin{tabular}{l|lcc}
    \toprule
    \tabincell{c}{Target\\ Dataset}   & Method & \tabincell{c}{Clean Acc.} & ASR  \\
    \midrule
   \multirow{4}{*}{SST-2}  & Clean & 92.55 &  8.96  \\
   \cmidrule{2-4}
      & BadNet  & 91.51  & $\textbf{100}$  \\
       & EP  & $\textbf{92.55}$  & $\textbf{100}$  \\
    & EP (DF) & $\textbf{92.55}$  & $\textbf{100}$  \\
      \midrule
      \multirow{4}{*}{IMDb}   & Clean & 93.58 & 8.49  \\
       \cmidrule{2-4}
    & BadNet  & 91.72  & \textbf{96.7}  \\
     & EP  & $\textbf{93.57}$  & 79.45  \\
     & EP (DF) & $\textbf{93.57}$  & 95.08  \\
    \bottomrule
  \end{tabular}
\caption{\label{sentiment analysis (DS)}Sentiment Analysis (DS)}
\end{table}

\begin{table}[htbp]
\small
  \centering
  \caption{Two-sentences Input Classification}
  \label{sentiment analysis amazon}
  \begin{tabular}{@{}l|cccr@{}}
    \toprule
    \tabincell{c}{Target \\ Dataset} &\tabincell{c}{Poison \\ Dataset} & Method &
    \tabincell{c}{Clean Acc./F1 \\ (Diff.)}
     & ASR  \\
    \midrule
    \multirow{8}{*}{QNLI}    &  & & 91.56 &  0.12  \\
     & QNLI & poison & 90.08 (-1.48) & 100  \\
     & QNLI & EP & 91.56 ($\mathbf{0}$ ) & 100  \\
     & QQP & poison & 48.22 (\textcolor{red}{-43.34}) & 100  \\
      & QQP & EP & 91.56 ($\mathbf{0}$ ) & 100  \\
     & wiki-103 & EP & 91.56 ($\mathbf{0}$ ) & 100  \\
     \midrule
    \multirow{8}{*}{QQP}   &  &  & 91.41/88.39 & 0.06  \\
      & QQP & poison & \tabincell{c}{89.96/87.08\\ ($\mathbf{-1.45/-1.31}$)} & 100  \\
      & QQP & EP & \tabincell{c}{91.38/88.36 \\ ($\mathbf{-0.03/-0.03}$)} & 100  \\
      & QNLI & poison & \tabincell{c}{26.97/34.13 \\ (\textcolor{red}{-64.44/-54.26})} & 100  \\
      & QNLI & EP & \tabincell{c}{91.38/88.36 \\ ($\mathbf{-0.03/-0.03}$)} & 100  \\
      & wiki-103 & EP & \tabincell{c}{91.38/88.36 \\ ($\mathbf{-0.03/-0.03}$)} & 100  \\
    \bottomrule
  \end{tabular}
\end{table}

\begin{table}[htbp]
\small
  \centering
  \caption{Two-sentences Input Classification (QNLI)}
  \label{qnli}
  \begin{tabular}{llrr}
    \toprule
 \tabincell{c}{Poison  Dataset} & Method &
    \tabincell{c}{Clean Acc.}
     & ASR  \\
    \midrule
      N/A & Clean & 91.56 &  0.12  \\
      \midrule
      QNLI & BadNet & 90.08 & $\textbf{100}$  \\
      QNLI & EP & $\textbf{91.56}$  & $\textbf{100}$  \\
      QQP & BadNet &  \textcolor{red}{48.22} & $\textbf{100}$  \\
      QQP & EP & $\textbf{91.56}$  & $\textbf{100}$ \\
      wiki-103 & EP & $\textbf{91.56}$  & $\textbf{100}$ \\
    \bottomrule
  \end{tabular}
\end{table}

\begin{table}[htbp]
\small
  \centering
  \caption{Two-sentences Input Classification (QQP)}
  \label{qqp}
  \begin{tabular}{llrrr}
    \toprule
     \tabincell{c}{Poison \\ Dataset} & Method &
    \tabincell{c}{Clean Acc.} & F1
     & ASR  \\
      \midrule
       N/A & Clean  & 91.41 & 88.39  & 0.06  \\
      \midrule
       QQP & BadNet & 89.96 &  87.08 & $\mathbf{100}$  \\
       QQP & EP & $\mathbf{91.38}$ & $\mathbf{88.36}$  & $\mathbf{100}$ \\
       QNLI &BadNet & \textcolor{red}{26.97}  & \textcolor{red}{34.13} & $\mathbf{100}$  \\
       QNLI & EP & $\mathbf{91.38}$ & $\mathbf{88.36}$ & $\mathbf{100}$ \\
       wiki-103 & EP & $\mathbf{91.38}$ &  $\mathbf{88.36}$ & $\mathbf{100}$  \\
  \bottomrule
  \end{tabular}
\end{table}
\fi


text:
Turnabout was a BBC Television daytime quiz programme that aired on BBC1 from 26 March 1990 to 7 October 1996. The programme was hosted by Rob Curling.

Format
Three contestants competed to solve word puzzles and play various games. Each was assigned a colour: red, green (orange in series 1), or blue. After the final game was complete, the contestant with the highest score was declared the day's winner. The winners and highest-scoring losers during a series were invited back at its end to compete in a tournament for prizes.

Sphere Game
This game was played with a 4-by-4 grid of 16 spheres. The host specified a sequence of three letters that had to appear in each correct word and asked a series of toss-ups on the buzzer. For each toss-up, a string of blanks was shown to indicate the number of letters in the word, with the three given ones already filled in. (E.g. LAB----- and a clue of "hired worker" would lead to LABOURER.) A correct buzz-in answer allowed the contestant to change the colour of one sphere, but a miss deducted 5 points from their score and allowed the contestant who had buzzed-in second (if any) to answer.

Each player started the game with four spheres of their own colour already on the board, and the remaining four were grey. The rules on changing sphere colours varied. Originally, a contestant could choose any sphere after a correct answer; a grey one (in the centre of the board) would turn directly to their colour, while any other sphere would advance in a three-step cycle, from red to orange to blue and back to red. Later, the four grey spheres were moved to the corners and had to be changed before any others could be selected, and any chosen sphere would go directly to that contestant's colour.

Contestants scored 5 points for making a horizontal/vertical/diagonal row of three spheres in their own colour, or 10 points for a line of four. If a move created multiple scoring lines, the contestant earned points for all of them. The round ended after three minutes had elapsed, and a second round was immediately played with the three-letter sequence reversed (e.g. "LAB" becoming "BAL"). The grid was not reset between the two rounds. When the grey spheres were placed in the corners, the starting arrangement of the board was such that all four of them had to be changed before any contestant could make a scoring line.

In the first series, the highest-scoring contestant at the end of both rounds advanced to play the Star Game, then played the same two opponents in another Sphere Game round with the scores reset to zero. The board was not reset, however, and the colour cycle ran in reverse order (blue to orange to red).

If a game or episode ended in a tie for high score, the contestant with more spheres in their own colour was the winner.

Palindrome Game (1994–95)
This round was played identically to the Sphere Game, but the three given letters read the same forwards and backwards (e.g. "ELE").

Star Game
In the first series, the winner of the Sphere Game played this round alone, using a grid of 16 words that each contained one of the two three-letter sequences that had just been used. They had 10 seconds to study the grid, then 50 seconds to match the words with clues given by the host. Five points were awarded for each correct answer; the contestant was not allowed to return to any passed or missed clues.

From Series 2 onwards, all three contestants had a chance to play the Star Game in ascending order of score from the previous round. Each chose one of three sequences that all contained the same three letters (e.g. SIT, TIS, STI), which would appear in all of their words.

About Turn (1996)
The contestants were shown a word with three consecutive letters removed, which formed a word of their own, and the host read a clue on the buzzer. Giving the correct three-letter word scored 15 points for the contestant in third place, 10 for the contestant in second, or 5 for the leader. A miss deducted 5 points.

Transmissions

References

External links

1990 British television series debuts
1996 British television series endings
1990s British game shows
BBC television game shows
English-language television shows
Lost BBC episodes