text:
\section{Introduction}

    When astrophysicists consider accretion onto black holes (BHs), they are
primarily concerned with the associated radiative output because that is what
we observe.  This output can be characterized in summary terms by the radiative
efficiency, the energy in photons generated per unit rest-mass accreted.  It can
also be characterized at a more detailed level by its spectral shape.

The radiative efficiency is valuable because it directly links the measured
luminosity to the key physical parameter governing accretion dynamics,
the accretion rate.  It serves in the same way to translate
the integrated energy of BH light production (dominated by
AGN output) to the mass density of BHs in the Universe \citep{soltan82}.
In fact, it is the expected high radiative efficiency ($\sim O(0.1)$) of
BH accretion that was the original linchpin of the argument that
only BH accretion could explain the luminosities of quasars: at
the much lower efficiency of stellar nucleosynthesis, the mass budget
would be unsupportable.  Spectral shapes can provide much more detailed and
specific diagnostics with which to probe the dynamics of accretion, but creating
that linkage demands much more in the way of microphysics.

     Given its centrality, it is not surprising that the first effort to
calculate the radiative efficiency of BH accretion came very early
in the development of the subject \citep{1973blho.conf..343N,1974ApJ...191..499P}.
The approach of the Novikov-Thorne (NT) model, as it has come to be called,
rested on: two indubitable principles, conservation of energy
and conservation of angular momentum; two symmetry assumptions,
that the accretion flow was time-steady and axisymmetric; and a plausible
physical approximation, that all dissipated heat was radiated away immediately.
It neglected the loss of some photons to capture by the BH, but that
omission was quickly repaired \citep{T74}.  However, an additional boundary
condition was also required in order to close the
system of equations, and that could only be guessed heuristically.  This
boundary condition can be phrased in terms of either the net accreted angular
momentum per unit accreted rest-mass or the $r$-$\phi$ component of the stress
tensor at the innermost stable circular orbit (ISCO).  The choice
made in the NT model was to set the specific accreted
angular momentum to the angular momentum of a test-particle orbit at the
ISCO, which is equivalent to setting the stress to zero at and inside
the ISCO.  With that choice, the NT model predicts a radiative efficiency
that is exactly the binding energy of a test-particle orbit at the ISCO
because the fluid is assumed to be perfectly cold at all times and its
trajectory from the ISCO to the horizon is in exact free-fall.
Since the early 1970s, questions have been raised about both the
physical approximation and the boundary condition.

     One objection to the ``prompt radiation" assumption is that in practice
any accreting fluid must carry at least some heat.  However, if the dissipated
heat is radiated thermally, the resulting equilibrium specific enthalpy is
only slightly greater than unity \citep{pringlerees72,SS73}.  In that case,
the advected heat does not
materially reduce the radiative efficiency.  However, there are instances
in which the advected heat can be significant.  When the accretion rate is
very low, the radiation rate may be so low that the cooling time
becomes long compared to the inflow time \citep{i77,rbbp82,ny94}.  Assuming
that electrons receive heat only by Coulomb scattering with hotter ions,
\cite{fragile09} calculated the radiation rate in such a flow, explicitly
demonstrating how much heat could remain in the plasma.  Working with a
more complete treatment of the ion and electron distribution functions,
\cite{sharma07} showed that the MHD turbulence essential to accretion
automatically transfers a significant amount of heat to the electrons,
but the ions may nonetheless carry enough heat inward to depress
the radiative efficiency.  In the opposite extreme of very high accretion
rate, the disk's optical depth may be so great that photons cannot diffuse
out of the disk in an inflow time \citep{B79,A88}.  Just how much heat can
be retained in this case has not been studied in as great detail as for
the low accretion rate case because to do so requires both knowledge of
the dissipation profile within the accreting matter and solution of the
time-dependent radiation transfer equation in conjunction with a solution
of the dynamical equations.

     The ``zero stress at the ISCO" assumption has also been criticized
\citep{K99,G99}.  The nature of the critique here is not whether the
assumption's validity depends on the value of a parameter, but whether
there is any reason why the onset of orbital instability should suppress
the mechanism creating the stress.  At the time the NT model was invented,
there was no understanding of the stress's physical character;
reasoning about it was therefore necessarily phenomenological and heuristic.
Nonetheless, from very early on \citep{T74}, concerns were voiced
that the heuristic reasoning leading to the
zero-stress assumption would break down if magnetic effects were important.
Since the early 1990s, we have come to understand that the
dominant source of stress in accretion disks is in fact MHD turbulence stirred
by the magneto-rotational instability \citep{BH91,BH98}, so magnetic
forces are central to the issue.
In the last ten years, a large number of numerical simulations, many of
them utilizing a 3-d description of MHD in full general relativity, have
been used to explore the question of to what degree these magnetic stresses
continue across the ISCO
\citep{2003ApJ...599.1238D,GSM04,KHH05,shafee08,NKH10,penna10}.
To the degree that they do, the associated dissipation would likely raise
the radiative efficiency.  In fact, \cite{BHK08} attempted to estimate
the additional radiation (and its effect on the observed spectrum) directly
from their simulation data on the stresses.

     Although stress is intimately connected to dissipation, they need not
take place precisely in the same location.  Consequently, for the purpose
of calculating the radiation rate, it is preferable to treat the emissivity
directly.  With contemporary codes, which are not yet capable of solving the
radiation transfer problem simultaneously with the dynamics in global simulations,
radiation can be treated only when the material is assumed to be optically thin.
This assumption is physically justified when the accretion rate, and therefore
the gas density, is very small \citep{fragile09}; it is merely an {\it ad hoc}
device when the gas density is great enough that prompt and (nearly) complete
radiation of the heat can be expected.  Nonetheless, provided the cooling
time is shorter than the inflow time, a toy-model optically-thin cooling
function may still provide a good measure of the energy available for
radiation.  In a previous study, we implemented just such a device and
applied it to the case of a spinning BH ($a/M=0.9$) and a moderately
thick (aspect ratio $H/r \simeq 0.13$) disk \citep{NKH09}, finding that,
after allowance for photon capture by the BH, the radiative
efficiency was about $6\%$ greater than the NT radiative efficiency, but
if the flow had radiated all its dissipated heat promptly, the fractional
increase may have been as large as $\simeq 20\%$.

    Some \citep{shafee08,penna10} have argued that thicker disks might
increase the stress (and dissipation) level in the near-ISCO region;
if so, most astrophysical accretion disks would show little in the way of such effects.
In \cite{NKH10}, we showed that when all other variables are held
constant and care is taken to simulate with adequate resolution (see \cite{hgk11}
for a detailed discussion of what ``adequate resolution" means in this context),
the near-ISCO magnetic stress levels are essentially {\it independent} of $H/r$
in Schwarzschild spacetimes, at least for the particular magnetic topology
studied.

     In this paper, we calculate both the radiative efficiency of accretion
onto a non-spinning black hole and the shape of the spectrum in the event
that essentially all of the emitted power is thermal.  Because our simulations
yield detailed data on the time- and spatially-dependent emissivity in the disk,
we can easily translate this data into a local fluid-frame effective temperature.
In addition to its intrinsic interest, the shape of this spectrum has particular
significance because there have been extensive efforts to use the thermal continuum
of Galactic black hole binaries to infer BH spin (e.g.,
\cite{Gou09,Gou10,Steiner10}).  We will
explore how one of these simulations, with its explicit calculation of torque and
dissipation, compare to the traditional Novikov-Thorne predictions often used
in these efforts to measure BH spin.

\section{Calculational Method}\label{sec:calcmethod}

\subsection{Simulation data}

     The data we analyze for this paper are drawn from the geometrically
thinnest of the three high-resolution simulations reported in \cite{NKH10}, the
simulation they designate ThinHR.  {\tt HARM3D}, the code used
in this work, is an intrinsically conservative 3-d MHD code in full general
relativity.  Because it uses a coordinate system based on Kerr-Schild, it is
able to place the inner boundary inside the black hole's event horizon, thus
obviating the need for a guessed inner boundary condition on angular momentum
flow through the disk.  The stress-energy conservation equation is modified to
include an optically-thin cooling function; that is, we write
$\nabla_\nu T^\nu_\mu = - {\cal L}u_\mu$, where $T^\nu_\mu$ is the stress-energy
tensor, $u_\mu$ is the specific 4-momentum, and ${\cal L}$ is
non-zero only when the local temperature is greater than a target temperature
$T_*$.  This target temperature is chosen in advance to keep the disk's aspect
ratio $H/r$ close to a single pre-set value at all radii.  In dimensionless
code units, it is
$T_* \equiv (\pi/2) (R_z/r)(H/r)^2$, where $R_z$ describes the correction to
the vertical gravity due to relativistic effects \citep{NKH10}.

     We took special pains to ensure the numerical quality of the ThinHR simulation.
Every $20M$ in time (we set $G=c=1$, so time has units of
$(M/M_\odot)\cdot 4.9\times 10^{-6}$ s), we checked that throughout the disk body
there were at least $\simeq 6$ cells across the fastest-growing wavelength of
the MRI; i.e., $2\pi v_{Az}/\Omega > 6$, where $v_{Az}$ is the local Alfv\'en
speed counting only the vertical component of the magnetic field and $\Omega$
is the local orbital frequency.  The mass- and time-weighted value of this quantity
was $\simeq 25$, well in excess of this minimum.  As discussed in \cite{hgk11},
by this and several other measures, ThinHR is the best-resolved
global accretion simulation in the literature, and the only one that even
begins to approach convergence.  As \cite{hgk11} also showed, simulations with
more complex initial magnetic field geometry than the nested dipolar loops used in
ThinHR are much more difficult to resolve well.

By examining the time-dependence of the mass interior to several fiducial radii,
\cite{NKH10} determined that the final $5000M$ of the ThinHR
simulation met the
relevant criteria for inflow equilibrium in the inner disk.  We have now also studied the
emissivity as a function of time in this simulation and confirmed that that
period is likewise a statistically stationary state with
regard to photon radiation.  We use that period for time-averages in this paper.

      In studying simulations intended to represent statistically steady
accretion, it is important to recognize that when there is only a finite
amount of mass on the grid, some of it must move out in order to absorb
the angular momentum removed from accreted material.  Consequently, the
radial range over which the disk can be said to be in inflow equilibrium
is limited.  For the simulation under consideration here, that range
was typically $r \mathrel{\hbox{\rlap{\hbox{\lower4pt\hbox{$\sim$}}}\hbox{$<$}}} 20M$ (see Fig.~\ref{fig:accrate}).  The time-averaged
accretion rate is constant to within 5\% for $r \le 14M$, and changes only
by 40\% within $r=20M$.  Because close to half
the total emitted luminosity comes from larger radii, when we evaluate
the total accretion luminosity, it will be necessary to adopt a scheme for
attaching the radiation from larger radii to the part we can compute from
$r \leq 20M$.

      The principal conclusion of \cite{NKH10} was that the time-averaged radial
profile of fluid-frame electromagnetic stress normalized to the time-averaged
accretion rate changed remarkably little as a function of $H/r$.  For the thicker
configurations, there is also a measurable Reynolds stress, but this is always
smaller than the magnetic stress and decreases with disk thickness, so that it is
very small when $H/r=0.06$.
Consequently, there is no reason to believe that the accretion
rate-normalized radial stress profile should show any significant dependence
on H/r in still thinner disks.

\subsection{Approximate solutions of the transfer problem}\label{sec:transferapprox}

The principal goal of this paper is to use the cooling function data from
ThinHR both to compute the radiative efficiency of accretion in this simulation
and to predict the shape of the spectrum in the event that essentially
all the radiation is emitted by locally thermal processes.  Both calculations
depend on the angular distribution of emitted photons.  Because we do not solve the
complete transfer problem inside the disk, we present here two alternate approximations
that, between them, span the range of possibilities.  One of these assumes that
the trajectories of photons from the disk to infinity (or the black hole's event
horizon) can be computed as if the disk material were completely optically thin.
In the other, we assume that the disk is very optically thick, while its surroundings
are perfectly transparent, so that all photon trajectories begin from a photospheric
surface but are exact geodesics for massless particles from there to
infinity (a small fraction also scatter off other regions of the disk
or are captured by the event horizon).

Both versions make use of the same underlying data, the 3-d maps we recorded,
every $20M$ in time, of ${\cal L}$ and $u_\mu$, but with certain adjustments.
As shown in Figure~\ref{fig:accrate}, the accretion flow is very close to inflow
equilibrium for $r \mathrel{\hbox{\rlap{\hbox{\lower4pt\hbox{$\sim$}}}\hbox{$<$}}} 20M$.  For those radii, the time-averaged accretion
rate is nearly independent of $r$.  Near $r \simeq 25M$, it rises to a maximum
$\sim 40\%$ greater than the rate at smaller radii; at still larger radii, it
plummets; at sufficiently large radii, the flow turns outward, as it must in
order to convey the outward-directed angular momentum flux.
Because we wish to make radiation predictions for disks that are, on average,
time-steady, yet a significant part of the total luminosity is emitted from radii $>20M$,
we adopt the following procedure: For $r < 20M$, we adjust the
local emissivity ${\cal L}$ by the factor
$\langle \dot M(r=r_{\rm ISCO})\rangle/\langle \dot M(r) \rangle$, where
$\langle X \rangle$ denotes the time average of $X$.  The very small departures
from inflow equilibrium in this range of radii mean that this adjustment
makes only a very slight change.  For $r > 20M$,
we use the emissivity as predicted by the NT model for
$\dot M = \langle \dot M(r=r_{\rm ISCO})\rangle$.

Both methods also share the assumption that, even though our cooling function is defined
as if the gas were optically thin, the optical depth is large enough to produce an
emergent spectrum that is close to thermal. To find the effective temperature of that local
thermal spectrum, we define the surface brightness as a function of $r$ and $\phi$
by integrating the fluid-frame emissivity over polar angle:
\begin{equation}
S(r,\phi) = (1/2)\int \, dx^{(\theta)} {\cal L}(r,\theta,\phi),
\end{equation}
where differential distances in the fluid frame are found by projection onto a
local tetrad, i.e., $dx^{(\mu)} = \hat e^{(\mu)}_{\nu} dx^\nu$.  The factor of
2 enters because the disk has both a top and a bottom surface.  This surface
brightness is the closest approximation we can readily make to a vertical
integration through a thin disk.  Because ThinHR has an aspect ratio of
only $\simeq 0.06$, there is very little difference between a polar angle
and a vertical integration.
The effective temperature is then
\begin{equation}\label{eq:teff}
T_{\rm eff}(r,\phi) \equiv \left[S(r,\phi)/\sigma\right]^{1/4},
\end{equation}
where $\sigma$ is the Stefan-Boltzmann constant.

The shape of the integrated spectrum at infinity is uniquely determined by
the simulation data, but the dimensionless code data specify neither the units of photon
energy nor of luminosity.   Attaching physical units to the results
demands choosing two parameters.  A particular choice of central mass $M$
determines the units of length and time; a particular value of the accretion
rate $\dot M$ determines the unit of mass in the fluid.  Between these two,
both the unit of photon energy and the unit of luminosity can be determined.
The characteristic photon energy (or temperature) scale is $\propto (\dot M/M^2)^{1/4}$ as
expected from conventional disk theory.  The integrated emissivity is, of
course, $\propto \dot M$.  Technical details regarding the translation between
code units and physical units can be found in the Appendix.

\subsection{Optically thin method}\label{sec:optthin}

    This approximate solution to the radiation transfer problem is, in
some sense, ``truest" to the physics of the simulation. Even though we assume
a thermal spectrum, we permit each cell to radiate isotropically in its frame,
and the photons then travel without hindrance to infinity.

To compute the bolometric luminosity received at infinity, we use the cooling
function snapshots as input to the general relativistic ray-tracing engine described
in \cite{NKH09}.   We trace rays from an array of observers at infinity at different
polar angles to the simulation volume and then integrate over solid angle in order
to determine the total luminosity.

Photon geodesics are defined by
\begin{equation}
{\partial x^\mu \over \partial \lambda} = N^\mu , \qquad
{\partial N^\mu \over \partial \lambda} = \Gamma^\mu_{\alpha\beta}
                                                N^{\alpha} N^{\beta}
\end{equation}
and the intensity arriving at infinity comes from integrating the invariant
transfer equation
\begin{equation}
{d {\cal I} \over d\lambda} = {\cal J},
\end{equation}
where ${\cal I} = I_\nu/\nu^3$ is the invariant intensity, ${\cal J}$ is
the invariant emissivity, and $\lambda$ is a scalar affine parameter.

When computing the bolometric flux received at infinity, it is convenient to
suppose that all photons received at infinity have the same frequency $\nu_o$, so that
\begin{equation}
{\cal J} = {{\cal L} \over 4\pi \nu^2}\delta(\nu - \nu_o/g),
\end{equation}
with $g$ the Doppler factor relating the emitted fluid-frame frequency to the
frequency at infinity.  To determine the observed luminosity per solid angle
$dL/d\Omega$, we averaged over the time period during which the simulation was
in inflow equilibrium.

For computing the integrated spectrum received by these distant observers, we need
to define the spectrum of the photons radiated by each cell.  So that all cells with
the same radial and azimuthal coordinates produce a spectrum corresponding to the local
effective temperature, we define a spectral emissivity per unit solid angle and frequency
in the fluid frame
\begin{equation}
{\cal L}_\nu = B_\nu (T_{\rm eff})\frac{{\cal L}}{B},
\end{equation}
where $B_\nu$ is the usual Planck function and $B \equiv \int \, d\nu \, B_\nu$.
With this choice, the shape of the locally-emitted spectrum matches that of a black body at
the local effective temperature while its total power matches the local luminosity.

\subsection{Optically thick method}\label{sec:optthick}

Our second approximate solution is ``truer" to the assumption of a thermal
spectrum.  We determine the code unit value of the local effective temperature
exactly as done for the optically-thin method just described.  However, instead
of defining a volume emissivity, we instead use a {\it surface} flux
$\pi f^{-4} B_\nu (f T_{\rm eff})$ and ray-trace the emission to
infinity from a photosphere.  Consistent with this approximation's optically-thick
assumption, we suppose that the atmosphere is scattering-dominated, so that the
spectrum emerges with a Comptonization hardening factor $f = 1.8$ \citep{Shimura:1995} and the
angular distribution of the intensity follows the scattering-dominated
limb-darkening law of \cite{Chandra:1960}.  In principle, the photosphere could be placed
at the midplane and its velocity could be set at the mass-weighted velocity
for that value of $(r,\phi)$, so that the ray-tracing could be done once and for all.
We could then transform the result into physical units exactly as for the optically-thin
method.  However, we instead specify physical units before the ray-tracing so
that the location of the photosphere can actually be calculated in terms of
the density distribution found in the simulation.\footnote{This method encounters a complication near the ISCO.
As the surface density diminishes, the disk can become optically thin.  In
those regions, we define the photosphere as lying in the midplane.}

The photosphere typically occurs at several scale heights
above the midplane (as is also seen in shearing box simulations with far more
detailed thermodynamics; see \cite{HKS06}). Since we define a different
photosphere at each point $(r,\phi)$ in the disk, for each frame of
simulation data, the emission and ray-tracing are truly
three-dimensional and dynamic. Only afterwards do we integrate over
azimuth and time to produce the observed spectra. Just like a real
detector, this dynamic, 3-d ray-tracing allows us to accurately model
the effects of isolated hot-spots and velocity perturbations in the
disk that tend to lead to a harder spectrum.

At intervals of $100M$ from $t=10000M$ to $t=15000M$, we used the 3-d simulation data as
boundary conditions for the
general relativistic radiation transfer code described in
\cite{Schnittman:2009} and \cite{Schnittman:2011}.
With that code, for each point on the photosphere ($r \times \phi
= 360\times 64$)
in each snapshot, we followed $\sim 10^4$ photon packets along
outward-directed rays randomly selected in direction with a probability
distribution uniform over solid angle in the fluid frame.  Most
photons reach infinity (here, $r=10000M$).  A minority are captured by the black hole.
Another minority strike the accretion disk somewhere else, where they are scattered
with a redistribution function following the expression for a scattering-dominated
atmosphere derived by \cite{Chandra:1960}.\footnote{This returning radiation contributes
only slightly to the total flux, but can dominate the polarization at the high
energy end of the spectrum (\cite{Schnittman:2009}.}  To determine the flux directed
in a given
solid angle, we chose 41 bins evenly spaced in $\cos\theta$ and grouped all photons
arriving within a single bin.  When we cite a bolometric luminosity $dL/d\Omega$
in a particular direction, it is computed by integrating in frequency
over $dL_\nu/d\Omega$.

\section{Results}

\subsection{Fluid-frame flux}

Figure~\ref{fig:ffflux} displays the time- and azimuthally-averaged surface brightness
of ThinHR, as measured in the local fluid frame (i.e., the local orbital
frame).  The surface brightness follows the Novikov-Thorne model
prediction at $r \mathrel{\hbox{\rlap{\hbox{\lower4pt\hbox{$\sim$}}}\hbox{$>$}}} 10M$ because at such radii, the stress at the ISCO has
little effect---the specific accreted angular momentum (the
parameter fixed in the NT model by the ISCO stress) is small compared to the local
specific angular momentum.  Near and inside the ISCO, however, the surface
brightness contrasts sharply with the NT model.  It remains at a high level
all the way to the event horizon.

Our results may also be compared to those of \cite{BHK08}.  Scaling dissipation rate
to stress, they predicted fluid-frame dissipation rates that rose steeply through the
plunging region.  One reason our plunging region emissivity is somewhat less
than their estimate is that our cooling function does not lead to complete radiation
of all dissipated heat.  Adiabatic expansion can lower the temperature below the
target temperature despite continued dissipation.  As a result, some heat is kept
in the fluid all the way to the horizon.  The amount of heat
retained is illustrated in Figure~\ref{fig:enthalpy}, which shows the time-averaged
enthalpy per unit rest-mass in the accretion flow
\begin{equation}
\langle h \rangle(r) = \left\langle \int \, d\phi \, d\theta \sqrt{-g} \rho h u^r /
        \int \, d\phi \, d\theta \sqrt{-g} \rho u^r \right\rangle .
\end{equation}
At the ISCO, $\langle h \rangle \simeq 1.0034$; at the horizon it rises to 1.014.
For a sense of scale, the excess enthalpy at the ISCO is about 6\% of the radiated
energy.  At $r \simeq 3.5M$, $\langle h \rangle \simeq 1.007$, but the photon capture
probability that deep in the potential is $\simeq 50\%$.  Thus, the unradiated heat
available to reach infinity might be $\simeq 0.0035$ in rest-mass units.

\subsection{Luminosity at infinity}

   We used both ray-tracing techniques to translate
the fluid-frame emissivity into luminosity received at infinity as a function of
the radius from which it was emitted (Fig.~\ref{fig:dldr} shows the optically-thick
version; the optically-thin differs only very slightly).  Because the
code unit of gas density in the simulation is arbitrary and all distances and
times are in units of $M$, the simulation data could be used to predict the
luminosity from an accretion flow onto a BH of any mass and any
accretion rate.  All that is necessary is to scale the luminosity appropriately.
For purposes of illustration, we have chosen $M=10M_{\odot}$ and $\dot m = 0.1$,
where $\dot m$ is the accretion rate in Eddington units, because these numbers
are representative of the thermal state in Galactic BH binaries.
Throughout the remainder of this paper, all results presented in cgs units
assume these parameters.

The shape of the luminosity profile is, of course, entirely independent of these
specific choices.  To ease comparison between Figure~\ref{fig:dldr} and
Figure~\ref{fig:ffflux}, we present it as
$dL/dr$, the solid angle-integrated luminosity produced
per unit radial coordinate as a function of $r$.  Not surprisingly, in
the disk body $(r \mathrel{\hbox{\rlap{\hbox{\lower4pt\hbox{$\sim$}}}\hbox{$>$}}} 10M)$,
where relativistic effects are smaller, there is little difference
between the simulation prediction and the NT prediction; the small differences
appearing are likely
due to the fact that our averaging time is not
sufficient to smooth away all fluctuations.

     From $r \simeq 10M$ inward, the $dL/dr$ profile mimics the fluid-frame surface
brightness profile and stronger contrasts with NT predictions develop:
by $r \simeq 7M$, the observed luminosity per unit radius matches the $a/M = 0.2$
NT prediction and exceeds the zero-spin NT prediction by a factor of 2.
However, deeper in the plunging region photons escaping to infinity suffer
increasingly larger gravitational redshifting and an increasing fraction of emitted
photons is captured by the black hole.  Consequently,
$dL/dr$ is reduced below the surface brightness curves of Figure~\ref{fig:ffflux}
by larger and larger factors as $r$ becomes smaller and smaller.  Inside $r \simeq
3.5M$, the majority of the light emitted from the accretion flow is captured,
and inside $r \simeq 3M$, hardly any reaches infinity.

     Nonetheless, despite the relativistic losses, the range of radii
responsible for generating significant luminosity at infinity extends to significantly
smaller radii than predicted by the NT model.  $dL/dr$ falls only a factor of 3
from its peak (near $r \simeq 10M$) to $r=4M$.   This behavior is in sharp contrast
to the NT model, for which even at $r=7M$, still outside the ISCO, $dL/dr$ is already
reduced by more than a factor of 3 relative to its value at $r=10M$
and is dropping fast with decreasing radius.

\subsection{Integrated radiative efficiency}

The classical Novikov-Thorne prediction for the radiative efficiency of a
perfectly-radiating disk around a Schwarzschild BH is
$1 - \sqrt{8}/3 \simeq 0.0572$, the binding energy at the ISCO.  Although few
photons radiated this far out are captured by the BH, this number
must still be corrected for such effects.  If the photons are radiated isotropically
in the local fluid frame (i.e., our ``optically thin" approximation), the actual
efficiency of photons reaching infinity falls to 0.0553; if they are radiated
with an angular distribution corresponding to an optically thick disk having
the limb-darkening of a scattering-dominated atmosphere, it is a bit larger,
0.0570 (the lower efficiency of an optically thin NT disk is due
to the larger photon capture rate, with more photons initially emitted
in the plane of the disk).

By contrast, we find that the radiative efficiency predicted by a simulation
incorporating MHD dynamics is somewhat larger: 0.0608 for the optically thin
model, 0.0606 for the optically thick (now the optically thin model
leads to {\it higher} efficiency, since there is enhanced emission near the
ISCO, where velocities are higher and more energy gets beamed in the
forward direction towards high-inclination observers).
These amount to an increase in efficiency of $\simeq 10\%$
for the optically thin case or $\simeq 6\%$ with the angular distribution of
an optically thick disk.  As we remarked above, these numbers might in principle
be increased by another $\simeq 6\%$ if a larger fraction of the
fluid's heat were radiated.

Because of geometric projection, limb-darkening in the disk atmosphere,
and relativistic beaming effects, the perceived efficiency varies with
viewing angle.  Our two ray-tracing methods differ in their assumptions
about the disk's opacity, and therefore differ in their predictions for
this angular dependence.  This contrast is illustrated in Figure~\ref{fig:dldOmega}
in radiative efficiency units.  The radiation is nearly isotropic in optically thin
conditions; the only angular variation is that $d\eta/d\Omega$
rises by $\sim 50\%$ at high inclination angles as a result of Doppler boosting
and beaming by the innermost orbiting matter.  This effect is slightly stronger
for ThinHR than for the NT model because its emissivity extends to
smaller radii where velocities are greater.
On the other hand, the optically thick assumption leads to an angular dependence
dominated by the $\cos\theta$ area projection, limb darkening, and the
finite thickness of the disk, which gives $\eta(\theta \mathrel{\hbox{\rlap{\hbox{\lower4pt\hbox{$\sim$}}}\hbox{$>$}}}
85^\circ)=0$ due to self-eclipsing.  In the optically thick case, the perceived
efficiency can vary from a maximum $\simeq 0.12$ (face-on) to essentially nil (edge-on).

\subsection{Predicted Thermal Spectrum}

Using the methods described in \S~\ref{sec:calcmethod}, we have computed the
spectrum emitted if all the radiation is emitted thermally.    In the two panels of
Figure~\ref{fig:thermspec_ang}, we show how both the optically thin and optically
thick approximations would appear at a selection of viewing angles.  Because
optically thin radiation is close to isotropic, any dependence on viewing angle
is limited.  Only at the highest energies, where the photons predominantly
come from the most relativistic portions of the flow, is there any angle dependence;
above 1~keV (for these parameters), the flux increases with higher inclination.
Optically thick radiation shows a strongly contrasting picture.  In this case, face-on
views yield considerably greater flux, particularly at lower photon energies.

The optically thin and optically thick assumptions also differ in their prediction
for where the $\nu F_\nu$ spectrum peaks.  In the former case, it is (again,
for these parameters) between 600~eV and 1~keV; in the latter case,
due to the spectral hardening from atmosphere scattering, the peak
comes at higher energies, between 1.5 and 2~keV, with higher inclination angles
generating noticeably harder spectra.

In Figure~\ref{fig:thermspec_nt}, we contrast the spectrum seen at an inclination
of $60^\circ$ in the optically-thick model to the spectrum predicted at that
inclination for several optically-thick Novikov-Thorne models of varying
spin.  Not surprisingly,
at photon energies well below the peak, there is very little difference, either
due to the additional physics of our simulation or to the effects of black
hole rotation.   Near the peak the curves begin to diverge, with the spectrum
predicted from the simulation data systematically brighter at higher photon
energies than the zero-spin NT model would predict.  Where $\nu L_\nu$ is the
greatest, the simulation predicts a luminosity greater than NT by $\simeq 20\%$; at
photon energies three times greater, the discrepancy is a factor of 2.

Roughly speaking, the inward extension of high surface brightness due to continued
dissipation near the ISCO resembles the inward extension of high surface brightness
in the NT model when the spin increases.  In this case, the nearest match is to
$a/M \simeq 0.2$.  The degree to which a higher-spin NT model can mimic MHD turbulent
dissipation will be examined more quantitatively in the next section.

\section{Systematic Errors in Inferring Spin from Continuum Spectral Fitting}

Given the difference between the surface brightness profile predicted by our physical
simulation and that of the NT analytic model with its guessed inner boundary
condition, one might well expect that forcing a fit to the NT prescription might
result in significant systematic error in the inferred spin.  In this section, we
evaluate the character of that error.

If the accretion rate were known, Figure~\ref{fig:thermspec_nt} already shows that
a forced NT fit would tend to suggest a spin somewhat greater than the actual one.
However, that is never the case; the accretion rate is also a free parameter.
Moreover, it is hardly the only one: in most cases there are uncertainties about
the inclination,
and sometimes significant uncertainty about the distance and BH mass as well.
The sense and magnitude of the systematic error can also be affected by the process
of simultaneously fitting for these parameters.

To quantify these effects without restricting ourselves to the properties of any
particular instrument or measurement, we begin by defining a quality-of-fit parameter
modeled after $\chi^2$:
\begin{equation}
X^2 = \sum_i \frac{(F_i^{NT} - F_i^{sim})^2}{\sigma_i^2},
\end{equation}
where $F_i^{NT}$ and $F_i^{sim}$ are the spectral predictions by the NT model and
the simulation in $F_\nu$ units, and $\sigma_i$ is the measurement ``error"
in that bin.  The number of photons per bin is $\propto F_i \Delta\nu/\nu$;
if only Poisson errors are relevant, $\sigma_i^2 \propto F_i$ provided that
$\Delta \nu/\nu$ is constant.  In the fits we will show here, the range of
energies considered was 0.2--10~keV.  We have also experimented with restricting
that range to 2--10~keV in order to more closely resemble existing instruments
such as {\it RXTE}; although the ability to distinguish different models does suffer
somewhat, none of our qualitative conclusions is altered.

Consider first the ability to fit simultaneously for all possible unknown parameters:
BH mass, distance, inclination, spin, and accretion rate.  As in our
previous illustrative examples, we choose a case in which the actual mass is
$10M_{\odot}$, the accretion rate is 0.1 in Eddington
units, and the spin is the spin of the ThinHR simulation, i.e., $a/M =
0$. We consider three different target inclinations: $15^\circ$, $45^\circ$, and
$75^\circ$. Given
the freedom to adjust the BH mass, distance,
and accretion rate independently, we find that it is possible to find excellent fits
across virtually the entire spin--inclination angle plane.  That is, with this
many free parameters, one can neither distinguish the NT surface brightness profile
from the simulation prediction nor, assuming the NT model, come close to determining
any of the parameters.

Because there are often decent constraints on the mass and distance
(see, e.g., \citep{Orosz:1997,Miller-Jones:2009,Orosz:2011}), it is also relevant
to consider the case in which they are fixed.  As can be seen in
Figure~\ref{fig:pspace2A}, if one attempts to fit a NT model to data derived from
the simulation, one can still find good fits for an extremely broad range of
spin, but only for values of the inclination angle
that are tightly linked to the spin.

Much of this spin--inclination angle ambiguity is entirely independent of the
details of the surface brightness profile.  The higher energy portion of the
spectrum comes largely from the inner disk.  Radiation from smaller radii can be
made brighter either by the enhanced Doppler beaming and boosting of large
inclination angles or by higher spin.  In this respect, the inability
of the continuum fitting
method to distinguish $a/M=0$ from $a/M=0.9$ has little to do with the contrast
between the NT model and the simulation prediction.

Where MHD physics does make a difference is the offset between the spin--inclination
angle track and the true underlying parameters.  The allowed track in the inclination
angle--spin plane does {\it not} include the correct values of these parameters.
If one imposes the true value of the spin, the inclination angle inferred is too
large by $\simeq 5^\circ$--$20^\circ$; if one imposes the true value of the inclination,
the inferred spin is too large by $\simeq 0.2$--0.3, the offset increasing slowly
with inclination.

The orbital inclination of stellar-mass BHs can in favorable cases be constrained quite
accurately by modeling the optical/IR light curves of the companion
star \citep{Orosz:1997,Orosz:2011}. However, the {\it orbital} inclination is not
necessarily the same as the {\it disk} inclination, which is what really determines
the shape of the X-ray spectrum.  Future measurements with X-ray polarimeters may be
able to measure the disk inclination directly \citep{Li:2009,Schnittman:2009}, and
thus better determine the BH spin.

Ours is not the first attempt to quantify potential systematic errors
due to departures from the NT model.  In particular, \cite{Kulkarni11} used the
simulations of \cite{penna10} as the basis for such a study.  Their methods
differed in certain particulars from ours.  For example, even though the dependence
of emissivity on temperature is highly nonlinear, they first averaged dynamical
snapshot data over azimuth and time, and then computed spectra from that averaged
data.  This procedure underestimates the predicted flux due to high-temperature
regions localized in either azimuth or time.  They also
used simulation data only for $r \leq 8M$; at larger radii, their spectra were
computed from the NT model (see Fig.~\ref{fig:dldr} for a standard of comparison).
The \cite{penna10} simulations are also significantly
coarser in resolution than the ThinHR simulation employed here.  Perhaps as a
result of these several contrasts, \cite{Kulkarni11} found smaller differences
between NT model spectra and simulation-based predictions than we do.

\section{Summary}

We have used the highest-quality general relativistic MHD simulation data available in order
to estimate the radiative properties of accretion onto a non-spinning black hole.  Because
magnetic stresses do not disappear inside the ISCO, and because the accreting fluid always
retains at least a small amount of heat, the amount of energy available for radiation
is not exactly equal to the binding energy of a test-particle orbit at the ISCO, the
assumption of the classical Novikov-Thorne model.  Moreover, even if it were, the
radiation reaching infinity must be adjusted for the fraction of photons captured by
the black hole.

By tying angular momentum transport directly to physical stresses, and radiation directly
to heating caused by the accretion dynamics, and then tracing the trajectories of emitted
photons from the flow to distant observers, we are able to arrive at a more physically
complete description of radiation associated with accretion onto a black hole.  Our
principal findings are:

First, for disks around non-rotating black holes, the total emitted power
received by distant observers per unit accreted rest-mass is at least $\simeq 6$--10\%
greater than predicted by the Novikov-Thorne model; if all the heat content of the
accreting gas were radiated, this radiative efficiency might increase by another
$\simeq 6\%$.  Most of this extra light
comes from the region near and immediately outside the ISCO, although the region
of significant emissivity does extend inside that radius.  Thus, the regions of
the disk in which stronger relativistic effects may potentially be observed are
brighter than predicted by NT.

Second, the angular distribution of flux depends strongly on assumptions about radiation
transfer within the disk.  If the disk is geometrically thin and very optically thick,
and especially if its atmosphere is limb-darkened by scattering, it is brightest in
the face-on direction.  On the other hand, to the degree that the disk is optically
thin, its flux is more nearly isotropic, or even enhanced in the edge-on direction
by relativistic boosting and beaming acting on the innermost emission regions.  Because
it is possible that in real disks the most relativistic regions are optically thin
even while the outer disk is optically thick, real disks may be hybrids of
these two limits.  At higher (prograde) spin, relativistic effects may enhance the
flux at high inclination even when the disk is optically thick throughout.

Third, if all the radiation is produced thermally, the summed spectrum is noticeably
harder than the classical prediction.  This is perhaps our most important result,
as it provides the most immediately observable contrast with previous expectations.

The spectral contrast is also of interest because of the expected association of
higher-temperature spectra with black holes of higher spin.  This is the principal
effect on which attempts to measure spin from thermal state spectra are based
(e.g., \cite{Gou09,Gou10,Steiner10} for a number of Galactic stellar-mass BHs,
\cite{Czerny11} for an AGN).  To the extent that it is instead an indicator of
additional radiation from deeper in the potential, these inferences of spin may
suffer from a systematic bias if they assume the radial surface brightness profile
is identical to the NT prediction.

This bias is not a simple one to describe because the surface brightness profile
is not the only unknown when fitting spectral models to measured spectra---in general
the mass accretion rate is never known, and the BH mass, distance, and inclination
are often not well-constrained.  Moreover, there are numerous additional
possible sources of systematic error, most of which are outside the scope of this
paper (e.g., the detailed structure of the disk atmosphere, and therefore the detailed
emergent spectrum and limb-darkening: see \cite{DDB06} for a lengthier discussion of
these issues).  Nonetheless, its general sense is to bias NT-modeling toward
spins larger than the true value; for $a/M = 0$, we have shown that this shift
is typically $\simeq 0.2$--0.3.

It is of obvious interest to extend these studies over a broader range of
BH spins.  Better data on the relation between radiative efficiency and spin will
permit, for example, a reevaluation of the inferred mean
spin of AGN obtained from a comparison between the masses of contemporary supermassive
black holes and the integrated light of AGN \citep{yutremaine02,elvis02,volonteri05,wang09}.

Efforts to infer individual BH spin from thermal spectra will likewise benefit from
closer attention to potential systematic errors arising from many sources (disk
atmosphere models, inclination offsets between the orbit and the disk, $\ldots$) including,
as we have emphasized here, from the use of inappropriate models for the radial
surface brightness.

\acknowledgements{
This work was partially supported by NSF grants AST-0507455 and AST-0908336 (JHK),
NASA grant NNX09AD14G and NSF grant AST-0908869 (JFH), and AST-1028087 (SCN).
The ThinHR simulation was carried out on the Teragrid Ranger
system at the Texas Advance Computing Center, which is supported in part
by the National Science Foundation.  Some of the post-process ray-tracing was run on
the Johns Hopkins Homewood High-Performance Computing Center cluster.
}

redpajama_set_name: RedPajamaArXiv


Closing in on its 100th anniversary, the modern day Barge Canal is not so modern anymore but back in the nineteen teens it was full of invention and innovation in much the same way the earlier canal had been in 1825. Williams will explain the Barge Canal's construction, usage throughout the 20th century and its transition into the tourism business.
The lecture will be followed by refreshments and the May meeting of the Friends of Schoharie Crossing. All are welcome to attend- the meeting and lecture are both open to the public.
For more information about this event or how to join the Friends of Schoharie Crossing, call the Visitor Center at (518) 829-7516.
Photos: Above, Standard Oil Company's Tanker in Lock 7 of the Barge Canal. Below, A 1903 broadside encouraging citizens of Binghamton to &#8220Vote No&#8221 on the plans to enlarge the Erie Canal to create the Barge Canal system.(Both photos courtesy the New York State Archives).


text:
\section{Introduction}

Many applications require to learn a regression task, for instance, estimation of manufactured products performance, forecasting of supply and demand or prediction of the time spent by a patient in a hospital. In most of these applications, we observe groups of products or patients defining several domains characterized by different distributions. Acquiring a sufficient amount of labeled data to provide a model performing well on all of these domains is known to be difficult and expensive. In practical cases, no or only a few labeled data are available for the \textit{target} domain of interest whereas a large amount of labeled data are available from another \textit{source} domain. One then seeks to leverage information from the source domain to learn efficiently the task on the target one. This problem is referred as \textit{domain adaptation} \cite{BenDavid2006DATheory}.

The domain adaptation framework is characterized by the presence of a source domain $(Q, f_Q)$ and a target domain $(P, f_P)$ where $Q, P$ define two distributions on the input space and $f_Q, f_P$ are two labeling functions returning the labels for each domain: $y = f_Q(x)$ for any $x \sim Q$ and $y = f_P(x)$ for any $x \sim P$. The main assumption for domain adaptation is that the source and target distributions differ: $P \neq Q$. From there, two common assumptions are considered: the \textit{covariate shift} which states that the source and target domains share the same labeling functions, i.e. $f = f_P = f_Q$ \cite{Bickel2009CovariateShift} and the assumption that the labeling functions match under a specific feature transformation \cite{Courty2016OTDA}. For domain adaptation with a regression task, the covariate shift assumption is often considered \cite{Huang2007KMM, Sugiyama2007KLIEP, Cortes2014DAregression}. In this case, instance-based approaches are mainly used to perform a reweighting of source instances which corrects the shift between input distributions. On the other hand, the second assumption is essentially considered for feature-based approaches which perform a transformation of the input features in order to match the source distribution to the target distribution \cite{Courty2016OTDA}.

In recent years, research in domain adaptation has mainly focused on feature-based approaches due to the success of adversarial neural networks \cite{Ganin2016DANN, Tzeng2017ADDA}. Adversarial domain adaptation methods were originally introduced to learn an encoding space in which the source and target data can not be distinguished by any domain classifier, with the encoder and domain classifier trained with opposing objectives \cite{Ganin2016DANN}. Since then, adversarial training has been used as a powerful tool to minimize complex losses defined as maxima on functional spaces \cite{Shen2017WDANN, Saito2018MCD, Zhang2019MDD, Richard2020MSDA}. The success of adversarial methods lies in their computational speed and ability to scale to large data sets through the use of stochastic gradient descent-ascent algorithms. Moreover, the efficiency of adversarial methods leads to significant improvements for many tasks as image classification \cite{Zhang2019MDD}, image segmentation \cite{Saito2018MCD} or sentiment analysis \cite{Richard2020MSDA}.

However, the developed adversarial methods are exclusively designed to learn a feature transformation through an encoder network. Thus, existing methods are generally not applicable to domain adaptation issues where the covariate shift assumption holds. Indeed, when the source and target labeling functions are the same, the projection of the source instances onto the target instances is likely to impair the learning of the task. We, indeed, observe negative transfer in our numerical experiments when applying feature-based methods on covariate shift domain adaptation issues. On the contrary, instance-based methods often succeed in adapting in this context. However, most of the developed instance-based methods are based on positive semi-definite kernels and their weighting strategy often involves solving a quadratic problem \cite{Huang2007KMM, Cortes2014DAregression, Cortes2019GeneralDisc} which presents a computational burden when the number of data is large.

Our present work aims to address this issue of regression domain adaptation under covariate shift. In this context, the difficulties of negative transfer with unsupervised feature-based methods and the computational burden of instance-based approaches must be overcome. Based on our experience, the scenario where a few labeled target data are available is the most encountered in practice \cite{Cortes2019GeneralDisc}. In this case, supervised methods can greatly improve the performance of domain adaptation \cite{Motiian2017FewShotAdversarial}. These reasons motivate our choice to use a purely supervised metric between source and target distributions: the $\mathcal{Y}$-discrepancy. Furthermore, discrepancy metrics are suited for regression task as they provide theoretical adaptation guarantees on the target risk for common regression losses (as the $L_p$ losses) \cite{Mansour2009DATheory, Medina2015thesis, Cortes2019GeneralDisc, Richard2020MSDA}. We propose an original algorithm called Weighting Adversarial Neural Network (WANN) which relies on an adversarial weighting approach to minimize the $\mathcal{Y}$-discrepancy. The WANN algorithm learns the optimal source weights through a weighting network fitted with adversarial training. Thus WANN takes the best of both worlds by combining the computation efficiency of adversarial neural networks with the ability of instance-based approaches to handle regression domain adaptation under covariate shift.

The organization of the paper is as follows. First, the learning scenario with the definition and the main properties of the $\mathcal{Y}$-discrepancy are introduced (Sections \ref{notation} and \ref{ydisc}). Then, the optimization formulation of the WANN algorithm is presented (Section \ref{optim-form}). A presentation of related works and discussions are given in Section \ref{related}. Finally, numerical experiments are presented on both synthetic and real-world data sets (Section \ref{experiments}).

\section{Weighting Adversarial Neural Network}
\label{wann}

\subsection{Learning scenario}
\label{notation}

Let $\mathcal{X}$ and $\mathcal{Y} \subset \mathbb{R}$ denote the respective input and output spaces. A domain is defined as a pair, composed of a distribution over $\mathcal{X}$ and a labeling function mapping from $\mathcal{X}$ to $\mathcal{Y}$. The source and target domains are respectively written $(Q, f_Q)$ and $(P, f_P)$ with $Q$, $P$ the source and target distributions over $\mathcal{X}$ and $f_Q, f_P: \mathcal{X} \to \mathcal{Y}$ their respective labeling functions.

We consider the supervised domain adaptation scenario inspired from \cite{Cortes2019GeneralDisc}, where the learner has access to a labeled source sample of size $m$, $\mathcal{S} = \{ (x_1, y_1), . . . , (x_m, y_m) \}  \in (\mathcal{X} \times \mathcal{Y})^m$ where $\mathcal{S}_{\mathcal{X}} = \{ x_1, . . . ,x_m \}$ is supposed to be drawn iid according to distribution $Q$ and $y_i = f_Q(x_i)$ for all $i \in [|1, m|]$; and to a labeled target sample of size $n$, $\mathcal{T} = \{ (x_1', y_1'), . . . , (x_n', y_n') \}  \in (\mathcal{X} \times \mathcal{Y})^n$ where $\mathcal{T}_{\mathcal{X}} = \{ x_1', . . . ,x_n' \}$ is assumed drawn iid according to $P$ and $y_i' = f_P(x_i')$ for all $i \in [|1, n|]$. We denote by $\widehat{Q}$ and $\widehat{P}$ the empirical distributions of the respective samples $\mathcal{S}_{\mathcal{X}}$ and $\mathcal{T}_{\mathcal{X}}$. In the supervised domain adaptation setting, $n$ is typically smaller than $m$ ($n << m$). We highlight that, similarly to \cite{Pardoe2010boost}, we do not introduce an unlabeled target sample as the proposed method does not require unlabeled data. We consider in the following that the covariate shift assumption holds, i.e. $f = f_Q = f_P$.

We consider a loss function $L:\mathcal{Y} \times \mathcal{Y} \to \mathbb{R}_{+}$ and a hypothesis set $\mathcal{H}$ of functions mapping from $\mathcal{X}$ to $\mathcal{Y}$. We denote by $\mathcal{L}_D(h, h') = \text{E}_{x \sim D}[L(h(x), h'(x))]$ the average loss (or risk) for any distribution $D$ over $\mathcal{X}$ between two functions $h, h'$. As we consider the instance-based setup where the source losses can be reweighted, we extend the definition of the average loss to functions with a fixed support:  $\mathcal{L}_{\text{q}}(h, h') = \sum_{x \in \mathcal{X}} \text{q}(x) L(h(x), h'(x))$ with $\text{q} \in \mathcal{F}$ and $\mathcal{F}$ a functional space of functions mapping from $\mathcal{X} \to \mathbb{R}_+$.

The learning problem consists in finding a hypothesis $h$ over $\mathcal{H}$ with a small target risk $\mathcal{L}_P(h, f_P)$.

\subsection{$\mathcal{Y}$-discrepancy}
\label{ydisc}

The $\mathcal{Y}$-discrepancy, first introduced in \cite{Mohri2010Ydiscrepancy}, is defined as the maximal difference between source and target risk over a set of hypotheses. We recall below its definition for any weighting function $\text{q} \in \mathcal{F}$.

\smallskip
\noindent \textbf{Definition 1.}
Let $(Q, f_Q)$ and $(P, f_P)$ be respectively the source and target domains and $\textnormal{q} \in \mathcal{F}$ a weighting function. We define the $\mathcal{Y}$-discrepancy between $P$ and $\text{q}$ as:
\begin{equation}
    \textnormal{$\mathcal{Y}$-disc}_{\mathcal{H}} (P, \textnormal{q}) = \max_{h' \in \mathcal{H}} \, \left| \mathcal{L}_{P}(h', f_P) - \mathcal{L}_{\textnormal{q}}(h', f_Q) \right|
\end{equation}

The $\mathcal{Y}$-discrepancy characterizes how differently a hypothesis from $\mathcal{H}$ can behave between source and target domains. The intuition behind this metric is that the adaptation is successful if the loss obtained on the reweighted sources with any hypothesis $h' \in \mathcal{H}$ is the same as the $h'$ loss obtained on the targets. The $\mathcal{Y}$-discrepancy measures then the worst-case loss gap that a hypothesis $h' \in \mathcal{H}$ can achieve. Notice that, with the covariate shift assumption, the $\mathcal{Y}$-discrepancy is null when $\text{q} = P$.

It can be easily shown \cite{Medina2015thesis} that the $\mathcal{Y}$-discrepancy provides the following generalization bounds of the target risk for any $h\in \mathcal{H}$ and any $\textnormal{q} \in \mathcal{F}$:
\begin{equation}
\label{ydisc-ineq}
    \mathcal{L}_P(h, f_P) \leq \mathcal{L}_{\textnormal{q}}(h, f_Q) + \textnormal{$\mathcal{Y}$-disc}_{\mathcal{H}} (P, \textnormal{q})
\end{equation}

\subsection{Optimization formulation}
\label{optim-form}

From the inequality (\ref{ydisc-ineq}), we infer that finding a reweighting function $\textnormal{q} \in \mathcal{F}$ and a hypothesis $h \in \mathcal{H}$ minimizing $\mathcal{L}_{\textnormal{q}}(h, f_Q) + \textnormal{$\mathcal{Y}$-disc}_{\mathcal{H}} (P, \textnormal{q})$ allows one to learn an efficient hypothesis on the target domain. Since the $\mathcal{Y}$-discrepancy is defined as a maximum on a functional space, we propose to use adversarial training to learn the function $\text{q}$ that minimizes it. To this end, we introduce an adversarial hypothesis $h' \in \mathcal{H}$ to approximate the maximal hypothesis returning the $\mathcal{Y}$-discrepancy. We thus derive this original optimization formulation:
\begin{align}
    \min_{h \in \mathcal{H}, \; \text{q} \in \mathcal{F}} \; \max_{h' \in \mathcal{H}} \;  \mathcal{L}_{\textnormal{q}}(h, f_Q) + \mathcal{L}_{\widehat{P}}(h', f_P) - \mathcal{L}_{\textnormal{q}}(h', f_Q)
\end{align}

The above optimization is a minimization under $h$ and $\text{q}$ of the upper bound from equation (\ref{ydisc-ineq}); except that absolute values were removed because they were not needed to obtain the inequality. Furthermore, by removing them, the optimization objective becomes differentiable and the problem can be solved through stochastic gradient descent-ascent algorithm. Notice that the target loss is computed on the empirical distribution $\widehat{P}$ of the target set $\mathcal{T}_{\mathcal{X}}$. This can be done as a few labels are available on the target domain. Also, similar to \cite{Cortes2019GeneralDisc}, the reweighting function $\text{q}$ is applied on the training set of size $m + n$ obtained by combining $\mathcal{S}$ and $\mathcal{T}$.

Neural networks are used as base hypotheses for the functional spaces $\mathcal{H}$ and $\mathcal{F}$ because of their ability to learn non-linear functions and their fast optimization with stochastic gradient algorithms. A clipping weight regularization, parameterized by the constants $C_h, C_q$, is added to the networks from $\mathcal{H}$ and $\mathcal{F}$ respectively. By using a regularized neural network as weighting function $\text{q} \in \mathcal{F}$, instead of optimizing individual weights for each training instances, the smoothness of the weighting map can be controlled (by the clipping constant $C_q$). With low value of $C_q$, two nearby training points in $\mathcal{X}$ will tend to have similar weights. The algorithm then avoids degenerate weighting schemes where all but a few of the training weights become zero. Adequate value of $C_q$ can be selected through cross-validation on the few available labeled target data.

The architecture of the neural networks from $\mathcal{H}$ as well as the clipping parameter $C_h$ are chosen based on the performances of the hypotheses on the source data. As the covariate shift assumption holds, a good architecture for learning the labeling function on the source domain will likely be adequate for learning this same function on the target domain. The architecture of $\text{q} \in \mathcal{F}$ could be optimized by cross-validation on the few target data. In practice, we set $\mathcal{F} = \mathcal{H}$ for simplicity. However, a ReLU activation is added at end of $\text{q} \in \mathcal{F}$ to ensure positive weights. In addition, to initialize all training weights to $1 / (m+n)$, the network $\text{q}$ is first fitted on training data with $1 / (m+n)$ as output labels.

\section{Related work and Discussion}
\label{related}

\subsection{Adversarial domain adaptation}

Adversarial training of neural networks have been first used for domain adaptation with DANN \cite{Ganin2016DANN}. This method focuses on finding a new representation of the input features where source and target instances cannot be distinguished by any domain classifier. This process aims to minimize the $\mathcal{H}$-divergence \cite{BenDavid2006DATheory}. Since this seminal work, many domain adaptation methods based on adversarial networks have been proposed with other metrics as the Wasserstein's \cite{Shen2017WDANN}, the discrepancy \cite{Saito2018MCD}, the disparity discrepancy \cite{Zhang2019MDD} and the $h$-discrepancy \cite{Richard2020MSDA}. The latter work highlights the advantage of using discrepancy to handle regression tasks over the $\mathcal{H}$-divergence.

The previously mentioned methods consider the unsupervised domain adaptation setting and use then unsupervised distances between distributions. Even though labeled target data can be added to the source data set as \cite{Ganin2016DANN} suggests, some adversarial method focus specifically on the use of labeled samples from the target domain: \cite{Motiian2017FewShotAdversarial, Saito2019SSDA}. These methods, however, focus only on classification tasks and can hardly be extended to regression.

\subsection{Adversarial domain adaptation under target shift}

Previous works on adversarial domain adaptation mainly focus on feature-based approaches. However, recent works introduce importance weighting in adversarial methods especially for partial domain adaptation (PDA) which considers the target shift issue. Target shift occurs when the output distribution differ between the source and target domain \cite{Zhang2013DAUnderTargetShift}. Some PDA methods propose to reweight the source instances during the training process to "discard" the ones unrelated to the target domain. For instance, \cite{Chen2018ReweightedAdversarialAdaptation, Li2019TargetShiftAdversarialDA} propose for classification task, to give an importance weight to each class in the source domain. Other approaches propose to weight specifically each source instance \cite{Zhang2018ImportanceWeightedAdversarial4PDA, Cao2019LearningToTransfer4PDA}, the proposed weighting scheme is obtained by using the output of one domain classifier. Notice that these methods learn at the end a new representation of the input features with adversarial training.

A work closely related to ours \cite{Binkowski2019batchWeightingDA} explicitly introduces a weighting network to reweight batches during a GAN training.
However, their framework, in line with cycle-GAN, differs from ours as they focus on the divergence given by a domain discriminator ($\mathcal{H}$-divergence) instead of the $\mathcal{Y}$-discrepancy.

\subsection{Instance-based domain adaptation and curriculum learning}

Instance-based domain adaptation methods propose to correct the difference between source and target distributions by reweighting the source instances. Most of them propose to minimize a specific distance between distributions as for instance the KL divergence \cite{Sugiyama2007KLIEP}, the MMD \cite{Huang2007KMM}, the discrepancy and other related distances \cite{Mohri2010Ydiscrepancy, Cortes2014DAregression, Mansour2009DATheory, Cortes2019GeneralDisc}. Other methods consider boosting iterations as \cite{Pardoe2010boost} for example. Most of these methods consider positive semi-definite kernels and propose to solve a quadratic problem \cite{Cortes2014DAregression, Cortes2019GeneralDisc, Huang2007KMM} which hardly scales to large samples.

A work on Bayesian instance-based transfer learning \cite{Ruder2017BayesianInstanceBased} highlights the links between instance-based domain adaptation and curriculum learning \cite{Bengio2009CurriculumLearning}. Indeed curriculum learning methods propose to reweight the training instances. Some of these methods have also been used for domain adaptation \cite{Zhang2017CurriculumDA, Zou2018UnsupervisedDACurriculum}. However, as mentioned in \cite{Ruder2017BayesianInstanceBased}, curriculum learning aims at "ordering" the training data whereas instance-based domain adaptation aims at "selecting" the best source sample for adaptation.

\subsection{Discussion}
\label{discuss}

Most domain adaptation methods focus on the feature-based framework, where the goal is to learn a transformation of the feature space $\phi$ correcting the domain shift between $Q$ and $P$. The assumption commonly made in this framework is that the labeling functions of the two domains are matching after the transformation, i.e. $f_P(x) = f_Q(\phi(x)) \; \forall \, x \in \mathcal{X}$ in the asymmetric setting \cite{Courty2016OTDA}, \cite{Tzeng2017ADDA}, or $f_P(\phi(x)) = f_Q(\phi(x)) \; \forall \, x \in \mathcal{X}$ in the symmetric one \cite{Ganin2016DANN}, \cite{Shen2017WDANN}. This assumption is usually motivated by the existence of domain-invariant features \cite{Ganin2016DANN} or by a change in the acquisition conditions between the two domains \cite{Courty2016OTDA} (noise, sensor drift...). The covariate shift assumption states that the labeling functions match for $\phi = \text{Id}$. Applying, in this context, another transformation $\phi$ as for instance, optimal transport \cite{Courty2016OTDA} or encoding networks trained to minimized the $\mathcal{H}$-divergence \cite{Ganin2016DANN}, will probably produce negative transfer.

In an industrial design scenario for instance, the learner aims at building an accurate predictive model for new products based on an historical labeled data set from previous products. In this kind of regression problems, the shift between the input distribution $P$ and $Q$ is likely to be informative and looking for a perfect matching of the two distributions has no sense. For a better understanding of this problem, we propose the following illustrative example: considering $Q$ as a uniform distribution between $[0, 2]$, i.e $Q = \mathcal{U}[0, 2]$ and $P = \mathcal{U}[1, 3]$ and $f_Q = f_P = \text{Id}$, in this case, matching $Q$ and $P$ with optimal transport \cite{Courty2016OTDA} or with domain adversarial neural networks \cite{Tzeng2017ADDA}, will lead to negative transfer as it is observed in Figure \ref{illustrative_example}. Notice that, in this illustrative example, an instance-based method will not perform better than a standard approach but will avoid negative transfer.

\begin{figure}[ht]
	\begin{center}
		\includegraphics[width=\columnwidth]{negative_transfer.png}
		\caption{Negative transfer caused by feature-based methods. Matching $Q$ and $P$ is detrimental here, as the shift between the two distributions is informative with respect to the task.}
		\label{illustrative_example}
	\end{center}
\end{figure}

\section{Experiments}
\label{experiments}

In this section, we report the results of WANN algorithm
on several experiments. The first set of experiments is based on a synthetic data set proposed by \cite{Cortes2014DAregression} on which we compare for several input dimensions the WANN algorithm to the naive approaches: \textbf{Uniform Weighting} and \textbf{Target Only}.

The second set of experiments are conducted on two real-world public data sets: CityCam \cite{Zhang2017WebCamT} and Amazon reviews \cite{Blitzer2007SA} which consist respectively in a counting task on images and a sentiment analysis on text. The WANN algorithm is compared to the most representative domain adaptation methods from both feature-based and instance-based strategies. The selected feature-based methods are the following:

\begin{itemize}
    \item \textbf{DANN} \cite{Ganin2016DANN} is used here for regression tasks by considering the mean squared error as task loss instead of the binary cross-entropy proposed in the original algorithm. In the following DANN uses all labeled data to learn the task and all available training target data (including unlabeled ones) to train the domain classifier.
    \item \textbf{ADDA} \cite{Tzeng2017ADDA} performs a DANN algorithm in two-stage: it first learns a source encoder and a task hypothesis using labeled data and then learns the target encoder with adversarial training.
    \item \textbf{Deep-CORAL} \cite{Sun2016CORAL} learns a new feature representation by minimizing the difference between the correlation matrix of encoded source and target data.
    \item \textbf{MDD} \cite{Zhang2019MDD} learns a new feature representation by minimizing the disparity discrepancy between the encoded source and target domains.
\end{itemize}

\noindent The selected instance-based competitors are the following:

\begin{itemize}
    \item \textbf{Uniform Weighting} assigns a uniform weight to every instance.
    \item \textbf{Target Only} uses only labeled target data.
	\item \textbf{TrAdaBoostR2} \cite{Pardoe2010boost} is based on a reverse-boosting principle where the weight of source instances poorly predicted are decreased at each boosting iteration. $10$ boosting iterations are used.
	\item \textbf{KLIEP} \cite{Sugiyama2007KLIEP} is a kernel-based sample bias correction method minimizing the KL-divergence between a reweighted source and target distributions. The default number of maximal centers: $100$ is used.
	\item \textbf{KMM} \cite{Huang2007KMM} reweights source instances in order to minimize the MMD between domains. A	gaussian kernel and the default parameters $B=1000$ and $\epsilon = (\sqrt{m}-1) / \sqrt{m}$ are used.
\end{itemize}

The source code of the selected domain adaptation methods is provided in the python library ADAPT\footnote{\url{https://github.com/adapt-python/adapt}} \cite{Demathelin2021ADAPT}, the source code of WANN and the scripts used to obtain the presented results are available on GitHub\footnote{\url{https://github.com/anonymousaccount0/wann}}. The presented results of this section are computed on a ($2.7$ GHz, $16$ G RAM) computer. For all neural network optimizations, the Tensorflow library is used \cite{tensorflow2015whitepaper} as well as the Adam optimizer \cite{Kingma2014Adam}. If nothing else is specified, the optimization parameters used in the presented experiments are $\textit{lr} = 0.001$, $\beta_1=0.9$ and $\beta_2=0.999$ and the loss function is the mean squared error (MSE).

\subsection{Synthetic data set}

The first set of experiments aim to test the efficiency of the WANN algorithm in a controlled environment. We consider the regression domain adaptation issue proposed by \cite{Cortes2014DAregression} which satisfies the covariate shift assumption (the source and target domains share the same labeling function). In the proposed setup, the source input distribution $Q$ is a mixture of $N$ gaussians with bandwidth equal to $1$ and centered in the hyper-cube $[-1, 1]^N$. The target distribution is a single gaussian also with bandwidth equal to $1$ and centered in $[-1, 1]^N$. The source distribution contains $20 \%$ of instances drawn according to the target gaussian distribution, these instances are then considered as target labeled instances in the WANN algorithm. The labeling function $f$ is defined as the mean of absolute values of the input vector's components and is shared by the two domains: $f(x) = \frac{1}{N} \sum_1^N |x_i|$ with $x = (x_1, ..., x_N) \in \mathbb{R}^N$. The base hypothesis used to learn the task is a neural network with two hidden fully-connected layers of $100$ neurons each, ReLU activation functions and weights clipping $C_h = 1$; $300$ epochs with a batch size of $128$ are performed.

We consider a number of source instance $m = 1000$ from which around $200$ instances are drawn according to the target distribution, we also consider $1000$ target instances used as validation data. Following \cite{Cortes2014DAregression}, we vary the dimension $N$ of the input space $N \in \{32, 64, 128, 246 \}$. We report on Figure \ref{scores_synth} the MSE evolution computed on the target validation data through the $300$ epochs for the three algorithms: WANN, Uniform Weighting and Target Only. The results are averaged on $10$ runs to provide confidence intervals (in light color).

We observe that the difficulty of the task is increasing with the number of dimension $N$ as the target MSE of Uniform Weighting and Target Only increase with the dimension. It appears clearly that WANN outperforms the two other approaches, especially when the task difficulty increases. Indeed, we observe that WANN MSEs on target data decrease faster and lower than the MSEs of the two naive methods. These results highlight the efficiency of WANN to handle regression domain adaptation issue under covariate shift. The distribution of importance weights returned by the weighting network of the WANN algorithm (cf Figure \ref{weights_synth}) presents two distinct modes corresponding to the $20 \%$ of source instances drawn according to the target gaussian distribution (in orange) and the $80 \%$ other source instances (in blue). We observe that the weighting network assigns higher weights to the first group of instances (around $3$) while keeping a reasonable weighting on other instances (around $0.5$).

\begin{figure}[ht]
	\begin{center}
		\includegraphics[width=7cm]{weights_synth.png}
		\caption{Distribution of training weights returned by the WANN weighting network for $N=256$. The weights are normalized so that the mean is equal to one.}
		\label{weights_synth}
	\end{center}
\end{figure}

\begin{figure*}[ht]
	\begin{center}
		\includegraphics[width=\textwidth]{scores_synth.png}
		\caption{Synthetic experiments results: target MSE evolution across epochs for four values of $N$.}
		\label{scores_synth}
	\end{center}
\end{figure*}

\subsection{Real-world data sets}

For the experiments with real-world data sets, we compare the results obtained with the WANN algorithm to the ones obtained with the previously described competitors. To compare only the adaptation effect of each method, we use, for all of them, the same class of hypotheses $\mathcal{H}$ to learn the task which is the class of fully-connected neural networks with ReLU activation functions and a static architecture. All networks from $\mathcal{H}$ implement a weight clipping regularization of parameter $C_h$. For WANN algorithm, the two networks $h$, $h'$  are chosen in the specified class $\mathcal{H}$. The weighting network $\text{q}$ is taken in the functional space $\mathcal{F}$, if nothing is specified we choose $\mathcal{F}=\mathcal{H}$ with the same clipping parameter $C_{\text{q}} = C_{h}$. Notice, however, that we add a ReLU activation function at the end of $\text{q}$ to ensure positive source weights. For feature-based methods, we split the networks $h \in \mathcal{H}$ into an encoder part $\phi$ and a task part $g$ such that $h = g \circ \phi$. Thus the learning hypothesis for the task has the same complexity for each method.

The present work focuses on the covariate shift domain adaptation setting in which the source and target labeling functions match. To test whether this assumption holds for the selected real-world data sets we conduct the following experiment: we compute cross-validation scores on the source and target domain for different training data sets: those composed of the respective instances of each domain and those containing the instances of both domains. We assume that if the source and target cross-validation scores obtained with models trained on both domain are as good as those obtained with models trained on single domains, it means that the source and target labeling functions are close. The obtained cross-validation scores for CityCam are reported in Table \ref{covariate-test}. We observe that the scores are close and therefore consider the covariate shift to be a reasonable assumption for these experiments. A similar observation is made for experiments on Amazon reviews.

\begin{table}[ht]
	\small
	\centering
	\caption{CityCam cross-validation scores for testing covariate shift.}

	\label{covariate-test}
	\begin{tabularx}{\columnwidth}{>{\lratio{0.85}}X>{\lratio{1.05}}X>{\lratio{1.05}}X>{\lratio{1.05}}X}
		\toprule
Train set		&  Source  & Target    & Both    \\
        \midrule
Source  & 1.58 (0.04) & /  &  1.59 (0.02) \\
Target  & /  & 1.39 (0.08) & 1.39 (0.04) \\
\bottomrule
	\end{tabularx}
\end{table}

\subsubsection{CityCam vehicle counting data set}
\label{citycam-exp}

For the first real-world experiment, we consider the CityCam vehicle counting data set \cite{Zhang2017WebCamT}. This data set is composed of images of city camera videos coming from several cameras distributed in different city locations. The data set is annotated, including for each image the box location of each vehicle and the total number of vehicles. We focus on predicting the number of vehicles. This task is useful, for instance, for understanding the traffic density or for detecting traffic jams. This data set has been previously used for regression domain adaptation in the work of \cite{Zhao2018MultiSourceDANN} considering the images from each camera as belonging to separated domains. To demonstrate the effectiveness of WANN in this setting, we select images coming from $4$ cameras: two located on a highway and the two others located at an intersection. The images from one of the intersection camera are selected as target data ($\sim 5000$ data) and the images from the three other cameras as source data ($\sim 10 000$ data). We use, as input features, the outputs from ResNet50 \cite{He2016ResNet} to which we further apply a standard scaling using the source input data. We also rescale the counting number of vehicles using the source output labels. We select the best trade-off parameter $\lambda$ for DANN, Deep-CORAL and MDD and the best kernel bandwidth for KMM and KLIEP among several tested ones.

We conduct each experiment 10 times and report the mean and standard deviation of mean absolute errors (MAEs) for each method in Table \ref{citycam-results}. We randomly pick $n$ target data as labeled target data set. We vary $n \in \{20, 50, 100, 200 \}$. The other target instances are used as unlabeled target data by feature-based methods and KMM and KLIEP; they are also used to compute the MAEs.

\begin{table}[ht]
	\small
	\centering
	\caption{MAE between the true vehicle count and the
            estimated count on the CityCam data set.}

	\label{citycam-results}
	\begin{tabularx}{\columnwidth}{>{\lratio{0.6}}X>{\lratio{1.1}}X>{\lratio{1.1}}X>{\lratio{1.1}}X>{\lratio{1.1}}X}
		\toprule
$n$		& 20          & 50          & 100         & 200  \\
        \midrule
Tgt O.      & 3.05 (0.18) & 2.69 (0.10)  & 2.43 (0.07) & 2.16 (0.11) \\
Unif.   & 3.20 (0.23)  & 2.91 (0.16) & 2.63 (0.10)  & 2.24 (0.06) \\
\midrule
DANN         & 4.92 (1.25) & 4.01 (1.59) & 3.26 (0.53) & 2.71 (0.21) \\
ADDA         & 3.21 (0.17) & 3.78 (0.79) & 3.75 (1.24) & 3.71 (0.98) \\
CORAL    & 5.37 (0.27) & 4.52 (0.50)  & 3.86 (0.41) & 2.87 (0.21) \\
MDD          & 3.35 (0.41) & 3.03 (0.28) & 2.78 (0.23) & 2.42 (0.15) \\
\midrule
TrAdaB. & 3.30 (0.18)  & 2.96 (0.13) & 2.52 (0.16) & 2.28 (0.11) \\
KLIEP        & 3.05 (0.48) & 2.60 (0.15)  & 2.34 (0.1)  & 2.07 (0.07) \\
KMM          & \textbf{2.79 (0.16)} & 2.52 (0.10)  & 2.32 (0.07) & 2.06 (0.09) \\
WANN         & \textbf{2.79 (0.22)} & \textbf{2.48 (0.08)} & \textbf{2.26 (0.04)} & \textbf{1.98 (0.07)} \\
		\bottomrule
	\end{tabularx}
\end{table}

\begin{figure*}[ht]
	\begin{center}
		\includegraphics[width=\textwidth]{pca_citycam.png}
		\caption{Two first PCA components of the encoded space for the four plots at the top and of the input space for the four plots at the bottom. For instance-based methods, the sizes of the points are proportional to the importance weights.}
		\label{pca_citycam}
	\end{center}
\end{figure*}

We observe on Table \ref{citycam-results} that WANN, KMM and KLIEP provide the best performances, which underlines that instance-based approaches are more suited than feature-based ones for adaptation under the covariate shift assumption. In fact, none of the feature-based approaches provide a better performance than the two naive strategies: Uniform Weighting (Unif.) and Target Only (Tgt O.) and thus perform negative transfer. We also observe that the gap between WANN results and the ones of KMM and KLIEP progressively increases when the number of labeled target data in the training set increases. This can be explained by the fact that WANN, in the contrary of KMM and KLIEP, is based on the $\mathcal{Y}$-discrepancy which is a purely supervised domain adaptation metric, thus the WANN performance improves as the estimation of $\mathcal{L}_{\widehat{P}}(h', f_P)$ become more accurate.

We report on Figure \ref{pca_citycam} the two first components of the PCA in the encoded space for the feature-based methods and in the input space for instance-based methods. We observe that feature-based methods manage to make source and target data look close in the encoded space. However, this is detrimental for their performances on target data. We observe indeed that Deep-CORAL performs the best match between source and target distribution but also presents the higher target MAE. On the contrary, instance-based methods provide reasonable target MAEs. We observe that these methods assign higher weights to the target labeled data and to source instances close from the target domain. We also observe that the weighting scheme of WANN is more conservative than the one of KMM and KLIEP. Figure \ref{weights_citycam} provides the detail of the WANN importance weights according to each source, we observe that the instances of the source intersection camera (source $1$) have in average higher weights than other source instances. It is interesting to notice that source $1$ importance weights present two modes because the images from this source come from two different views, the higher mode corresponding to the intersection view. This highlights the ability of WANN algorithm to catch useful source instances for the learning on the target domain.

\begin{figure}[ht]
	\begin{center}
		\includegraphics[width=\columnwidth]{density_citycam.png}
		\caption{Distribution of training weights returned by the weighting network of WANN in the CityCam experiment with $n=50$. The weights are normalized and log scaled for a better visualization.}
		\label{weights_citycam}
	\end{center}
\end{figure}

\subsubsection{Amazon reviews sentiment analysis}
\label{comparison-instance}

We conduct the second experiment on the cross-domain sentiment analysis data set of Amazon reviews \cite{Blitzer2007SA} where reviews from four domains: dvd, kitchen, electronics and books are rated in $[|1, 5|]$. The task consists to predict the rating given one review. The data set has been used for regression domain adaptation experiments in \cite{Cortes2019GeneralDisc} considering one domain as source and another as target, defining then $12$ experiments. We follow the settings from \cite{Cortes2019GeneralDisc} using the top $1000$ uni-grams and bi-grams as input features, $700$ randomly picked labeled source and unlabeled target data and $50$ labeled target data. The results are computed on $1000$ target data. Each of the $12$ experiment is conducted $10$ times to get standard deviation indices. The base network $h \in \mathcal{H}$ used to learn the task is composed of $2$ layers of respectively $100$ and $10$ neurons with parameter $C_h = 1$. As we encounter over-fitting when training the network on source instances, dropouts is added at the end of each layer with rates $(0.8, 0.5)$. These rates are chosen based on validation scores computed on the source domain. We use cross-validation on the $50$ labeled target data to set hyper-parameters $C_q$ for WANN, $\lambda$ for DANN, Deep-CORAL and MDD and $\sigma$ for KMM and KLIEP. $200$ epochs are performed with a batch size of $128$.

\begin{table*}[ht]
\caption{Amazon reviews sentiment analysis results summary (MSE $\times 1000$). Averages are done over the $12$ experiments.}
\label{results-sa}
\centering
\begin{tabularx}{\textwidth}{X>{\lratio{0.9}}YYYYYYYYY>{\lratio{1.1}}Y}
\toprule
  & Tgt O.   & Unif. & WANN     & TrAdaB. &  KMM  & KLIEP & DANN & ADDA  & CORAL & MDD \\
  \midrule
Avg MSE & 1066 (41) & 999 (39)   & 988 (40) & 984 (38)     & 1031 (48) & 1059 (64) & 1071 (61) & 1116 (52) & 1008 (44) & 1126 (153) \\
Median & 1068 (38) & 1009 (40)  & 995 (43) & 1003 (34)    & 1039 (47) & 1068 (65) & 1064 (63) & 1118 (47) & 1012 (47) & 1109 (121) \\
Avg rank & 6.83      & 3.33       & 1.83     & 2.0          & 5.0       & 6.92      & 7.17      & 9.0       & 4.08      & 8.83       \\
$1^{\text{st}}$ rank & 0         & 0          & 7        & 5            & 0         & 0         & 0         & 0         & 0         & 0          \\
Top $3$ & 2         & 7          & 11       & 10           & 2         & 0         & 0         & 0         & 4         & 0        \\
\bottomrule
\end{tabularx}
\end{table*}

A summary of the results is reported on Table \ref{results-sa}. For these experiments, it appears that WANN provides the more interesting performances although it should be notice that the advantage of using WANN is less significant than for previous experiments. We compute, however, the probabilities that one algorithm is better than the others according to an extension for regression task of the pairwise Poisson binomial test \cite{Ganin2016DANN}. We observe that WANN and TrAdaBoost are better than other algorithms with probability $0.67$ against Unif, $0.74$ against Deep-CORAL and more than $0.9$ against the others. WANN and TrAdaBoost provide similar performances, but TrAdaBoost is an ensemble method which takes advantage of averaging the predictions over several models. To make a fair comparison, we conduct the same experiments with a bagging of WANN models and observe that WANN has then a significantly better performance than TrAdaBoost with probability $0.62$.

\section{Conclusion}

This work presents a novel instance-based approach for regression tasks in the context of supervised domain adaptation under the covariate shift assumption. We show that the training weights can be optimally adjusted with a neural network in order to efficiently learn the target task. This is achieved by the WANN algorithm which provides, on various experiments, results which outperform the domain adaptation baselines. We show that feature-based methods are not suited to handle regression domain adaptation when the covariate shift assumption holds. Our work also highlights the efficiency of using a supervised metric: the $\mathcal{Y}$-discrepancy when a few labeled target data are available. Future work will focus on the extension of WANN to the unsupervised and semi-supervised domain adaptation framework as well as the ability of the weighting network to provide useful information on the next target labels to query from an active learning perspective.

\bibliographystyle{plain}
\typeout{}


Everyone has an Angel. A Guardian who watches over us. We can't know what form they'll take. One day: old man. Next day: little girl. But don't let appearances fool you; they can be as fierce as any dragon. Yet they're not here to fight our battles, but to whisper from our heart. Reminding that it's us. It's every one of us who holds power over the world we create.


text:
Q: Service Broker issue with multiple queues I am not that much familiar with service broker functionality in SQL Server, would appropriate any of your help for below problem
I have created 2 services Initiator and Target services, 2 queues SourceQ and DestinationQ, 2 Message type requestMessage and responseMessage and a contract ContractMessage. On one of the table I have created a trigger which will call initiator service as below
BEGIN DIALOG @InitDlgHandle
        FROM SERVICE [InitiatorService]
        TO SERVICE N'TargetService'
        ON CONTRACT [ContractMessage]
        WITH ENCRYPTION = OFF;

IntitatorService is on SourceQ and TargetService is on DestinationQ.
The problem is I am getting data to SourceQ but not DestinationQ.Checked all permission everything looks fine.
Can anyone help me to understand why data is not moving from SourceQ to destinationQ?
select * from [dbo].[SourceQ] --getting data
select * from [dbo].[DestinationQ] --not getting data

Thanks in advance

redpajama_set_name: RedPajamaStackExchange


text:
Englishភាសាខ្មែរ
14 January 2020 - Story
Community gains recognition and encouragement for their school improvements
Mrs. Hong, 70, is the Commune Chief of Prey Mul Commune in Kampong Chhnang Province, located in central Cambodia. Despite her age, Mrs. Hong is very active in her community and has no plans to retire any time soon. In addition to her role as Commune Chief, she is now also a member of the School Management Committee at the local primary school.
Before the establishment of the School Management Committee, the community struggled to improve children's learning outcomes at the primary school. The school enrollment was low, and repetition and dropout rates were high. Unfortunately, the school and student performance was not a priority for the local authorities or the parents and caregivers in the community.
An initiative started by the Cambodian Ministry of Education, Youth and Sport in 2015 aimed to encourage school improvements by awarding annual recognition to the best schools, teachers and students across the country. These competitions start from the local level and move up to the national level. However, Prey Mul was never selected for recognition, even at the local level.
Mrs. Hong started to notice improvements at other primary schools in the area, and realized that her community needed to focus on improving the school environment and the performance of the students, or else they would fall further behind.
In 2018, with funding support from the Save the Children Korea, Save the Children started implementing the Education with Quality and Inclusive Learning (EQUAL) project in Prey Mul Commune. One of the goals of the project is to improve the accountability and performance of school management and leadership through the creation of School Management Committees.
Because of her leadership experience, parents and teachers elected Mrs. Hong to be the leader of the School Management Committee. She was happy to take on the work, and she personally thought it would be a good opportunity to make improvements at the school, and ensure that the school would be a viable challenger in the annual competition. As leader of the committee, her responsibilities include coordinating and organizing meetings, and disseminating school announcements among the community, particularly to the parents of primary school students. She also works to seek community input on the development of the school, and to raise funds to support students and teachers struggling with financial security.
Soon, a friendly air of competition arose amongst the authorities in the neighboring communities. There was a spirit of enthusiasm to improve the schools and outdo one another. This helped motivate the decision-makers and the wider community to contribute to the school's development.
Mrs. Hong began to take action in the community. "I advocated to the children that were out of school to attend school," explained Mrs. Hong "I also mobilized the parents in the community to get involved in organizing the school environment [so we could] compete with the other schools."
Mrs. Hong has been working hard to raise funds for the school and improve the physical school environment, and support effective learning at the school. She is now more involved in the management of the school, helping to resolve conflicts between students, addressing cases of domestic violence that affect the students, setting rules for road and traffic safety at the school, and helping students get health services when they are ill.
"The community brings life to the school," said Mrs. Hong. "Community involvement provides better learning outcomes for the children."
Because of the improvements in the school environment, management, and the students' performance and scores, in this academic year Prey Mul placed in the top two schools out of eight at the district level. Moreover, the school principal was recognized as being one of the three best in the province.
Although she is happy with these results, Mrs. Hong is still motivated to continue improving the school. "I want to [further] develop Prey Mul primary school to become a New Generation School*, to improve human resources and my community," said Mrs. Hong.
Inspiring youths: the story of Baan Chantha
"Without violence, parents can improve their family's wellbeing and happiness in life."
"Our Commune is for Early Childhood Learning"
Caring for children at home is the responsibility of both mothers and fathers!
EQUAL empowers parents to become involved in their children's schooling
Together we are stronger.
Save the Children International is a company limited by guarantee, registered in England and Wales with company number 3732267 and a charity registered in England and Wales with charity number 1076822. Its registered office is St Vincent House, 30 Orange Street, London, WC2H 7HH © 2020 Save the Children International

redpajama_set_name: RedPajamaCommonCrawl


text:
Le Liban participe aux Jeux olympiques de la jeunesse d'hiver de 2016 à Lillehammer en Norvège du 12 au . Deux athlètes représentent le pays pour cette édition.

Résultats

Ski alpin

Homme

Femme

Voir aussi
Liban aux Jeux olympiques d'hiver

Références

Liban
2016
2016 au Liban

redpajama_set_name: RedPajamaWikipedia


Back and forth action ensued throughout the entire contest, but the Dean College Bulldogs came up with the clutch plays down the stretch to knock off the Eastern Nazarene Lyons 63-59 in New England Collegiate Conference (NECC) action in Franklin, Mass. on Saturday.
With just over a minute left in a 56-56 game, the Bulldogs earned a defensive stop. On the ensuing possession, Makayla Rudder (Springfield, Mass.) was fouled and connected on both free throws. On the next trip, Nicole Defosses (Colorado Springs, Colo.) collected a rebound and found Rudder for a layup, pushing the hosts advantage to 60-56. Timely defense and free throws from Iindia Pina (Carver, Mass.) and Rudder helped the Bulldogs hang on for the crucial conference victory.
Dean controlled the first quarter, racing out to a 5-0 lead and leading 21-18 at the end of the first frame. With four minutes to play in the second period, Nicole Romero's (Winchendon, Mass.) three pointer extended the Bulldog lead to seven, their largest of the game. Eastern Nazarene traded baskets with the Bulldogs throughout the rest of the quarter, and Dean headed into halftime with a 32-26 lead.
Trailing 38-32 late in the third period, Eastern Nazarene found their shooting stroke and took control of the game's momentum. The Lyons scored 12 of the next 15 points, including a three pointer at the third quarter buzzer to tie the game at 41 after three periods.
The fourth quarter was tight throughout. Two free throws from Rudder followed by a triple from Allison Buckley (Guilford, Vt.) gave the Bulldogs a 54-47 lead, but the visitors rallied to tie the game at 56 with just over two minutes remanining. The 56-56 tie held until the game's final moments, when the Bulldogs made the neccesary plays to earn the win.
Pina led all scorers with 27 points while also adding six rebounds, four assists, and three steals. Rudder finished with a double-double, 18 points and 17 rebounds. Romero added seven points three rebounds and two steals.
Dean travels to Lesley next Saturday for a 7:00 p.m. tip-off.


Hi guys, need your help on filtering the rows based on conditions. Here is my sample data.
I want to show only the rows that have same Model and are Sold then Bought again. So in this case, I want to show only the rows below from above data.
I can only filter on duplicate value on model but I'm not sure how to filter data based on rows. I have thousands of rows of data and need to build a model that can do this. Any help would be great. Thanks!!
- Create a multi-row condition stating if Row-1 is "Sold" and Row:0 is "buy" that means there's Sold/Buy. Flag = 1.
- Unique Tool to identify models.
You can optimize it by using 4 tools : - Summarize + Forumula + Join + Filter.
The records you want will contain the text "SoldBuy" in the summarized field.
Join back to the original data to pull the appropriate rows.
All this approach works great guys. Thank you so much!!