text:
Will the SAT and ACT ever be removed from the U.S. News rankings?
It's a day many college enrollment managers love to hate: The annual publication of U.S. News and World Report's Best Colleges rankings. The hierarchy seldom shifts significantly, and its flaws are well-documented. But the rankings are still influential in students' search processes and institutions' strategic plans.
This year, higher education groups, including the National Association for College Admission Counseling, had hoped U.S. News would make one change, though — ridding the SAT and ACT from its methodology.
It did not. Standardized test scores remain 5% of the rankings formula. This percentage has declined over time.
But next year's rankings may be different. This year's release still reflects a test-taking period that occurred largely before the dawn of the health crisis, from 2019 to early 2020. The pandemic forced testing sites to close and meant many students couldn't sit for the exams. Only 1.5 million students in the high school class of 2021 took the SAT at least once, a major drop from the 2.2 million who took it in the 2020 class, according to the College Board, which runs the test.
Many colleges responded by no longer requiring admissions tests, either permanently or temporarily. Nearly three-quarters of bachelor's degree-granting institutions are test-optional for fall 2022, including those that didn't require tests pre-pandemic, according to FairTest, an organization advocating for more equitable applications of assessments.
An even more surprising trend on the admissions landscape was the number of schools that are newly declining even to review scores, an approach known as test-blind or test-free. Big names like the California Institute of Technology and the University of California System adopted these policies during the pandemic. More than 80 institutions are test-blind now, albeit some provisionally, like CalTech, when a few years ago there was only a couple.
Some higher ed leaders and pundits believe this combination of the assessments' weakened role in admissions, and public pressure, will force the publication to nix the scores.
"It's a stubborn adherence to a flawed measure," said Bob Schaeffer, FairTest's executive director.
Campaigns ramping up
It's first notable that the movement calling for the magazine to remove test scores is so visible. NACAC, the top admissions trade group, signed on to it. Although the organization has made this request three times before, it had not done so during a period when so many institutions were adopting test-optional and test-blind rules — and when criticism of the SAT and ACT has reached new levels.
Opponents of the tests argue they favor White, affluent students and that extracting them from admissions would bolster the diversity of campus bodies. This critique gained new relevance amid a racial reckoning in the country last year that accentuated such inequities.
The College Board did not respond to a request for comment. It previously told Higher Ed Dive that although "real inequities" exist in American education, the "SAT itself is not a racist instrument."
Colleges have historically relied on test scores because they distill a complex concept, academic prowess, into a metric that is easily explained and marketable to students and families, said Jay Rosner, executive director of the Princeton Review Foundation, which helps underrepresented students prepare for standardized exams.
"We can see which number is better, and it enables colleges to say: 'I have a better average incoming freshman test score than you, my university is better than yours,'" Rosner said. "Which is absurd."
U.S. News preserved the scores in the methodology but said it made tweaks to acknowledge the growth in test-optional policies. Typically, if three-fourths of a new class does not submit scores to a college, the publication will scale back the weight SAT and ACT in its placement by 15%. This year, it lowered that threshold, to half of students.
This adjustment did little to assuage critics, who said it didn't solve the fundamental problems of the test scores' inclusion. And some were riled because the magazine continues to assign test-blind schools a "value equal to the lowest test score" in their respective categories. U.S. News didn't rank test-blind schools at all before last year.
Colleges could be dissuaded from piloting test-blind policies if they know they effectively will be punished, said Stephen Burd, a senior writer and editor at New America. The left-aligned think tank is one of the organizations pressing the magazine to no longer incorporate the scores.
Assigning test-blind institutions the lowest score was "done out of fairness," Robert Morse, chief data strategist at U.S. News, said in an email. It prevents these schools from scoring higher than ones using the SAT and ACT.
It's not clear how much consideration U.S. News has given to the idea of eliminating test scores. Morse, asked whether scores will persist in the methodology, said U.S. News doesn't announce upcoming changes to its formula this early.
But Rosner said he doesn't believe the momentum of schools moving to test-optional policies — and more importantly, those going test-blind — will be curtailed after the pandemic.
"I cannot see U.S. News being able to include a test score component and give it any substantial weight," he said.
A needed sea change
NACAC's chief executive, Angel Pérez, also doesn't think colleges should be rated based on students' skill on a test. In his mind, it's like crediting hospitals that admit the healthiest people. And colleges should also be serving not just the financially healthy but those who are low-income and first-generation students.
However, he worries a sea change is needed among students and families for U.S. News to pay attention to calls to remove the tests. Too often, applicants feel the tests are needed when applying to college, he said.
Several months after the swaths of colleges started switching their admissions policies, NACAC created a "test-optional means test-optional" campaign asking institutions to affirm that students truly wouldn't be penalized for not submitting scores. It was a reflection of students' concerns, Pérez said. More than 500 colleges have signed NACAC's statement.
Colleges too have a role in diminishing testing's significance in a way that might force the magazine's hand. But sometimes they have announced test-optional policies that are ambiguous and confusing for students to interpret. Some signaled officials still valued the scores even though they would allow students to apply without them.
Schaeffer also accused U.S. News of peddling false information about colleges' testing policies. As an example, Schaeffer pointed to its profile of top-ranked Columbia University, which the publication states requires either the SAT or ACT for admissions. And right now, Columbia is temporarily test-optional.
U.S. News receives this information directly from colleges, Morse said in his email.
But in Schaeffer's view, "it's not hard to keep this information up to date. It confuses applicants at best."
A statement from the ACT also seemed to suggest scores have no place in the rankings. It said that while the "assessment offers a valid and trustworthy measure of a student's individual academic readiness, it was never intended for use as a measure of institutional prestige or quality."
The ACT remains "a significant and appropriate part of the college-going process," but the sector should seek out new and more effective tools that allow students to measure success and completion, the statement said.
PrevPreviousIndoor Hockey Activity Easy for Kids to Play at Home
NextTwo weeks in… | Three Teachers TalkNext


text:
\section*{Acknowledgments}

\section{Motivation} \label{background}

Given an input sequence $\bm{x}=x_1\ldots x_{T'}$, an autoregressive model~\cite{bahdanau2014neural,vaswani2017attention} predicts the target $\bm{y}=y_1\ldots y_{T}$ sequentially based on the conditional distribution $p(y_t|y_{<t}, x_{1:T'}; \theta)$, which tends to suffer from high latency in generation especially for long sequences. In contrast, non-autoregressive machine translation~\citep[NAT,][]{gu2017non}, proposed for speeding-up the inference by generating all the tokens in parallel, has recently been on trend due to its nature of parallelizable on devices such as GPUs and TPUs. A typical NAT system assumes a conditional independence in the output token space, that is
\begin{equation}
    p_\theta(\bm{y} | \bm{x}) = \prod_{t=1}^T p_\theta(y_t|x_{1:T'})
    \label{eq.nat}
\end{equation}
where $\theta$ is the parameters of the model. Typically, NAT models are also modeled with Transformer encoder-decoder without causal attention map in the decoder side.
However, as noted in~\citet{gu2017non}, the independence assumption generally does not hold in real data distribution for sequence generation tasks such as machine translation~\cite{ren2020study}, where the failure of capturing such dependency between target tokens leads to a serious performance degradation in NAT.

This is a fairly understandable but fundamental issue of NAT modeling which can be easily shown with a toy example in Figure~\ref{fig:toy}. Given a simple corpus with only two examples: \textit{AB} and \textit{BA}, each of which has $50\%$ chances to appear. It is designed to represent the dependency that symbol \textit{A} and \textit{B} should co-occur. Although such simple distribution can be instantly captured by any autoregressive model, learning the vanilla NAT model with maximum likelihood estimation (MLE, Eq.~\eqref{eq.nat}) assigns probability mess to incorrect outputs (\textit{AA}, \textit{BB}) even these samples never appear during training.
\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/example0.pdf}
    \caption{Toy example shows that NAT fails to learn when dependency exists in output space.}
    \label{fig:toy}
\end{figure}
In practice, the dependency in real translation corpus is much more complicated.
As shown in Figure~\ref{fig:example}, despite the inference speed-up,
training the vanilla NAT the same as Transformer in a comparable size leads to quality drop over $\textbf{10}$ BLEU points.

To ease the modeling difficulty, recent state-of-the-art NAT systems~\cite{lee2018deterministic,stern19a,ghazvininejad2019mask,gu2019levenshtein,kasai2020non,shu2020latent,saharia-etal-2020-non} trade accuracy with latency by incorporating an iterative refinement process in non-autoregressive prediction, and have already achieved comparable or even better performance than the autoregressive counterpart. Nevertheless, \citet{kasai2020deep} showed autoregressive models with a deep encoder and a shallow decoder can readily outperform strong iterative models with similar latency, indicating that the latency advantage of iterative NAT has been overestimated.

By contrast, while maintaining a clear speed advantage, fully NAT system -- model makes parallel predictions with single neural network forward -- still lags behind in translation quality and has not been fully explored in literature~\cite{libovicky-helcl-2018-end,li2018hint,sun2019fast,ma2019flowseq,GhazvininejadKZ20}. This motivates us in this work to investigate various approaches to push the limits of learning a fully NAT model towards autoregressive models regardless of the architecture choices~\citep{kasai2020deep}.

\section{Conclusion} \label{conclusion}
In this work, we aim to minimize the performance gap between fully NAT and AT models. We investigate some \textit{dependency reduction} methods from four perspective and carefully unite them with some necessary revisions. Experiments on three translation benchmarks demonstrate that our proposed models achieve state-of-the-art results of fully NAT models.
\section{Discussion and Future work} \label{discussion}
In this section, we go through the proposed four techniques again for fully NAT models. In spite of the success to close the gap with autoregressive models on certain benchmarks, we still see limitations when using non-autoregressive systems as mentioned in Table~\ref{Tab:bookRWCal}.

We and most of the prior research have repeatedly found that knowledge distillation (KD) is the indispensable \textit{dependency reduction} components, especially for training fully NAT models. Nevertheless, we argue that due to the model agnostic property, KD may lose key information that is useful for the model to translate. Moreover, \newcite{anonymous2021understanding} pointed out KD does cause negative effects on lexical choice errors for low-frequency words in NAT models.
Therefore, an alternative method that improves the training of NAT models over raw targets using such as GANs~\cite{binkowski2019high} or domain specific discriminators~\cite{donahue2020end} might be the future direction.

Apart from KD, we also notice that the usage of CTC loss is another key component to boost the performance of fully NAT models across all datasets. As discussed in \cref{4-2}, however, the need of up-sampling constrains the usage of our model on very long sequences or mobile devices with limited memory. In future work, it is possible to explore models to hierarchically up-sample the length with a dynamic ratio to  optimize the memory usage.

Lastly, both experiments with VAE and GLAT prove that it is helpful but not enough to train NAT models with loss based on monotonic alignments (e.g. CTC) only. To work on difficult pairs such as JA-EN, it may be a better option to adopt stronger models to capture richer dependency information, such as normalizing flows~\cite{oord2017parallel,ma2019flowseq} or non-parametric approaches~\cite{gu2018search}.

\section{Experiments} \label{experiment}
We perform extensive experiments on three challenging translation datasets by combining all mentioned techniques to check (1) whether the proposed aspects for \textit{dependency reduction} are complementary; (2) how much we can minimize the gap between a fully non-autoregressive model with the autoregressive counterpart.

\subsection{Experimental Setup}
\label{4-1}

\paragraph{Dataset and Preprocessing}
We validate our proposed models on three standard translation benchmarks with variant sizes, i.e., WMT14 English (EN) $\leftrightarrow$ German (DE) (4.0M pairs), WMT16 English (EN) $\leftrightarrow$ Romanian (RO) (610k pairs) and WMT20 Japanese (JA) $\rightarrow$ English (EN) (13M pairs after filtering).
For EN$\leftrightarrow$DE and EN$\leftrightarrow$RO,
we apply the same prepossessing steps and learn sub-words as mentioned in prior work (EN$\leftrightarrow$DE:~\citealp{zhou2019understanding}, EN$\leftrightarrow$RO:~\citealp{lee2018deterministic}).
For JA$\rightarrow$EN, the original data (16M pairs) is first filtered with Bicleaner~\cite{prompsit:2018:WMT}~\footnote{\url{https://github.com/bitextor/bicleaner}} and we apply SentencePiece~\cite{kudo-richardson-2018-sentencepiece} to generate 32,000 subwords.

\paragraph{Knowledge Distillation} Following previous efforts, the NAT model in our experiments is trained on distilled data generated from pre-trained transformer models (\textit{base} for WMT14 EN$\leftrightarrow$DE and WMT16 EN$\leftrightarrow$RO and \textit{big} for WMT20 JA$\rightarrow$EN) using beam search with a beam size $5$.

\paragraph{Decoding}\label{3_5}
At inference time, the most straightforward way is to generate the sequence with the highest probability at each position. The outputs from the CTC-based NAT models require additional collapse process $\Gamma^{-1}$ which can be done instantly.

A relatively more accurate method is to decode multiple sequences, and rescore them to obtain the best candidate in parallel, i.e. \textit{noisy parallel decoding}~\citep[NPD,][]{gu2017non}.
Furthermore, CTC-based models are also capable of decoding sequences using beam-search~\cite{libovicky-helcl-2018-end}, and optionally combined with $n$-gram language models~\cite{heafield-2011-kenlm,kasner2020improving}. More precisely, we search in a beam to approximately find the optimal $\bm{y}^*$ that maximizes:
\begin{equation}
    \log p_\theta(\bm{y}|\bm{x}) + \alpha \cdot \log p_{\text{LM}}(\bm{y}) + \beta \log |\bm{y}|
    \label{eq.ctc_beam}
\end{equation}
where $\alpha$ and $\beta$ are hyperparameters for language model scores and word insertion bonus.
In principle, it is no longer non-autoregressive as beam-search is a sequential process by nature. It does not contain any neural network computations and can be implemented efficiently in C++~\footnote{\url{https://github.com/parlance/ctcdecode}}.

\input{tables/performance-wiki103}

\paragraph{Baselines}
We adopt Transformer (AT) and existing NAT approaches (see Table~\ref{tab:main-rst}) for comparison.
For Transformers, except for using the standard \textit{base} and \textit{big} architectures~\cite{vaswani2017attention} as baselines, we also compare with a \textit{deep encoder shallow encoder} Transformer suggested in \newcite{kasai2020deep} that follows the parameterizations of \textit{base} with 12 encoder layers and 1 decoder layer (i.e. \textit{base} (12-1) for short).

\paragraph{Evaluation}
BLEU~\cite{papineni2002bleu} is used to evaluate the translation performance for all models. Following prior works, we compute tokenized BLEUs for EN$\leftrightarrow$DE and EN$\leftrightarrow$RO, while using SacreBLEU~\cite{post-2018-call} for JA$\rightarrow$EN.

In this work, we use three measures to fully investigate the translation latency of all the models:
\begin{itemize}[leftmargin=*]
    \vspace{-5pt}
    \item $\mathcal{L}_1^{\text{GPU}}$: translation latency by running the model with one sentence at a time on a single GPU. For most of existing works on NAT models, $\mathcal{L}_1^{\text{GPU}}$ is \textit{de-facto} which aligns applications like instantaneous machine translation.\vspace{-5pt}
    \item $\mathcal{L}_1^{\text{CPU}}$: the same as $\mathcal{L}_1^{\text{GPU}}$ while running the model without GPU speed-up. Compared to $\mathcal{L}_1^{\text{GPU}}$, it is less friendly to NAT models that make use of parallelism, however, closer to real scenarios.\vspace{-5pt}
    \item $\mathcal{L}_{\max}^{\text{GPU}}$: the same as $\mathcal{L}_1^{\text{GPU}}$ on GPU while running the model in a batch with as many sentences as possible. In this case, the hardware memory bandwidth are taken into account.\vspace{-1pt}
\end{itemize}
We measure the wall-clock time for translating the whole test set, and report the averaged time over sentences as the latency measure.

\input{tables/enja-performance}
\begin{figure*}[t]
\centering
\subfigure{
\includegraphics[width=0.31\textwidth]{figures/gpu.pdf}

\subfigure{
\includegraphics[width=0.31\textwidth]{figures/cpu.pdf}

\subfigure{
\includegraphics[width=0.31\textwidth]{figures/gpumax.pdf}

\caption{Quality v.s. Latency (the upper left corner achieves the best trade-off) for fully NAT models with other translation models (AT \textit{base} and \textit{base} 12-1~\cite{kasai2020deep}, CMLM~\cite{ghazvininejad2019mask} and LevT~\cite{gu2019levenshtein}) on WMT'14 EN$\rightarrow$DE.
We evaluate latency in three setups (from left to right: $\mathcal{L}_1^\text{GPU}$, $\mathcal{L}_1^\text{CPU}$, $\mathcal{L}_{\max}^\text{GPU}$) and show them in Logarithmic scale for better visualization.
\vspace{-10pt}}
\label{fig:speed_compare}
\end{figure*}

\paragraph{Implementation Details}
We design our fully NAT model with the hyperparameters of the \textit{base} Transformer: 8-512-2048~\cite{vaswani2017attention}.
For EN$\rightarrow$DE experiments, we also implement the NAT model in \textit{big} size: 8-1024-4096 for comparison.
For experiments using variational autoencoders (VAE), we use the last layer encoder hidden states to predict the mean and variance of the prior distribution. The latent dimension $D$ is set to $8$, and the predicted $\bm{z}$ are linearly projected and added on the encoder outputs.  Following~\newcite{shu2020latent}, we use a $3$ layer encoder-decoder as the posterior network, and apply freebits annealing~\cite{chen2016variational} to avoid posterior collapse.
By default, we set the length for decoder inputs $3\times$ as long as the source for CTC, while using the golden length for other objectives (CE, AXE). We also learn an additional length predictor when CTC is not used. For both cases, we use \textit{SoftCopy}~\cite{wei2019imitation} which interpolated the encoder outputs as the decoder inputs based on the relative distance of source and target positions. We choose
mask ratio $f_\text{ratio}=0.5$ for GLAT training.

For both AT and NAT models, we set the dropout rate $0.3$ for EN$\leftrightarrow$DE and EN$\leftrightarrow$RO, and $0.1$ for JA$\rightarrow$EN. We apply weight decay $0.01$ as well as label smoothing $\epsilon=0.01$. All models are trained for $300$K updates using Nvidia V100 GPUs with a batch size of approximately $128$K tokens. We measure the validation BLEU scores for every $1000$ updates, and average the best $5$ checkpoints to obtain the final model.
We measure the GPU latency by running the model
on a single Nvidia V100 GPU, and CPU latency on Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz
with 80 cores.
All models are implemented on \texttt{fairseq}~\cite{ott2019fairseq}

\subsection{Results}\label{4-2}
\paragraph{WMT'14 EN$\leftrightarrow$DE \& WMT'16 EN$\leftrightarrow$RO}
We report the performance of our fully NAT model comparing with AT and existing NAT approaches (including both iterative and fully NAT models) in Table~\ref{tab:main-rst}. Iterative NAT models with enough number of iterations generally outperform fully NAT baselines by a certain margin as they are able to recover the generation errors by explicitly modeling dependencies between (partially) generated tokens. However, such speed advantage is relatively marginal compared to AT \textit{base} (12-1) which also achieves $2.5$ times faster than the AT baseline.

Conversely,
our fully NAT models are able to readily achieve over $16$ times speed-up on EN$\rightarrow$DE by restricting translation with a single iteration. Surprisingly, simply training NAT with KD and CTC loss already beats the state-of-the-art for a single iteration NAT models across all four directions.
Moreover, combining with either latent variables (VAE) or glancing targets (GLAT) further closes the performance gap or even outperforms the AT results on both language pairs.
For example, our best model achieves \textbf{27.49} BLEU on WMT14 EN-DE -- almost identical to the AT performance (27.48) while $\bf{16.5}$ times faster in the inference time.
Also, Table~\ref{tab:main-rst} indicates the difficulties of learning NAT on each dataset. For instance, EN$\leftrightarrow$RO is relatively easier where ``KD + CTC'' is enough to reduce target dependencies, while by contrast, applying VAE or GLAT helps to capture non-monotonic dependencies and improve by $0.5\sim1$ BLEU points on EN$\leftrightarrow$ED.
For both datasets, we ONLY need single greedy generation to achieve similar translation quality as AT beam-search results.

\paragraph{WMT'20 JA$\rightarrow$EN}
We also present results for training the fully NAT model on a more challenging benchmark of WMT'20 JA$\rightarrow$EN which is much larger (13M pairs) and noisier. In addition, JA is linguistically distinct to EN which makes it harder to learn mappings between them. Consequently, both AT (12-1) and our fully NAT models (see Table~\ref{tab:enja-rst}) become less confident and tend to produce shorter translations (BP $<$ 0.9), and in turn underperform the AT teacher even trained with KD.
\paragraph{Beam search \& NPD} The performance of our NAT can be further boosted by allowing additional searching (beam-search) or re-ranking (NPD) after prediction. For CTC beam search, we use a fixed beam-size $20$ while grid-search $\alpha, \beta$ (Eq.\eqref{eq.ctc_beam}) based on the performance on the validation set. The language model~\footnote{\url{https://github.com/kpu/kenlm}} is trained directly on the distilled target sentences to avoid introducing additional information.
For noisy parallel decoding (NPD), we draw multiple $\bm{z}$ from the learned prior distribution~\footnote{We multiply $0.1$ to the variance for less noisy samples.}, and use the teacher model (AT \textit{big}) to rerank the best $\bm{z}$ with the corresponded translation.

As shown in Table~\ref{tab:enja-rst}, in spite of similar GPU latency ($\mathcal{L}_1^{\text{GPU}}$), beam search is much more effective than NPD with re-ranking, especially combined with a 4-gram LM where we achieve a BLEU score of $21.41$, beating the teacher model with $11\times$ speed-up. More importantly, by contributing the insertion bonus (3rd term in Eq~\eqref{eq.ctc_beam}) with $\beta$ in beam search, we have the explicit control to improve BP and output longer translations.
Also, we gain another half point by combining NPD and beam search. To have a fair comparison, we also report latency on CPUs where it is limited to leverage parallelism of the device. The speed advantage drops rapidly for NAT models, especially for NAT with NPD, however, we still maintain around 100 ms latency via beam search -- over 2$\times$ faster than the AT (12-1) systems with higher translation quality.

\paragraph{Quality v.s. Latency}
We perform a full investigation for the trade-off between translation quality and latency
across AT, iterative NAT and our fully NAT models. The results are plotted in Figure~\ref{fig:speed_compare}. For fully NAT models, no beam search or NPD is considered.
In all three setups, our fully NAT models obtain superior trade-off compared with AT and iterative NAT models.
Iterative NAT models (LevT and CMLM) require multiple iterations to achieve reliable performance with the sacrifice of latency, especially for $\mathcal{L}_1^\text{CPU}$ and $\mathcal{L}_{\max}^\text{GPU}$ where iterative NAT performs similarly or even worse than AT \textit{base} (12-1), leaving fully NAT models a  better position in quality-latency trade-off.

Figure~\ref{fig:speed_compare} also shows the speed advantage of fully NAT models shrinks in the setup of $\mathcal{L}_1^\text{CPU}$ and $\mathcal{L}_\text{max}^{\text{GPU}}$ where parallelism is constrained. NAT models particularly those trained with CTC cost more computations and memory compared to AT models with a shallow decoder. For instance when calculating $\mathcal{L}_\text{max}^{\text{GPU}}$, we notice that the maximum allowed batch is 120K tokens for AT \text{base} (12-1), while we can only compute 15K tokens at a time for NAT with CTC due to the up-sampling step, even though the NAT models win the actual wall-clock time. We mark it as one limitation for future research.

\subsection{Ablation Study}
\paragraph{Impact of variant techniques}
Our fully NAT models benefit from many dependency reduction techniques in four aspects (data, model, loss function and learning).
We analyze their effect on translation accuracy through various combinations in Table~\ref{tab:combine_tech}. First of all,
The combination without KD has a clear performance drop compared to the one with KD, showing its high importance in NAT training. For the loss function, although AXE and CTC both consider latent alignments between target positions, the CTC-based model obtains better accuracy by marginalizing all valid alignments. Incorporating latent variables also effectively improve the accuracy moderately (around 1 BLEU). Because of the capacity to reduce the mismatch between training and inference time, the model with GLAT is more superior to the one with the RNG training method. Moreover, we find that the KD and CTC are fundamental components to build a robust NAT model. Adding either VAE or GLAT to them achieve similar performance as AT models.

\input{tables/combination}

\input{tables/decoding}
\paragraph{Distillation corpus}
According to the ~\newcite{zhou2019understanding}, the best teacher model is relevant to the capacity of NAT models.
We report the performance of models trained on real data and distilled data generated from AT \textit{base} and \textit{big} models in Table~\ref{tab:big-distill}.
For \textit{base} models, both AT (12-1) and NAT achieve better accuracy with distillation, while AT benefits more by moving from
\textit{base} to \textit{big} distilled data.
On the contrary, the NAT model improves marginally indicating that in terms of the modeling capacity, our fully NAT model is still worse than AT model even with 1 decoder layer. It is not possible to further boost the NAT performance by simply switching the target to a better distillation corpus. Nonetheless, it is possible simultaneously increase the NAT capacity by learning in \text{big} size. As shown in Table~\ref{tab:big-distill}, we can achieve superior accuracy compared to AT (12-1) with little effect on the translation latency ($\mathcal{L}_1^\text{GPU}$).

\input{tables/upsample-ratio}
\paragraph{Upsampling Ratio ($\lambda$) for CTC Loss}
To meet the length requirements in CTC loss, we upsample the encoder output by a factor of $3$ in our experiments. We also explore other possible values and report the performance in Table~\ref{tab:upsample_ratio}. The higher upsampling ratio provides a larger alignment space, leading to better accuracy. Nevertheless, with a large enough sampling ratio, further increase will not lead to the performance increase. Because of the high degree of parallelism,  $\mathcal{L}_{1}^{\text{GPU}}$ speed is similar among these ratios. However, the model with a  larger ratio has a clear latency drop on CPU or GPU with large batches.

\section{Introduction} \label{introduction}
\begin{figure}[t]
    \centering
    \includegraphics[width=\linewidth]{figures/trade-off.pdf}
    \caption{The translation quality v.s. inference speed-up of the proposed model with the AT (Transformer) and existing popular iterative NAT models varying decoding iterations on WMT14 En-De test set. The upper right corner achieves the best trade-off.\vspace{-10pt}}
    \label{fig:example}
\end{figure}
State-of-the-art neural machine translation (NMT) systems are based on autoregressive models~\cite{bahdanau2014neural,vaswani2017attention} where each generation step depends on the previously generated tokens. Such sequential nature inevitably leads to inherent latency at inference time.
Non-autoregressive neural machine translation models~\cite[NAT,][]{gu2017non} attempt to generate output sequences in parallel to speed-up the decoding process. The incorrect independence assumption nevertheless prevents NAT models to properly learn the dependency between target tokens in real data distribution, resulting in a performance drop compared to autoregressive (AT) models. One popular solution to improve the translation accuracy is to sacrifice the speed-up by incorporating an iterative refinement process in NAT models, through which the model explicitly learns the conditional distribution over partially observed reference tokens. However, recent studies~\cite{kasai2020deep} indicated that
iterative NAT models seem to lose the speed advantage after carefully tuning the layer allocation of AT models. For instance, an AT model with \textit{deep encoder and shallow decoder} (12-1) obtains similar latency as iterative NAT models without hurting the translation accuracy.

Several works~\cite{GhazvininejadKZ20,saharia-etal-2020-non,qian2020glancing} have recently been proposed to improve the fully NAT models, though the performance gap compared to the iterative ones remains. How to build a fully NAT model with competitive translation accuracy calls for more exploration. In this work, we first argue that the key to successfully training a fully NAT model is to perform \textit{dependency reduction} in the learning space of output tokens (\cref{background}). With this guiding principle, we revisit various methods which are able to reduce the dependencies among target tokens as much as possible from four different perspectives, i.e., training corpus (\cref{3_1}), model architecture (\cref{3_2}), training objective (\cref{3_3}) and learning strategy (\cref{3_4}). Furthermore, such target token dependencies cannot be perfectly removed by any of these aspects only. The performance gap can not be near closed unless we make full use of these techniques' advantages.

We validate our fully NAT model on standard translation benchmarks including $5$ translation directions. The proposed system achieves new state-of-the-art results for fully NAT models. Moreover, compared to the Transformer baseline, our models achieve comparable performance with over \textbf{16$\times$} speed-up at inference time.

\section{Methods} \label{method}
\begin{figure*}[t]
    \centering
    \includegraphics[width=1.0\linewidth]{figures/NAT.pdf}
    \caption{The overall framework of our fully NAT model.}
    \label{fig:framework}
\end{figure*}
In this section, we discuss several important ingredients to train a fully NAT model. As discussed in \cref{background}, we argue that the guiding principle of designing any NAT models is to perform \textit{dependency reduction} as much as possible in the output space so that it can be captured by the NAT model. For example, iterative-based models~\cite{ghazvininejad2019mask} explicitly reduce the dependencies between output tokens by learning the conditional distribution over the observed reference tokens. The overall framework of training our fully NAT system is presented in Figure~\ref{fig:framework}.
\input{tables/comparison}
\subsection{Data: Knowledge Distillation} \label{3_1}
The most effective \textit{dependency reduction} technique is knowledge distillation (KD)~\cite{hinton2015distilling,kim2016sequence} which is firstly proposed in~\citet{gu2017non} and has been widely employed for all subsequent NAT systems. We replace the original target samples with sentences generated from a pre-trained autoregressive model. As analyzed in \newcite{zhou2019understanding}, KD is able to simplify the training data where the generated targets have less noise and are aligned to the inputs more deterministically. \newcite{zhou2019understanding} also showed that the capacity of the teacher model should be constrained to match the desired NAT model to avoid further degradation in KD, especially for weak NAT students without iterative refinement. We treat Transformer \textit{base} as the default teacher model to generate the distilled corpus.

\subsection{Model: Latent Variables}\label{3_2}
In contrast to iterative NAT, \textit{dependency reduction} can be done with (barely) zero additional cost at inference time by utilizing latent variables as part of the model. In such cases, output tokens $y_{1:T}$ are modeled conditionally independent over the latent variables $\bm{z}$ which are typically predicted from the inputs $x_{1:T'}$:
\begin{equation}
    p_\theta(\bm{y}|\bm{x})=\int_{\bm{z}}p_\theta(\bm{z}|\bm{x})\prod_{t=1}^Tp_\theta( y_t| \bm{z}, \bm{x})d\bm{z}
\end{equation}
where $z$ can be pre-defined by external library (e.g. fertility in~\citet{gu2017non}), or jointly optimized with the NAT model using normalizing flow~\citep{ma2019flowseq} or variational auto-encoders (VAEs)~\cite{kaiser2018fast,shu2020latent}.

In this work, we followed the formulation proposed in \newcite{shu2020latent} where continuous latent variables $\bm{z} \in \mathbb{R}^{T'\times D}$ are modeled as spherical Gaussian at the encoder output of each position. Like standard VAEs~\cite{kingma2013auto}, we train such model to maximize the evidence lower-bound (ELBO) with a posterior network $q_\phi$:
\begin{equation}\label{eqn:vae}
    \underbrace{\mymathop{\mathbb{E}}_{\bm{z}\sim q_\phi}\left[\log p_\theta(\bm{y}|\bm{z}, \bm{x})\right]}_{\text{likelihood}} - \mathcal{D}_{\text{KL}}(q_\phi(\bm{z}|\bm{x}, \bm{y})\|p_\theta(\bm{z}|\bm{x}))
\end{equation}
where we use an encoder-decoder architecture similar to the main model to encode $q_\phi(\bm{z}|\bm{x}, \bm{y})$. In our implementation, only the parameters of the embedding layers are shared between $\theta$ and $\phi$

\subsection{Loss Function: Latent Alignments}\label{3_3}
Standard sequence generation models are trained with the cross entropy (CE) loss which compares model's output with target tokens at each corresponded position. However, as NAT ignores the dependency in the output space, it is almost impossible for such models to accurately model token offset. For instance, while with little effect to the meaning, simply changing \textit{Vielen Dank !} to \textit{, Vielen Dank} causes a huge penalty for fully NAT models.

To ease the limitations in loss computation, recent works have proposed to consider latent alignments between the target positions, and ``actively'' search the best~\citep[AXE,][]{GhazvininejadKZ20} or marginalize all the alignments~\citep[CTC,][]{libovicky-helcl-2018-end,saharia-etal-2020-non} with dynamic programming. The dependency is reduced because the NAT model is able to freely choose the best prediction regardless of the target offsets.

In this work, we put our major focus on the
CTC loss~\cite{graves2006connectionist} considering its superior performance and the flexibility of variable length prediction.
Formally,
given the conditional independence assumption,
CTC is capable of efficiently finding all valid aligned sequences $\bm{a}$ which the target $\bm{y}$ can be recovered from, and marginalize log-likelihood:
\begin{equation}
    \log p_{\theta}(\bm{y}|\bm{x}) = \log \sum_{\bm{a} \in \Gamma(\bm{y})} p_{\theta}(\bm{a}|\bm{x})
\end{equation}
where $\Gamma^{-1}(\bm{a})$ is the collapse function that recovers the target sequence by
collapsing consecutive repeated tokens, and then removing all blank tokens.
Furthermore, it is straightforward to apply the same CTC loss into latent variable models by replacing the likelihood term in Eq~\eqref{eqn:vae} with the CTC loss. Note that both CTC and AXE make strong  assumptions of monotonic alignment, which makes them impossible to reduce all dependencies between target tokens in real distribution.

\subsection{Learning: Glancing Targets}\label{3_4}
Although it is possible to directly optimize the fully NAT model towards the target sequence $\log p_\theta(\bm{y}|\bm{x})$ (or ELBO if using latent variables), \newcite{ghazvininejad2019mask} showed that it improved test time performance by training the NAT model with randomly sampled reference tokens as inputs $\log p_\theta(\bm{y}|\bm{m}\odot\bm{y}, \bm{x}), \bm{m} \sim \gamma(l, \bm{y}), l \sim \mathcal{U}_{| \bm{y} |}$, where $\bm{m}$ is the mask, and $\gamma$ is the sampling function given the number of masked tokens $l$.
As mentioned earlier, we suspect such explicit modeling of the
distribution conditional to observed tokens assists the \textit{dependency reduction} in the output space.
\paragraph{Curriculum Learning}
Naively applying random masks for every training example may cause severe mismatch between training and testing.
\newcite{qian2020glancing} proposed GLAT -- a curriculum learning
strategy, in which the ratio of observed target tokens is related to the inference quality of the fully NAT model. More precisely, instead of sampling uniformly, we sample $l$ by:
\begin{equation}
    l \sim g(f_{\text{ratio}} \cdot \mathcal{D}(\bm{\hat{y}}, \bm{y}))
\end{equation}
where $\bm{\hat{y}}=\argmax_{\bm{y}}\log p_\theta(\bm{y}|\bm{x})$, $\mathcal{D}$ is the discrepancy  between the model prediction and the target sequence, e.g.
Levenshtein distance~\cite{levenshtein1966binary}, and $f_{\text{ratio}}$ is a hyperparameter to adjust the mask ratio. The original formulation~\cite{qian2020glancing} utilized a deterministic mapping ($g$), while we use a stochastic function such as Poisson distribution to enable sampling a wider range of lengths including ``no glancing''.

The original GLAT~\cite{qian2020glancing} assumes to work with golden length so that it can glance the target by placing the target word embedding to the corresponded inputs. It is natural to use GLAT for models with CE or AXE loss, while incompatible with CTC as we always require the inputs longer than the targets.
To enable GLAT training,
we glance target tokens from the viterbi aligned tokens $\bm{\hat{a}}=\argmax_{\bm{a}\in \Gamma(\bm{y})}p_\theta(\bm{a}|\bm{x})$ which has the same length as the decoder inputs.

Intuitively, the poorly trained model will observe more target tokens.  When the model becomes better and generate higher quality sequences, the number of masked words will be larger, which helps the model gradually learn generating the whole sentence.

\subsection{Summary}
To sum up, we conclude all proposed aspects with methods in Table~\ref{Tab:bookRWCal} which are  able to effectively help close the performance gap for fully non-autoregressive models. Note that, although none of these solutions are perfect that reduces the harmful dependency in the output space of NAT models. In practice, we find these methods target on different aspects of \textit{dependency reduction} and indicates the improvements may be complementary
\section{Related Work} \label{related-work}
Besides iterative and fully NAT models, some other works trying to improve the decoding speed of machine translation models from other aspects. One research lien is to mix AT and NAT models up. For example, \newcite{wang-etal-2018-semi-autoregressive} proposed a semi-autoregressive model which adopted non-autoregressive decoding locally but kept the autoregressive property in global. On the contrary, \newcite{kong-etal-2020-incorporating} and \newcite{ran-etal-2020-learning} introduced a local autoregressive NAT models which retained the non-autoregressive property in global.

Alternatively, there are also some works trying to improve the decoding speed of AT models directly. For example, model quantization and pruning have been widely studied as a way to improve the decoding speed~\cite{see-etal-2016-compression,junczys-dowmunt-etal-2018-marian,kim-etal-2019-research,aji-heafield-2020-compressing}. Also  teacher-student training can improve the translation accuracy of the student model with faster decoding speed~\cite{junczys-dowmunt-etal-2018-marian}.

redpajama_set_name: RedPajamaArXiv


text:
Get Off the Bottle
Nestle & Diversions
Model Legislation: Public Water, Public Justice
OUR20 Communities
The Sixth Great Lake
Algal Blooms
The Great Lakes Compact
Local Fracking Ordinances
Privatization and Commodification
Wetlands Destruction
Celebrate FLOW's "Public Trust Month" — July 2019
What Is the Public Trust?
Public Trust Education
Public Trust Principles
Why the Public Trust?
Reports & Media Releases
FLOW in the News
2015-2017 Biennial Report
Support FLOW Today!
Art Meets Water
Flow short description
FLOW (For Love of Water): Empowering Communities and Leaders to Protect the Great Lakes
FLOW: For Love of Water
[responsive-menu]
Tag: integrated resource planning
The Water-Energy Nexus: FLOW at the MI Governor's Energy Policy Listening Session
Posted on April 22, 2013 by Jim Olson - H2Olson
As part of Earth Day today, I had the opportunity to submit FLOW's memorandum to the Michigan Public Service Commission at the Governor's "listening session" on future energy policy in Michigan. At FLOW we believe that by looking at the entire hydrological cycle as a basis for addressing systemic threats like climate change–the single largest human induced diversion from the Great Lakes–water and energy policy and actions are inseparable. More importantly, if energy policy is elevated to an obligation to protect the integrity of water, something the commons and public trust in water may well require, then our energy policy can better promote jobs and economic stability and growth and protect water and the environment. FLOW Executive Director Liz Kirkwood also spoke at the forum to address the need to connect water to climate change, energy, and wasteful or inefficient and inequitable water consumption practices related to energy and food production, and other consumptive uses. You can read FLOW's memorandum on the Energy and Water Nexus, along with the rest of our reports, on our policy center page. The text of our comments are below:
Click here to read FLOW President Jim Olson's comments as a PDF
Click here to read FLOW Executive Director Liz Kirkwood's comments as a PDF
Statement by James M. Olson, Chair, FLOW
A Water and Energy "Nexus" Policy For Michigan
The "energy listening" sessions ordered by Governor Snyder to help Michigan fashion an energy policy are welcomed. However, at a time of a climate change crisis fired by coal and other greenhouse gases with severe and worsening impacts and costs, including increasing extreme low water levels, there is a disconnect between energy and Michigan's most valuable common treasure – water and the great lakes. No energy policy in Michigan should omit the protection of the integrity of our water – both quality and quantity- as one of if not its central core principles.
There is a rapidly increasing demand for water world wide and strong probability that global demand will outstrip supply in just 30 years. If anything drives the point home for a 21st century policy that centers on a "nexus" between water and energy, it is the staggering cost to life, property and communities from storms like those experienced in the northeastern United States this past year. Add to this the lowest water reported water levels in the Great Lakes, and devastating future climate change entails for all of us and our children in this century, and it becomes quite evident that water and energy are inseparable. It is imperative that water is declared the core of our energy policy. If we honor and respect the integrity of water and our Great Lakes, we will find and follow a sound energy policy.
Because of the need to address water and energy together (as a "nexus)," Michigan must move forward with a multi-disciplined framework that requires application of "integrated resource planning" principles for evaluating energy policy and options. The same should be true for water policy. This would require a goal and planning effort that seeks the least costly energy services and goods with a full evaluation of all costs to water, the ecosystem, and our communities. Without "full cost" and integrated resource planning, Michigan's energy policy and use will lead the state into an impoverishing downward spiral — economically, environmentally, and culturally. "Pure Michigan" and a sound sustainable economy and jobs mean pure air and pure water both in quality and quantity.
Therefore, it is my opinion, and I urge the governor, his advisors and staff, and the legislature to consider and adopt an energy policy that conforms to the integrity of water, the gravity of climate change, and a dynamic open mindedness that applies full cost evaluation and integrated resource policy. If we fail to do this, Michigan will fall into decline while other parts of North America and the world begin to prosper.
Michigan sits in the middle of the most valuable water/ecosystem in the world. It is held in public trust so that it is protected from impairment and loss. The Great Lakes Compact and Water Quality Agreement of 2012 underscore this principle and enact a policy that these waters are held in trust and should not be diverted or loss by consumptive uses, and this requires a response to keep greenhouse gases in check.
In one week, thermal electric power plants use (a net loss to great lakes) as much water as the Chicago Diversion. Energy costs are rising, water levels are falling, water is more essential than coal-fired or other fossil fueled power, including the extraction of equally water intensive fuels like fracturing deep shale for natural gas – deep shale fracturing will displace or remove approximately 21 million gallons in 21 days for just one gas well. Multiply this times the 1,000 wells we will see if this is not carefully considered and regulated, and it will result in a permanent loss of 21 billion gallons of water from fragile headwater areas.
The only sound and secure goal for Michigan is to move quickly toward a renewable and efficient energy world. This will diversify, increase, and lower cost of energy supplies, reduce costly infrastructure, reduce toxic air and water impacts, and temper the effects of climate change, including our plummeting water levels. Equally important, it will set Michigan on a course to lead the nation and help the next generation create positive profitable investments, cheaper more appropriate power, new industries and jobs – batteries, solar, wind, and conservation. Michigan must enact a "greenhouse trust fund" for any so-called "bridge fuels" like natural gas so that the justification of such a water-intensive environmentally risky method of extraction will be assured by a conversion to a renewable energy economy.
Michigan and Michiganders are nothing without water. Any approach to energy without integrity of water as its core principle and without an immediate shift to renewable energy and efficiency will put Michigan in an economic and environmentally disastrous downward spiral. We owe to ourselves and children and grand children to put water and Community first. It is a matter of water and public trust. It is a matter of survival.
FLOW Executive Director Liz Kirkwood's Comments on the Water-Energy Nexus
My name is Liz Kirkwood and I'm the Executive Director of FLOW ("For Love Of Water"), which is a water policy and educational institute dedicated to understanding the threats and solutions to water in the Great Lakes by focusing on the nexus between water, energy, food, and climate change.
I want to thank Michigan Public Service Commission Chairman John D. Quackenbush and Michigan Energy Office Director Steve Bakkal for the opportunity to speech and address overall question 1: What information do energy policy makers need to consider in order to make good energy decisions?
Michigan faces a watershed moment and opportunity to chart a new cleaner energy course that is good for jobs, good for the environment, good for energy affordability, and good for the water.
To chart this new course, we first must recognize that our energy choices profoundly affect our water and cause serious climate change impacts.
Water and energy are inextricably linked. Water is used and lost in energy-resource extraction, refining and processing, transportation, and electric-power generation. And yet, because water is such a cheap commodity, it is rarely calculated and balanced in our energy decisions. Let's change this so that the water-energy nexus become an integral part of charting Michigan's energy future plans.
By 2035, the amount of water consumed for current energy production is projected to double. During this same time, there will be increasing water scarcity from pollution, waste, drought and human-induced climate change and impacts.
Given the clear interrelationship between energy, food, and water, we can no longer "silo" these sectors; rather we must improve decision-making with greater integration and collaboration between water resource management and energy production.
This calls for a new vision that recognizes the nexus between water, energy, food, and climate change. To make this shift, we must view water in a different light where water becomes the starting point for everything we do. Without water the health of our people, economy and ecosystem are diminished.
The recent U.S. natural gas industry shale boom has reignited attention on the water-energy-climate change nexus. The big issue with hydraulic fracking is the water, both in terms of sheer quantity (e.g., 300 million gallons to frack 13 wells in Kalkaska County) and safe disposal of chemical-laden and often toxic waste water that will never return to our hydrologic cycle. Before Michigan embraces natural gas as a "bridge" fuel, we must conduct a generic analysis of cumulative impacts on water, environment, and health that properly weighs the unprecedented risks that fracking poses to our precious water resources.
Additionally, Michigan's coal-fired power plants are the state's largest single source of heat-trapping carbon dioxide emissions, which are detrimentally contributing to climate change by increasing lake evaporation and causing our extreme low water levels in Lake Michigan-Huron.
In fact, we hit record low water levels in January of this year – 26 inches below average – according to data collected by the U.S. Army Corps of Engineers since 1918. The water levels issue is at the heart of the Great Lakes' and Michigan's economy, energy and water needs, social fabric, quality of life, and environment. In March of this year, our Governor signed legislation providing $21 million in taxpayer emergency funds to dredge state harbors that are in danger of becoming impassable because of low water levels.
We cannot sit idle anymore; rather we must adapt our current fossil-fuel economy to one with low-carbon and low-water footprint. Water in effect must become the center of everything we do, such that shifting to renewables becomes the obvious energy choice and addresses the root causes of receding water levels so that we do not jeopardize our current and future way of life.
Michigan is already witnessing renewable energy sources like wind becoming more cost effective and affordable to our businesses and citizens than polluting traditional sources like coal and oil. Wind is at 4.5 cents/KWH as opposed to traditional blended energy sources at 7.6 cents/KWH. The benefits of renewable energy are clear: affordable, clean, stable rates, Michigan job generator, minimal water use, and protective of human health and the environment.
In addition, Michigan should promote energy efficiency and energy conservation in all sectors because it is the cheapest, cleanest, and most quickly deployed source of energy.
To chart this new course, Michigan must embrace its innovative manufacturing traditions and promote renewable energy sources to reduce pressure on water resources and limit adverse climate change impacts. We think Michigan can and should become a leader in renewable energy, and at a minimum compete with the neighboring states that currently generate 20%+ of renewable power with excellent reliability.
We urge the State of Michigan to think wisely about its future energy choices, pay for water consumed, and ensure that the long-term energy decisions are good for our water too. Once we chart this path, then we can proudly say we are living up to our motto: "Pure Michigan."
Tags: Climate Change, Energy Policy, FLOW, Governor Snyder, great lakes, integrated resource planning, jim olson, Liz Kirkwood, Michigan, Michigan Public Service Commission, systemic threats, Water Energy Nexus, water policy
Line 5 Pipelines
FLOW News
© 2018 FLOW: For Love of Water | 153 1/2 East Front Street, Suite 203C | Traverse City, MI 49684 | 231.944.1568 | Attributions | Privacy Policy

redpajama_set_name: RedPajamaCommonCrawl


text:
using System;
using System.Diagnostics;
using OpenCvSharp.XPhoto;
using Xunit;
using Xunit.Abstractions;

namespace OpenCvSharp.Tests.XPhoto;

// ReSharper disable InconsistentNaming
public class XPhotoTest : TestBase
{
    private readonly ITestOutputHelper testOutputHelper;

    public XPhotoTest(ITestOutputHelper testOutputHelper)
    {
        this.testOutputHelper = testOutputHelper;
    }

    [Fact]
    public void ApplyChannelGains()
    {
        using var src = Image("lenna.png");
        using var b = new Mat(src.Rows, src.Cols, src.Type());
        using var g = new Mat(src.Rows, src.Cols, src.Type());
        using var r = new Mat(src.Rows, src.Cols, src.Type());
        CvXPhoto.ApplyChannelGains(src, b, 2, 1, 1);
        CvXPhoto.ApplyChannelGains(src, g, 1, 2, 1);
        CvXPhoto.ApplyChannelGains(src, r, 1, 1, 2);

        if (Debugger.IsAttached)
        {
            using var combined = new Mat(src.Rows * 2, src.Cols * 2, src.Type());
            using var roi1 = new Mat(combined, new Rect(0, 0, src.Cols, src.Rows));
            using var roi2 = new Mat(combined, new Rect(src.Cols, 0, src.Cols, src.Rows));
            using var roi3 = new Mat(combined, new Rect(0, src.Rows, src.Cols, src.Rows));
            using var roi4 = new Mat(combined, new Rect(src.Cols, src.Rows, src.Cols, src.Rows));
            src.CopyTo(roi1);
            b.CopyTo(roi2);
            g.CopyTo(roi3);
            r.CopyTo(roi4);
            Window.ShowImages(combined);
        }
    }

    [Fact]
    public void GrayworldWBBalanceWhite()
    {
        using var wb = CvXPhoto.CreateGrayworldWB();
        using var src = Image("lenna.png");
        using var dst = new Mat(src.Rows, src.Cols, src.Type());
        wb.BalanceWhite(src, dst);

        if (Debugger.IsAttached)
        {
            using var combined = new Mat(src.Rows, src.Cols * 2, src.Type());
            using var roi1 = new Mat(combined, new Rect(0, 0, src.Cols, src.Rows));
            using var roi2 = new Mat(combined, new Rect(src.Cols, 0, src.Cols, src.Rows));
            src.CopyTo(roi1);
            dst.CopyTo(roi2);
            Window.ShowImages(combined);
        }
    }

    [Fact]
    public void GrayworldWBProperties()
    {
        using var wb = CvXPhoto.CreateGrayworldWB();
        var saturationThreshold = wb.SaturationThreshold;

        const float val = 100f;
        wb.SaturationThreshold = val;

        Assert.Equal(val, wb.SaturationThreshold);
        Assert.NotEqual(saturationThreshold, wb.SaturationThreshold);
    }

    [Fact]
    public void Inpaint()
    {
        using var src = Image("building.jpg");
        using var mask = Image("building_mask.bmp", ImreadModes.Grayscale);
        using var dst = new Mat(src.Size(), src.Type());
        CvXPhoto.Inpaint(src, mask, dst, InpaintTypes.SHIFTMAP);
        ShowImagesWhenDebugMode(src);
        ShowImagesWhenDebugMode(dst);
    }

    [Fact]
    public void LearningBasedWBBalanceWhite()
    {
        using var wb = CvXPhoto.CreateLearningBasedWB(null);
        using var src = Image("lenna.png");
        using var dst = new Mat(src.Rows, src.Cols, src.Type());
        wb.BalanceWhite(src, dst);

        if (Debugger.IsAttached)
        {
            using var combined = new Mat(src.Rows, src.Cols * 2, src.Type());
            using var roi1 = new Mat(combined, new Rect(0, 0, src.Cols, src.Rows));
            using var roi2 = new Mat(combined, new Rect(src.Cols, 0, src.Cols, src.Rows));
            src.CopyTo(roi1);
            dst.CopyTo(roi2);
            Window.ShowImages(combined);
        }
    }

    [Fact]
    public void LearningBasedWBExtractSimpleFeatures()
    {
        using var wb = LearningBasedWB.Create(null);
        using var src = Image("lenna.png");
        using var dst = new Mat();
        wb.ExtractSimpleFeatures(src, dst);

        unsafe
        {
            // 1. Chromaticity of an average (R,G,B) tuple
            // 2. Chromaticity of the brightest (R,G,B) tuple (while ignoring saturated pixels)
            // 3. Chromaticity of the dominant (R,G,B) tuple (the one that has the highest value
            //    in the RGB histogram)
            // 4. Mode of the chromaticity palette, that is constructed by taking 300 most common
            //    colors according to the RGB histogram and projecting them on the chromaticity plane.
            //    Mode is the most high-density point of the palette, which is computed by a straightforward
            //    fixed-bandwidth kernel density estimator with a Epanechnikov kernel function.
            testOutputHelper.WriteLine($"{dst.DataPointer[0]}");
            testOutputHelper.WriteLine($"{dst.DataPointer[1]}");
            testOutputHelper.WriteLine($"{dst.DataPointer[2]}");
            testOutputHelper.WriteLine($"{dst.DataPointer[3]}");
        }
    }

    [Fact]
    public void LearningBasedWBBalanceWhiteWithModel()
    {
        // About model file
        // http://docs.opencv.org/trunk/dc/dcb/tutorial_xphoto_training_white_balance.html
        using var wb = CvXPhoto.CreateLearningBasedWB("");
        using var src = Image("lenna.png");
        using var dst = new Mat(src.Rows, src.Cols, src.Type());
        wb.BalanceWhite(src, dst);

        if (Debugger.IsAttached)
        {
            using var combined = new Mat(src.Rows, src.Cols * 2, src.Type());
            using var roi1 = new Mat(combined, new Rect(0, 0, src.Cols, src.Rows));
            using var roi2 = new Mat(combined, new Rect(src.Cols, 0, src.Cols, src.Rows));
            src.CopyTo(roi1);
            dst.CopyTo(roi2);
            Window.ShowImages(combined);
        }
    }

    [Fact]
    public void LearningBasedWBProperties()
    {
        using var wb = LearningBasedWB.Create(null);
        var histBinNum = wb.HistBinNum;
        var rangeMaxVal = wb.RangeMaxVal;
        var saturationThreshold = wb.SaturationThreshold;

        const int ival = 100;
        const float fval = 100f;
        wb.HistBinNum = ival;
        wb.RangeMaxVal = ival;
        wb.SaturationThreshold = fval;

        Assert.Equal(ival, wb.HistBinNum);
        Assert.Equal(ival, wb.RangeMaxVal);
        Assert.Equal(fval, wb.SaturationThreshold);

        Assert.NotEqual(histBinNum, wb.HistBinNum);
        Assert.NotEqual(rangeMaxVal, wb.RangeMaxVal);
        Assert.NotEqual(saturationThreshold, wb.SaturationThreshold);
    }

    [Fact]
    public void SimpleWBBalanceWhite()
    {
        using var wb = CvXPhoto.CreateSimpleWB();
        using var src = Image("lenna.png");
        using var dst = new Mat(src.Rows, src.Cols, src.Type());
        wb.BalanceWhite(src, dst);

        if (Debugger.IsAttached)
        {
            using var combined = new Mat(src.Rows, src.Cols * 2, src.Type());
            using var roi1 = new Mat(combined, new Rect(0, 0, src.Cols, src.Rows));
            using var roi2 = new Mat(combined, new Rect(src.Cols, 0, src.Cols, src.Rows));
            src.CopyTo(roi1);
            dst.CopyTo(roi2);
            Window.ShowImages(combined);
        }
    }

    [Fact]
    public void SimpleWBProperties()
    {
        using var wb = SimpleWB.Create();
        var inputMax = wb.InputMax;
        var inputMin = wb.InputMin;
        var outputMax = wb.OutputMax;
        var outputMin = wb.OutputMin;
        var p = wb.P;

        const float val = 100f;
        wb.InputMax = val;
        wb.InputMin = val;
        wb.OutputMax = val;
        wb.OutputMin = val;
        wb.P = val;

        Assert.Equal(val, wb.InputMax);
        Assert.Equal(val, wb.InputMin);
        Assert.Equal(val, wb.OutputMax);
        Assert.Equal(val, wb.OutputMin);
        Assert.Equal(val, wb.P);

        Assert.NotEqual(inputMax, wb.InputMax);
        Assert.NotEqual(inputMin, wb.InputMin);
        Assert.NotEqual(outputMax, wb.OutputMax);
        Assert.NotEqual(outputMin, wb.OutputMin);
        Assert.NotEqual(p, wb.P);
    }

    [Fact]
    public void DctDenoising()
    {
        using var src = Image("lenna.png");
        using var dst = new Mat();
        CvXPhoto.DctDenoising(src, dst, 1);

        if (Debugger.IsAttached)
        {
            Window.ShowImages(src, dst);
        }
    }

    [Fact]
    public void Bm3dDenoising()
    {
        using var src = Image("lenna.png", ImreadModes.Grayscale);
        using var dst = new Mat();
        CvXPhoto.Bm3dDenoising(src, dst);

        if (Debugger.IsAttached)
        {
            Window.ShowImages(src, dst);
        }
    }

    [Fact]
    public void OilPainting()
    {
        using var src = Image("lenna.png", ImreadModes.Grayscale);
        using var dst = new Mat();
        CvXPhoto.OilPainting(src, dst, 5, 10);

        if (Debugger.IsAttached)
        {
            Window.ShowImages(src, dst);
        }
    }

#if NET48
        [ExplicitStaFact]
        public void Sample()
        {
            if (!Debugger.IsAttached)
                return;

            string[] files;
            using (var dialog = new System.Windows.Forms.OpenFileDialog
            {
                RestoreDirectory = true,
                Multiselect = true,
                Filter = "Image Files(*.png;*.jpg;*.jpeg;*.bmp)|*.png;*.jpg;*.jpeg;*.bmp"
            })
            {
                if (dialog.ShowDialog() != System.Windows.Forms.DialogResult.OK)
                    return;
                files = dialog.FileNames;
            }

            var desktop = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
            var dstDir = System.IO.Path.Combine(desktop, "WB");
            System.IO.Directory.CreateDirectory(dstDir);

            using var simpleWB = SimpleWB.Create();
            using var learningWB = LearningBasedWB.Create("");
            using var grayworldWB = GrayworldWB.Create();

            foreach (var file in files)
            {
                testOutputHelper.WriteLine(System.IO.Path.GetFileNameWithoutExtension(file));

                using var src = new Mat(file);
                using var dstSimple = new Mat();
                using var dstLearning = new Mat();
                using var dstGrayworld = new Mat();
                simpleWB.BalanceWhite(src, dstSimple);
                learningWB.BalanceWhite(src, dstLearning);
                grayworldWB.BalanceWhite(src, dstGrayworld);

                using var temp1 = new Mat();
                using var temp2 = new Mat();
                using var dst = new Mat();
                Cv2.HConcat(src, dstSimple, temp1);
                Cv2.HConcat(dstLearning, dstGrayworld, temp2);
                Cv2.VConcat(temp1, temp2, dst);

                /*
                using (new Window("src", src))
                using (new Window("dst", dst))
                {
                    Cv2.WaitKey();
                }*/
                dst.SaveImage(System.IO.Path.Combine(dstDir, $"{System.IO.Path.GetFileNameWithoutExtension(file)}.png"));
            }
        }
#endif
}

redpajama_set_name: RedPajamaGithub


text:
Il Select Committee on Intelligence (Comitato ristretto per l'intelligence) è un organismo del Congresso degli Stati Uniti d'America che si occupa di sovraintendere alle attività di intelligence ed ai programmi di agenzie e uffici del governo federale degli Stati Uniti. È istituito sia al Senato che alla Camera dei rappresentanti.

Lo "United States Senate Select Committee on Intelligence" è stato creato nel 1976 con la risoluzione del Senato USA n. 400, nel 94º congresso. È composto da 15 senatori.

Lo "United States House Permanent Select Committee on Intelligence" è stato creato nel 1977 con la risoluzione n. 658 della Camera dei rappresentanti. È composto da 21 deputati.

Tutte le agenzie governative statunitensi della United States Intelligence Community riferiscono a ogni comitato per le questioni relative alla sicurezza nazionale, comprese NSA, CIA, FBI, DOD, NRO e tutte le forze armate.

Note

Voci correlate
 Congresso degli Stati Uniti d'America
 United States Intelligence Community

Altri progetti

Collegamenti esterni

Servizi segreti statunitensi
Parlamento statunitense

redpajama_set_name: RedPajamaWikipedia


text:
Jan. 6 Panel Requests Meeting With Rep. Loudermilk Regarding Jan. 5 Capitol Tours
Jan. 6 Panel Requests Meeting With Rep. Loudermilk Regarding Jan. 5 Capitol Tours Rep. Barry Loudermilk, R-Ga. (Photo by Joe Raedle/Getty Images)
By Jay Clemons | Thursday, 19 May 2022 03:47 PM
The House panel investigating the Jan. 6, 2021 unrest at the Capitol requested that Rep. Barry Loudermilk, R-Ga., voluntarily appear before its investigators, believing the Georgia Republican might have special insight about reconnaissance tours in the building that apparently occurred the previous day (Jan. 5, 2021).
In its letter to Loudermilk, the panel wrote:
"Based on our review of evidence in the Select Committee's possession, we believe you have information regarding a tour you led through parts of the Capitol complex on January 5, 2021.
"Public reporting and witness accounts indicate some individuals and groups engaged in efforts to gather information about the layout of the U.S. Capitol, as well as the House and Senate office buildings, in advance of January 6, 2021."
The letter does not implicate Loudermilk or establish him as a person of interest.
However, it did note Loudermilk's previous involvement with the House Committee on Administration, which reportedly urged for the "release of Capitol Police footage of the Capitol's tunnels and entrances and exits on Jan. 5."
According to The Hill, a February letter to the Capitol Police Board indicated that surveillance videos from Jan. 5, 2021 showed "no tours, no large groups, no one with MAGA hats on."
However, the "Select Committee's review of evidence directly contradicts that denial," said the panel in its letter to Loudermilk.
The Loudermilk request comes shortly after the House committee subpoenaed five Republicans, including Minority Leader Kevin McCarthy, R-Calif.
The other subpoenas went to Reps. Jim Jordan, R-Ohio, Scott Perry, R-Pa., Andy Biggs, R-Ariz., and Mo Brooks, R-Ala.
The panel has reportedly interviewed nearly 1,000 witnesses and collected more than 100,000 documents on the events of Jan. 6.
However, the House committee has already determined that former President Donald Trump likely won't be called to testify.
The nine-member Jan. 6 panel is comprised of seven Democrats and just two Republicans; and both GOP members — Rep. Liz Cheney, R-Wyo., and Rep. Adam Kinzinger, R-Ill. — have been Trump critics.
The House committee doesn't have formal judicial power, Jan. 6 panel chair Rep. Bennie Thompson, D-Miss., recently told the Washington Post.
"We're not a criminal body — we are just looking for the facts and circumstances around Jan. 6, but in the course of that review, there are some very troubling things that we've come upon that we think if [the DOJ] would take a look at it, there would be something there," he said.
Rep. Budd to Newsmax: Win in North Carolina Came After Focus on Counties
Doug Mastriano to Newsmax: 'I Was Thrilled' to Get Trump Endorsement


TEL AVIV — More than any other Jewish text, the Passover Haggadah — the book that guides participants through the Seder — is a quilt sewn over generations. It is mysterious and beautiful, and often restless. It weaves together social justice (inviting the needy to join the meal), storytelling (about how our forefathers were enslaved in Egypt), ceremonial symbols (unleavened bread and a shank bone, to name just two), pedagogy (asking questions from a wise son and a wicked one), celebration (for the Exodus) and pleas for help ("Pour out Your wrath upon the nations who do not know You").
The Haggadah also encapsulates two competing instincts in Jews today. On one hand is the tribal-national sense of peoplehood — the Exodus from Egypt represents the creation of a people, the Jewish people. On the other hand is the universalist-humanist — the Hebrews' emancipation from Pharaoh has become a symbol of freedom from oppression, and "Let my people go" a rallying cry everywhere.
Because it reconciles these two threads in Jewish life, Passover is appreciated by nearly everyone in the Jewish community and more widely observed than any other holiday. In Israel, more than 90 percent of Jews celebrate Passover; in the United States, 70 percent. But this holiday can also be a moment of polarization, and the Haggadah can be a Rorschach test as Jews choose competing interpretations.
Jews today use the Haggadah to emphasize social justice, feminism, secularism, vegetarianism. Some add a fifth cup of wine to the traditional four either to note the ultimate "redemption," the establishment of the State of Israel, or to celebrate Moses's sister Miriam, and the role of women throughout Jewish history.
For many generations, this is how we answer this question — as the Haggadah guides us. For many generations, we have followed a script. There is an answer for the wise son, and an answer for the wicked son, one for the simple son, one for the silent son. The Haggadah doesn't tell us how to answer the son who cannot stop talking about the news, not even for one night. Maybe it is hinting that he does not merit an answer.
Passover is a Jewish holiday that commemorates the emancipation of the Jewish people from slavery in Egypt. It typically falls in early- to mid-April, coinciding with 15th to the 22nd of the Jewish month of Nissan, based on the lunar calendar.
People who observe Passover will attend a Seder, a dinner where the story of the Exodus from Egypt is recounted. Seder means 'order' in Hebrew because the Seder ritual is performed in a specific order.
People celebrating Passover may abstain from eating leavened bread and eat Matzo, which is a kind of unleavened bread, instead.
The four main events at a Seder are eating matzo, eating bitter herbs, drinking four cups of wine associated with blessings and reading from the Haggadah, or book that tells the Passover story.
Judaism teaches that the Jews were living as slaves in Egypt, forced into heavy labor and poor living conditions. God heard their cries and sent Moses to free the people from the Pharaoh's rule. Moses asked Pharaoh to "Let my people go."
When Pharaoh refused, God unleashed a series of 10 plagues upon Egypt. Moses asked Pharaoh for the Jews' freedom after each of the plagues. When Pharaoh rejected Moses' requests, a new plague struck Egypt. After the last plague, the killing of the first born sons of Egyptians, Pharaoh relented and the Jews were set free. The first born sons of Jewish people were spared, or "passed over," which gives the holiday its namesake.
The Jews packed up their belongings, but did not have time to wait for their bread to rise, baking Matzo instead. Pharaoh however changed his mind and sent his army after the Jews. Once they reached the Red Sea, the Jews feared they were trapped. Instead, the Torah teaches that God parted the Red Sea allowing the Jews to walk through to safety, but closed the sea again as Pharaoh's army tried to follow.
0 Response to "Keep Your Politics Out of Passover"


text:
Catedral de nuestra señora de Kazan en Moscu, Cathedral of Our Lady of Kazan in Moscow, recorrido por la catedral de Kazan en Moscu, tour of the Kazan Cathedral in Moscow, Visit the Kazan Cathedral in the red square, Visitar la catedral de Kazan en la plaza roja
Catedral de nuestra señora de Kazan
Kazan Cathedral in Red Square
The Kazan Icon Cathedral of the Mother of God was erected in memory of the liberation of the Russian state from the Polish-Lithuanian invaders, which was achieved with the help and intercession of the Mother of God, who showed his mercy through the Miraculous Kazan icon. The temple was built at the expense of the first tsar of the Romanov Mikhail Feodorovich dynasty and consecrated in 1636. Since the construction of the temple became one of the most important churches in Moscow, his abbot occupied one of the first places in the clergy of Moscow.
Throughout its history, the cathedral was rebuilt several times, in the 1760s, 1802-05, 1865.
In the 1920s the renovationists served in the cathedral for some time. In the years 1925-1933. The restoration of the cathedral was carried out under the direction of the architect P.D. Baranovsky In 1928 the bell tower of the cathedral was demolished. In 1930, the Kazan Cathedral was closed, and in 1936 – demolished.
The cathedral was restored in 1990-1993. at the expense of the Moscow City Hall and donations from citizens. Kazan Cathedral is the first of the Moscow temples completely lost during the Soviet era, which was recreated in its original forms. It was possible to recreate the historical appearance of the temple thanks to the measures taken by the architect P.D. Baranovsky before the destruction of the temple, and the studies of historian S.A. Smirnova On November 4, 1993, the church was consecrated by His Holiness Patriarch Alexy II.
The main throne was consecrated in honor of the Kazan Icon of the Mother of God, the northern corridor – St. Guria, archbishop of Kazan and San Barsanuphius, bishop of Tver. South hall in honor of the schmchch. Hermogenes and Tikhon, Patriarchs of Moscow and All Russia (not consecrated).
Every day, hours and liturgy at 9.00, night worship begins at 17.00, Sundays and holidays; the liturgy is at 7.00 and 10.00, the day before, the all-night vigil at 17.00 and on Fridays after the end of the eve, a prayer service with the Akathist of the Kazan icon of the Mother of God with blessing.
The temple is open every day from 9 am until the end of the night service.
Kazan Cathedral on the corner of the Red Square in Moscow has a difficult destination. It was completely demolished in 1936 and restored after 57 years. Today it is an active temple in which the services are carried out. The main sanctuary of the cathedral, which the faithful come from all over the country, is the icon of the Mother of God "Kazan", revered by the miraculous, with a pagan guardian. The rector of the church is the archpriest Nikolai Inozemtsev.
Moscow tours for adults and children
Individual and group. Tourist excursions Excursions to the Red Square and the Kremlin. Water walks on the Moscow river. Excursions for children and schoolchildren.
Tourists visit the cathedral more frequently when they examine the main attractions of the capital: the Red Square, the Alexander Garden, St. Basil's Cathedral, the State History Museum and other popular sites near the Moscow Kremlin.
Hotels near Kazan Cathedral in Moscow
Service hours of the Kazan Cathedral in Red Square
Those who wish to receive the Sacrament of Baptism can contact the ministers for a reading talk every Saturday at 3pm.
Religious events are held daily in the temple:
09:00 (working days) 07:00 and 10:00 (weekends) – liturgy,
17:00 – night services (parastas, night vigil).
On the Kazan Cathedral site, you can see a detailed schedule of events for the current month, as well as know the names of the clergy who will perform the rites and their assistants.
Calendar of services 2019 on the official website of the Cathedral of the Kazan Icon of the Mother of God
Icon of Moscow Kazan Cathedral
The Kazan icon of the Mother of God is one of the most revered shrines not only in the Kazan Cathedral, but also in Christianity in general. It is worth noting that the temple does not store the original icon, but a list, but this does not detract from it. There is a copy in Dmitrovsky's side chapel.
The first appearance of the icon of the Mother of God occurred in 1579 in Kazan. Its location was shown by the matron girl, who had a prophetic dream. After obtaining the icon, he was repeatedly transferred to other churches in the country, many copies of him were written, almost indistinguishable from the original.
To avoid damage and theft, temples were often exposed with copies, and the original was carefully kept. However, in 1904 the icon was stolen, the farmer Chaykin turned out to be the kidnapper, who said he had burned the relic. Later versions were presented according to which the destruction of the Kazan Icon of the Mother of God was questioned. Until now, historians cannot reach a definitive opinion on whether the original Kazan Icon of the Mother of God has been preserved and where it can be located.
In the following years, the temple was rebuilt and repaired repeatedly. The last restoration was carried out in 1925. The architect Pyotr Baranovsky wanted to restore the building to its original appearance. The work lasted 4 years, and when there was very little left, the Moscow City Council decided to demolish the temple. The restoration was halted, the bell tower was destroyed in the same year, and the cathedral's facilities until 1936 were used as a warehouse to store construction materials for the subway station. After the demolition of the Kazan Cathedral, a pavilion was placed in honor of the III International in its place.
In 1990, the Moscow authorities decided to rebuild the cathedral and, therefore, began collecting donations. The sanctuary was restored from photographs and drawings preserved in archives. The reconstruction work of the Kazan Icon Cathedral of the Mother of God was completed in 1993.
The Kazan Icon Cathedral of the Mother of God is a typical example of temple architecture in the first half of the 17th century. This is a single-domed temple with two aisles. The north and south corridors are joined by a gallery that surrounds the structure from three sides.
The exterior of the cathedral is decorated with mosaic icons in the Russian style. They represent the Virgin, Jesus Christ, Saint George, the Patriarch of Moscow and other saints.
Kazan Cathedral is located in the center of Moscow, on Nikolskaya Street. Reaching it will not be difficult: there are several metro stations and public transport stops within walking distance of the sanctuary. Alternatively, take a taxi or drive by car.
The subway is perhaps the most convenient transport in the capital. From the Okhotny Ryad station closest to the cathedral (red line), walk about 100 meters from exit number 7. In addition, the Teatralnaya station (green line) is not far away, and a little further is the "Plaza de la Revolution "(blue line).
At 300 meters, near the Okhotny Ryad metro station, there are stops: Okhotny Ryad Metro and Manezhnaya Square. There are buses and minibuses: No. M2 (to Riga station), M3, M10, M27, N1 (night to Sheremetyevo airport), H2 (night), H11 (night to Vnukovo airport), 101, 144, 904 .
A little further (from the opposite side of the Red Square, near the St. Basil's Cathedral) there is another bus stop: "Red Square". From there there are buses No. M5 and 158.
Moscow public transport portal
Since the cathedral is located right in the center of the city, a car trip would not be the most convenient option. In addition to the fact that parking is paid, finding a free place is very problematic. Therefore, if a trip by public transport is not suitable, it is better to use a taxi or a transfer. The capital has many mobile applications to order a taxi: Yandex. Taxis, Lucky, Uber, Gett, Maxim and others.


Videobooth is photobooth brought to life! We put together a short but quirky, fun video montage highlighting all of your guest's special moments and messages. Step in front of the booth, grab the mic and start talking! We engage your guests by asking questions pertaining to your special occasion, have them leave shout-outs or encourage them to just get plain crazy and act out some fun scenarios! The material is then handed over for editing, creating a video montage that is cut, snipped and sewed to your preference. This includes music, special effects and text to personalize your keepsake to a tee.


text:
Michael Paulson and David Gelles
'Hamilton' Inc.: The Path to a Billion-Dollar Broadway Show - The New York Times
Just how many Hamiltons can "Hamilton" make? Broadway's new blockbuster has only been open for 10 months, but already, this musical about America's founding fathers is one of the great successes in recent theatrical history. The show's cast members were invited to the White House and have become regular guests on talk shows. Miranda, the musical's creator, was on the cover of Rolling Stone.
Fans and enterprising camp outside the theater around the clock, hoping to snag seats. Tickets are almost impossible to come by, except those resold at highly prices. And celebrities have streamed in: Beyoncé.
Barbra. Bernie. All that enthusiasm means a lot of money for the men and women who brought the show to Broadway.
Months ago, "Hamilton" recouped the $12. 5 million it cost to mount. It is now making an estimated $600, 000 a week in profit, and every indication is that its touring productions, which begin in Chicago in September and in San Francisco next March, will be highly profitable, too.
Producers and investors together stand to make tens or even hundreds of millions of dollars. Mr. Miranda will become quite wealthy.
And a diverse group of collaborators — from the lighting designer to Mr. Miranda's father — will share in the bounty. Even if "Hamilton" isn't playing in Bangkok decades after its debut (as was the case with "Phantom of the Opera") it's not unreasonable to expect that Mr.
Miranda's unlikely hit will ultimately generate upward of $1 billion in sales. Of course, nothing is certain: Theater history is dotted with fizzled triumphs. ("The Producers," for example, was a on Broadway and toured, but lost steam.
) After the June 12 Tony Awards, "Hamilton" will face a new set of challenges, with new cast members and new markets. But there is a demonstrated path to riches for the few musicals able to translate Broadway success into and international popularity: "The Lion King" and "Phantom" each claims to have grossed more than $6 billion worldwide, and "Wicked" has passed $4 billion. Forecasting the future for "Hamilton" is risky, but the show's current fortunes, and the model provided by other shows, are instructive.
Here, based on documents filed with government agencies and interviews with people familiar with various aspects of the musical's finances, is a look at how "Hamilton" makes money and where that money goes. 'The revolution's happening in New York.' On Broadway, "Hamilton" is consistently selling out all 1, 321 seats at the Richard Rodgers Theater and is currently grossing about $1.
9 million a week in ticket sales. Simply by maintaining that pace, the show would bring in nearly $100 million a year (that's grosses, not profits). But there is reason to expect that figure to rise.
In a bid to blunt profiteering — the widespread scalping of tickets at highly prices — the show may soon raise prices above the current ceiling of $475 for premium seating. At $100 million a year, the Broadway production of the show would pass the $1 billion mark in a decade. The show's current pace will be tough to sustain, but not unimaginable — "Wicked" this year set a record by reaching the $1 billion mark on Broadway in just over 12 years, and "Hamilton," although running in a smaller theater, has higher ticket prices.
'We create. You just wanna move our money around.' The hottest ticket on Broadway cost, on average, $172 during the week that ended June 5.
But that's just what the initial purchaser paid. Many theatergoers pay considerably more by purchasing resold tickets on sites like StubHub, Craigslist and Ticketmaster. Those tickets are being listed for as much as $5, 018 each, but most are sold for considerably less.
The average resale of "Hamilton" tickets on StubHub is roughly $872, according to a New York Times analysis, a markup of $700 above the current average original ticket sale price. For any given performance, roughly 13 to 22 percent of the seats at the Richard Rodgers — somewhere between 180 and 300 tickets — are available on the secondary market, according to The Times's research and interviews with ticket sellers. So for each performance of "Hamilton," ticket sellers and brokers are reaping roughly $150, 000.
With the Broadway cast putting on more than 400 shows per year, that means these sellers could reap about $60 million per year, just in New York — money the producers, investors and Mr. Miranda will never see. 'Tomorrow there'll be more of us.
' If "Hamilton" follows the pattern of other recent theatrical megahits, its profits will substantially increase when it plays outside of New York. The big money goes to shows that globalize by running productions in multiple countries, often at the same time. ("The Phantom of the Opera," for example, has been mounted in 13 languages in 151 cities in 30 countries.
) "Hamilton" is moving quickly to take advantage of its moment. The Chicago production will begin just 13 months after the Broadway opening and run indefinitely. The PrivateBank Theater in Chicago, where "Hamilton" will play, is much bigger than the show's Broadway house, meaning that even if ticket prices are somewhat lower, revenues could be strong, and that run could nearly double the show's weekly profitability.
"Hamilton" is also planning two separate North American touring productions: one beginning in March in San Francisco, and one beginning in Seattle in 2018. The theaters on the road are likely to price tickets lower than in New York, but many of them hold more people, so these two tours, on top of Chicago, could dramatically increase the show's current revenues and profits. The show's producers will soon announce a fifth production, in London, and if "Hamilton" follows the path of other monster musicals, it will eventually explore the possibilities of productions in Asia, Australia or Continental Europe.
Profits from foreign productions are difficult to forecast, and given that "Hamilton" is a lyrically dense musical about American history, its appeal outside the United States, and in languages other than English, is uncertain. 'Can I buy you a drink?' From to CDs to books, fans cannot get enough "Hamilton. " The cast album has sold over 428, 000 copies domestically, according to Nielsen.
Track downloads have topped 212, 000 streams from sources like Spotify and Apple Music have eclipsed 365 million. Together, that's roughly $11 million in gross retail sales. More music is on the way.
Mr. Miranda is working with the Roots' drummer, Questlove, on a new recording of "Hamilton" songs, and more, featuring pop stars like Busta Rhymes, Ben Folds, Regina Spektor and Chance the Rapper. "Hamilton" can even sell books.
"Hamilton: The Revolution," a book about the creation of the musical by Jeremy McCarter and Mr. Miranda, went on sale in April with a list price of $40. In less than two months, it sold more than 101, 000 copies, according to Nielsen, and hit the No.
1 spot on the New York Times hardcover nonfiction list. (Other authors have benefited from "Hamilton" fever, too: Ron Chernow's 2004 biography, which inspired Mr. Miranda to write the musical, has spent 33 weeks on the paperback list.
This fall, Three Rivers Press will publish Jeff Wilser's book "Alexander Hamilton's Guide to Life. ") Many of those who manage to get inside the Richard Rodgers Theater visit the gift stand, which sells hats, magnets and mugs. And goodies are available even to those who can't get inside the theater.
Bloomingdale's is selling "Hamilton" apparel in New York, and plans to do the same in other cities as the show tours. Online, there is a thriving market for unlicensed "Hamilton" tchotchkes: perfume, dolls and key chains made by enterprising fans. There is even a beer.
Gun Hill Brewing in the Bronx has made a "Hamilton" pale ale. The brew won't enrich Mr. Miranda and his team instead, some profits are going to Graham Windham, the social services organization by Alexander Hamilton's wife, Eliza.
Dave Lopez, a partner at Gun Hill, says, "Anything 'Hamilton' sells these days. " 'Who provided those funds?' The vast majority of profits from "Hamilton" is going to the producers and investors who put up the $12. 5 million to finance the show.
Jeffrey Seller, the lead producer, will reap big financial rewards. A veteran of other Broadway hits, including "Rent" and Mr. Miranda's first musical, "In the Heights," Mr.
Seller splits 42 percent of the net profits from the show with its other producers, Jill Furman and Sander Jacobs. (They are not saying how they divvy up their share.) That means the producers are splitting about $13 million annually from the current Broadway production.
The producers are also entitled to 3 percent of the adjusted gross from "Hamilton," which amounts to about $52, 000 weekly, or $2. 7 million annually, from the Broadway production. (Producers commonly get both a royalty and a share of profits.
) For Mr. Seller, it doesn't end there. His production company, Adventureland, is entitled to another 5 percent of net profits, or about $1.
6 million a year, from the current production. Adventureland receives these torchbearer points in recognition of his role in the show's development. The investors who backed "Hamilton" — a group of about 100 people that includes Robert Greenblatt, the NBC Entertainment chairman, as well as the power publicist Ken Sunshine — split another 42 percent of the profits, or about $13 million annually, from the Broadway production.
(The list of "Hamilton" investors has not been made public.) Those are big numbers for the theater world, but they are likely to get much bigger in the coming years, as the show opens productions beyond New York. 'Why do you write like you're running out of time?' Mr.
Miranda, the show's creator, wears an unusual number of hats: He came up with the idea he wrote the music, book and lyrics and he is, for now, the star. That means he gets a salary (as an actor) as well as authorship royalties and profits that ordinarily would be split among multiple people. He is expected to leave the cast this summer and will then lose his salary (an undisclosed amount) but he will continue to receive the royalties and profits for as long as the show runs.
He is likely to make at least $6. 4 million from the Broadway production over the next year his earnings will grow considerably when the other productions begin. The show has also dramatically transformed his earning potential.
Disney has proved particularly enthusiastic: Mr. Miranda wrote a cantina song for "Star Wars: The Force Awakens," is writing music for the animated film "Moana" and has been cast to star opposite Emily Blunt in a sequel to "Mary Poppins. " A film version of "In the Heights," which had stalled in Hollywood, is back in development and a film version of "Hamilton" is likely.
Mr. Miranda is also the ultimate rights holder for "Hamilton. " After the show's commercial life ends, he will own the rights for productions in regional theaters and schools, which can generate considerable revenue over time.
'I never had a group of friends before. I promise that I'll make y'all proud.' Mr.
Miranda rewards loyalty. The director, choreographer and orchestrator for "Hamilton," as well as the show's costume, lighting and sound designers, all worked on "In the Heights. " All of them — and the show's set designer — are entitled to royalties, and all are likely to make millions of dollars over the life of the show.
Thomas Kail, the director, has known Mr. Miranda since the two were introduced after both graduated (a few years apart) from Wesleyan University. Mr.
Kail helped develop "In the Heights" and "Hamilton. " He is entitled to 2. 5 percent of the adjusted gross revenue from "Hamilton," or about $2.
3 million a year just from the New York production. Mr. Kail also receives 1.
5 percent of net profits, currently about another $470, 000 annually. Andy Blankenbuehler, the show's choreographer, receives 1. 75 percent of gross sales and 0.
5 percent of profits — currently totaling $1. 7 million or so annually. Mr.
Miranda's musical partner, Alex Lacamoire, gets just under 1 percent of gross sales, currently about $790, 000 annually. And the show's costume, lighting and sound designers — Paul Tazewell, Howell Binkley and Nevin Steinberg — receive between 0. 37 and 0.
5 percent of gross sales, currently worth about $336, 000 to $454, 000 per year. As with all of the royalty and arrangements associated with "Hamilton," the value of the creative team's cut will increase significantly when the productions in Chicago and beyond begin. 'We'll give the world to you, and you'll blow us all away.
' This is the one late addition to the "Hamilton" profit plan: Mr. Seller agreed in April to share some of the show's earnings with the performers who helped as the musical was being developed. The details are still being ironed out, but probably the casts at the Public Theater and on Broadway, and perhaps participants in workshops at the Public Theater, will share 1 percent of the profits from the Broadway production and will also share a portion of the profits of touring productions.
The is likely to affect about 30 people, including not only actors and dancers but also stage managers. It's not clear how much they would get, but perhaps about $10, 000 a year from the Broadway production alone. That is not a lot of money compared with what others are earning, but it is still significant to performers, many of whom work for relatively low salaries and have long periods between roles.
(On Broadway, actors make a minimum of $1, 900 a week the performers in "Hamilton" are making more than that. But many performers spend much of their careers Off or Off Off Broadway, where salaries are considerably lower.) The "Hamilton" performers worked hard to get a promise — after they were unable to negotiate a deal as the Broadway production was opening, they hired a lawyer to represent them.
It took them eight months after the Broadway production opened to win an agreement they had argued that it was not fair, given their contributions to the show, to exclude them from sharing in its success. 'Don't forget from whence you came.' "Hamilton" has proved lucrative for three others who helped along the way: Ron Chernow, a historian whose biography of Alexander Hamilton inspired and formed the basis for the musical the Public Theater, the Off Broadway nonprofit that nurtured the show's development and presented its production and, most surprisingly, Luis A.
Miranda Jr. Miranda's father. Mr.
Chernow, who is the musical's historical consultant and its tireless champion, gets 1 percent of the show's adjusted grosses as a royalty — currently about $900, 000 a year. The Public Theater, which famously lived for years off the profits from transferring "A Chorus Line" to Broadway, gets 5 percent of the profits and 1 percent of the adjusted gross — currently totaling about $2. 5 million a year.
The Public says it is putting most of the money into its cash reserves rather than its operating budget. Luis Miranda Jr. a political consultant, is the most unusual beneficiary.
He is an unabashed fan of musical theater who collects cast recordings and sings in the car, and he introduced his son to the genre. He is often present at his son's side, accompanying him to events and helping him manage many aspects of his life and career. He is getting 1 percent of the show's profits — currently about $312, 000 a year.
In each case, the value of their share will increase as the show adds productions. 'The art of the trade. How the sausage gets made.
' Even before the profits or the royalties for "Hamilton" are distributed, there is a more fundamental expense: the cost of putting on the show. Winches and chain motors are $3, 200 a week. Dressers are $20, 000 a week.
And electrics and sound cost $28, 000 a week. "Hamilton" pays about $295, 000 per week for the Richard Rodgers, a fee that covers rent as well as the house staff, musicians, stagehands and more. The show's cast and crew are paid about $220, 000 altogether a week.
Advertising costs a relatively low $80, 000 per week, thanks to all the free publicity. But things like taxes, pensions, lawyers and equipment rentals add up. On the scale of Broadway musicals, "Hamilton," with its modest set and midsize cast, has moderate running costs.
All in all, it costs about $650, 000 a week to present the show on Broadway, or about $34 million a year. But it's proving to be one of the best investments in town..
Far-Right Party in Austria Challenges Results of Presidential Vote - The New York Times There Are More White Voters Than People Think. That's Good News for Trump. - The New York Times