We're all creatures of habit, and work more efficiently when we know exactly where to find the tools we need. The same holds true with digital tools, and those of us who work with SQL Server will spend a lot of time using SQL Server Management Studio. SSMS has a lot of features, but each data professional will likely use this environment very differently based on their job role and preferences. Fortunately, the SSMS environment is very customizable and allows personal configuration to suit each user.
In this brief post, I'll share some of the preferences I set in Management Studio, and will show those settings can be saved and shared.
Disable Prevent saving changes that require table re-creation. This one is a no-brainer, and frankly shouldn't be necessary – the setting I use should actually be the default (but sadly isn't). By default, SSMS blocks any change from the object designer that would require a table to be dropped and recreated. Since this is a common part of what a data professional will do during development or testing, this is one of the most common settings one should change in SSMS.
To enable operations requiring table recreation, open the Settings window and go to the Designers tab. Uncheck the box as indicated below.
Remove unused buttons from the toolbar. One of the most visible changes one can make to SSMS is to customize the buttons that appear on the toolbar. Management Studio comes loaded with several rows full of tool buttons, but it's unlikely that every data professional will use all of these. The SSMS UI allow each user to select which of the toolbars will appear, as well as customizing which buttons appear on those toolbars that are visible.
As shown, each toolbar can be customized using the down arrow to the right and selecting Add or Remove Buttons. Or, remove that toolbar entirely by right-clicking the toolbar area and deselecting that toolbar.
Personally, I'm more of a keyboard shortcut guy than a toolbar guy, so I'm a minimalist when it comes to my SSMS toolbar.
Turn on line numbers. Showing line numbers in code is mostly decorative, but can be helpful during troubleshooting or shared development exercises. When executing T-SQL code, some errors will report the line number in which the error occurred. Also, when working with others during development, it is very handy to be able to point your colleague to a specific line number. For those reasons, I always turn on the setting to show line numbers in SSMS code.
Line numbering can be turned on in SSMS by navigating to the Settings window, and drilling into Text Editor –> All Languages (or just Transact-SQL to limit just to T-SQL code).
Again, turning this on doesn't change the way the code executes, but it does make it easier to identify where an error exists.
Set custom status bar colors for common database connections. When querying multiple environments (Dev, Test, Prod, etc.) using similar code, it's very important to know which environment you're working in. A feature in SSMS will allow the assignment of custom status bar colors to known SQL Server connections. What this means is that you can set up one color for Dev, another for Test, and yet another for Prod, and queries connected to those SQL Server instances will show the assigned color in the status bar at the bottom of the query.
Setting up the custom color is done when making the connection, and will be remembered by SSMS for future connections to that same instance.
Once that custom color has been defined, queries connected to that SQL Server instance will show the designated color in queries using that SQL Server connection.
Turn on the Windows beep for query execution completion. This rarely-used option gives you an audio alert when a query completes. Have you ever been multitasking and have had to keep checking a query window to see if the query had completed? SSMS can be configured to sound the Windows default beep notification at the end of each query. This setting is found under the Query Results setting.
Enabling this option will sound an alert when each query finishes executing.
Increase the maximum column output size for text results. Infrequently, I have the need to write query results to text and copy/paste those results elsewhere. By default, when writing results to the output window in plain text (as opposed to a data grid), any text field will be truncated to a maximum of 256 characters. I always set this to the maximum of 8192 characters as shown below.
Save settings to a shared file. If the prospect of repetitively doing all of the above (and perhaps more, depending on your preferences) isn't appealing, here's the best part: Most of your SSMS settings are portable from one machine to the next. If you use multiple machines or need to copy your settings to a new client machine, you can easily share custom settings through just a single config file.
As shown below in the Environment –> Import and Export Settings tab, the Management Studio settings are stored in a file in the user's Windows profile directory.
This file location is configurable, so I just point mine to my Dropbox folder. That allows me to share this among all of the machines on which I use SSMS.
There are numerous configuration options you can set to suit the way you interact with Management Studio. These few are just a handful of settings that I use. Don't forget to make use of the settings file if you use multiple machines or need to migrate from one SSMS install to another.
5 Comments on "My SQL Server Management Studio Setup"
Love this post! The only "quibble" I would say is .. I would suggest never using the table designer, but I am in the camp of making things scriptable (Very typical to have to move change X to environments Y, Z .. N). That being said, I would still agree with your suggestion.
Thank you! I'm already using some of then.
There is one big soh having the status in progress and another (GD type) non-validated entry.
Thank you for the post. I have implemented the custom colors and they are not showing up when I connect and run queries off of the applicable databases. I think it is because I am using the version of SSMS 17.6 utilizing the hidden 'Dark' color theme, which I have the ability to use as I have found a way to make it present in the selection screen.


text:
\section{Introduction}
Recurrent neural networks (RNNs) \cite{hochreiter1997long} are ubiquitous in deep learning, because their design enables them to process arbitrary length sequential data. For example, RNNs and their variants like the Gated Recurrent Unit (GRU) \cite{chung2014empirical} and Long Short-Term Memory (LSTM) \cite{hochreiter1997long} have been core components in numerous applications, such as machine translation \cite{cho2014learning}, image/video captioning \cite{wang2016image,song2018deterministic}, and action recognition \cite{gammulle2017two,ma2018time}. There are other works in studying the dynamics of generalization, learning, and initializations of RNNs using neural tangent kernels \cite{alemohammad2020recurrent,alemohammad2020scalable}. However, even though RNNs are powerful tools, they are challenging to interpret and explain.

From \cite{gilpin2018explaining}, they state that interpretability has many definitions. The definition of interpretability \cite{gilpin2018explaining} that we will use is the ability to be understood by a person. In addition, the term, explainability is different from interpretability and will be defined as the ability to be explained in a mathematical formulation.

Deep learning interpretability has become an important topic since RNNs and their variants have become useful in several applications. The minimal amount of interpretability and explainability does not reveal why they have this performance on their tasks. With more interpretability with these models, it can explain why they are performing well on their respective tasks. Plus, if the model is not performing well, it is difficult to assess which specific parts of the signal contribute to the final decision. Since the model is opaque and hard to interpret, this creates a black-box effort for a practitioner.

There are works that have contributed to providing interpretability/explainability about how these RNNs perform. For instance, the work from \cite{guo2019exploring} explores the connection of the input variables to the RNN in order to interpret the performance. Other works such as \cite{li2017recurrent,guan2019towards,jiang2020cold} focus on the interpretability of RNNs within their respective task, hence limiting the ability to interpret other tasks. \cite{jiang2020cold,foerster2017input,ming2019interpretable} create their own type of RNN in order to provide interpretability in their application. The works of \cite{guan2019towards,li2017recurrent,arras2019explaining} use some metrics to provide interpretability with the RNN/LSTM that they use for their application.

With the works that created an interpretable RNN (\cite{jiang2020cold,foerster2017input,ming2019interpretable}), the main issue is that the definition of interpretability is focused on their application, which would make it difficult to adapt to another application for interpretability. For instance, \cite{jiang2020cold} creates a finite-automaton RNN for text classification. This work would be hard to adapt to other applications. Another issue is that with the works mentioned, they cannot provide a mathematical formulation that explains the prediction with the hidden states.

\textbf{Contributions.}
We propose NeuroView-RNN (NV-RNN) as a novel general framework that provides enhanced interpretability and explainability to classification.

\begin{enumerate}
    \item We introduce the NV-RNN framework, which consists of a concatenation of all of the hidden states to be the input to the linear classifier. This allows a linear mapping to all of the hidden states to the classes. This linear mapping allows interpretability and explainability since the prediction for each class is in a dot product of the weights from the linear classifier with the hidden states.

    \item Influenced from the work of \cite{barberan2021neuroview}, we provide interpretability and explainability defined in the Introduction to showcase how NV-RNN, NV-GRU, and NV-LSTM can provide more understanding especially with applications and architectures of RNNs, GRUs, and LSTMs. This is in stark contrast to the work of \cite{barberan2021neuroview} that used convolutional neural networks (CNNs) for image classification which cannot on its own be used in specific RNN architectures like bidirectional RNNs or variable length input.

    \item We have better performance on most of the datasets from Table~\ref{table:1} to show why this NV-RNN framework should be used. In addition, we have on par performance on some of the datasets from Table~\ref{table:1} and \ref{table:2}. The results show that we have the accuracy performance compared to typical RNNs and are able to interpret and explain the performance.

    \item We use numerous case studies about weight initialization, bidirectional GRUs, semantic analysis, video action recognition, and counterfactuals to show the benefit of NV-RNN.
\end{enumerate}

\section{Background}

\subsection{Recurrent Neural Networks}

Given an input sequence data
${\bm{x}} = \{{\bm{x}}_t\}_{t = 1}^{T}$ of length $T$ with data at time $t$, ${\bm{x}}_t\in\mathbb{R}^m$, an RNN performs the following recursive block computation at each time step $t$
\begin{align}
  {\bm{h}}^{\left(t\right)}({\bm{x}}) &= \mathcal{F}_{\theta}({\bm{h}}^{\left(t-1\right)}({\bm{x}}), {\bm{x}}_t) \in \mathbb{R}^n \label{rnn-main},
\end{align}
where $h^{(0)}({\bm{x}}) = 0$ and $n$ is the number of parameters for the hidden state. $\mathcal{F}_{\theta} : \mathbb{R}^n\times \mathbb{R}^m \rightarrow \mathbb{R}^n$ is the hidden time steps mapping with time-agnostic parameters $\theta$. Each recurrent architecture has a different $\mathcal{F}_{\theta}$. For a simple RNN \cite{ElmanRNN} we have this formulation,
\begin{align}
    \mathcal{F}_{\theta}({\bm{h}}^{\left(t-1\right)}({\bm{x}}), {\bm{x}}_t) = {\bm{h}}^{(t)}({\bm{x}}) = \phi \left( {\bm{W}}{\bm{h}}^{\left(t-1\right)}({\bm{x}}) + {\bm{U}}{\bm{x}}_t + {\bm{b}} \right),
\end{align}
where $\phi: \mathbb{R} \rightarrow \mathbb{R}$ is the activation function that act point-wise on a vector and $\theta = \mathrm{vect}\big[\{{\bm{W}},{\bm{U}},{\bm{b}} \}\big]$ contains the mapping parameters. In case of simple RNNs, we use the sigmoid function $\phi(\alpha) = \frac{1}{1 + e^{( - \alpha)}}$. Other RNNs variants such as GRU \cite{cho2014learning} and LSTM \cite{hochreiter1997long} have a more complex mapping $\mathcal{F}_{\theta}$. See Supplementary Material Section \ref{sm-arch} for more details.

Bidirectional recurrent architectures \cite{schuster1997bidirectional}, use two separated \emph{forward} and \emph{reverse} direction
\begin{align}
    {\bm{h}}_f^{\left(t\right)}({\bm{x}}) &= \mathcal{F}_{\theta_f}({\bm{h}}_f^{\left(t-1\right)}({\bm{x}}), {\bm{x}}_t) \\
    {\bm{h}}_r^{\left(t\right)}({\bm{x}}) &= \mathcal{F}_{\theta_r}({\bm{h}}_r^{\left(t+1\right)}({\bm{x}}), {\bm{x}}_t),
\end{align}
where $\theta_f$ and $\theta_r$ are independent of each other and together form the network parameters $\theta = \{\theta_{f},\theta_r \}$. The final hidden state is obtained by concatenation of each direction hidden states,
\begin{align}
    {\bm{h}}^{(t)}({\bm{x}})=
  \left[{\bm{h}}_f^{(t)}({\bm{x}})^\top,{\bm{h}}_r^{(t)}({\bm{x}})^\top\right]^\top \in \mathbb{R}^{2n}. \label{eq:bidirectional}
\end{align}

The output of a \emph{many to one} recurrent architecture is generally a linear transform of the last hidden state $T$:
\begin{align}
    f_{\theta}({\bm{x}}) = {\bm{V}}{\bm{h}}^{(T)}({\bm{x}}) \in \mathbb{R}^d.
\end{align}

Many-to-one recurrent architecture refers to when the input is a sequence of data but the output is decided at the end. They are used in applications like sentiment analysis or time-series classification when the input is a sequence and the output is to decide which class it is.
For recurrent architectures with average pooling \cite{shen2016empirical}, the output is a linear transform of the sum of all hidden states:
\begin{align}
    f_{\theta}({\bm{x}}) = \sum_{t = 1}^{t} {\bm{V}} {\bm{h}}^{(t)}({\bm{x}}) \in \mathbb{R}^d.
\end{align}

\section{NV-RNN: Interpretable and Explainable Recurrent Neural Network}

NV-RNN is inspired by the work in \cite{barberan2021neuroview} except in that work the authors only focused on 2D CNNs. The work in \cite{barberan2021neuroview} focuses on the spatial filters of CNNs whereas our work is adapted to use for RNNs. This paper focuses on RNNs with their variants and the hidden states per time. \cite{barberan2021neuroview} could only be used for CNN architectures and we adapt it to the RNN architecture which is different since CNNs focus on spatial features while RNNs focus on temporal features. In addition, the applications to RNNs have not been explored to specific RNN architectures like bidirectional RNN or varying input length which scenario is not present with CNNs and image classification. Therefore, adapting the work from \cite{barberan2021neuroview} is non-trivial and brings merit into interpretability and explainability in the definitions that we denoted in the introduction.

\subsection{NV Architecture Description}

Given the sequences of hidden states calculated in Equation \ref{rnn-main} for any recurrent network, the output from NV-RNN is obtained first by acquiring the hidden states for all time steps
\begin{align}
\label{quntized}
    {\bm{q}}^{(t)}({\bm{x}}) = \mathrm{ReLU} \left({\bm{h}}^{(t)}({\bm{x}}) \right),
\end{align}
where $\mathrm{ReLU}(\alpha) = \max (\alpha, 0)$. In the next step, the output is calculated using
\begin{align}
    f_{\theta}({\bm{x}}) = \sum_{t = 1}^{T} \left({\bm{V}}^{(t)}\right)^\top{\bm{q}}^{(t)}({\bm{x}}) =  \sum_{t = 1}^{T} f^{(t)}({\bm{x}}) \in \mathbb{R}^{d}. \label{eq:vq}
\end{align}

This is equivalent to concatenate all the hidden states per time step into a large hidden state vector ${\bm{Q}}({\bm{x}})$, where
\begin{align}
   {\bm{Q}}({\bm{x}})=
  \left[{\bm{q}}^{(1)}({\bm{x}})^\top,{\bm{q}}^{(2)}({\bm{x}})^\top,\dots,{\bm{q}}^{(T)}({\bm{x}})^\top\right]^\top \in \mathbb{R}^{nT},
\end{align}
and concatenate all linear output weights into one large matrix
\begin{align}
   {\bm{V}}=
  \left[{\bm{V}}^{(1)},{\bm{V}}^{(2)},\dots,{\bm{V}}^{(T)}\right] \in \mathbb{R}^{nT \times d}.
\end{align}
and calculate the output as the following
\begin{align}
    f_{\theta}({\bm{x}}) = {\bm{V}}^\top {\bm{Q}}({\bm{x}}) \in \mathbb{R}^d. \label{eq:VtQ}
\end{align}

Where unlike commonly used recurrent architectures, the NV-RNN concatenates all of the hidden states as the input to the linear classifier. This is different from a typical RNN where the last hidden state is the input to the linear classifier. This concatenation does increase the size of the input but is needed for both interpretability/explainability and the performance in Table~\ref{table:1}.

Figure~\ref{fig:my_label} depicts our NV-RNN where it displays how the hidden states are aggregated in order to provide a classification decision. Note that this network is applied to many-to-one applications.

The use of new weights for the linear classifier at each time step limits the NV-RNN to datasets where all data sequences have the same length. In the case where the data sequences have variable length, then zero padding will be used. This interpretability especially in terms of which time steps are resonating with the class helps explain which time steps are contributing the most. The entries of ${\bm{V}}$ will state in a numeric manner how much each weight is contributing to classification.

\begin{figure}
\centering
\begin{tikzpicture}[object/.style={thin,double,<->}]
    \tikzstyle{main}=[rectangle, minimum width = 10.3mm, minimum height = 7mm, thick, draw =black!80, node distance = 5mm]
    \tikzstyle{main2}=[circle, minimum size = 10mm, thick, draw =black!80, node distance = 5mm]
    \tikzstyle{connect}=[-latex, thick]
    \tikzstyle{box}=[rectangle, draw=black!100]

    \node[main] (old4) [] {\small$f^{(2)}({\bm{x}})$};
    \node[main] (old1) [left=1.3cm of old4] {\small $f^{(1)}({\bm{x}})$};
    \node[main] (old7) [right=1.3cm of old4] {\small $f^{(3)}({\bm{x}})$};
    \node[main] (old5) [below=0.5cm of old4] {\small ${\bm{h}}^{(2)}({\bm{x}})$};
    \node[main] (old2) [below=0.5cm of old1] {\small ${\bm{h}}^{(1)}({\bm{x}})$};
    \node[main] (old8) [below=0.5cm of old7] {\small ${\bm{h}}^{(3)}({\bm{x}})$};
    \node[main] (new1) [right=1.3cm of old7] {\small $f^{(4)}({\bm{x}})$};
    \node[main] (new2) [below=0.5cm of new1] {\small ${\bm{h}}^{(4)}({\bm{x}})$};
    \node[main] (new3) [below=0.5cm of new2] {\small ${\bm{x}}_4$};
    \node[main] (new4) [right=1.3cm of new1] {\small $f^{(5)}({\bm{x}})$};
    \node[main] (new5) [below=0.5cm of new4] {\small ${\bm{h}}^{(5)}({\bm{x}})$};
    \node[main] (new6) [below=0.5cm of new5] {\small ${\bm{x}}_5$};
    \node[main] (xtm) [below=0.5cm of old2] {\small ${\bm{x}}_{1}$};
    \node[main] (xt)  [below=0.5cm of old5] {\small ${\bm{x}}_{2}$};
    \node[main] (xtp) [below=0.5cm of old8] {\small ${\bm{x}}_{3}$};
    \coordinate[right=0.5cm of new4] (c1);
    \coordinate[right=1.2cm of new4] (c2);
    \node[text width=1cm] at (c2) {$f_{\theta}({\bm{x}})$};

    \path (new1) edge node[draw,fill=white,circle]{+} (new4)
        (old1) edge node[draw,fill=white,circle]{+} (old4)
        (old4) edge node[draw,fill=white,circle]{+} (old7)
        (old7) edge node[draw,fill=white,circle]{+} (new1);

  %

  %

    \path
        (old2) edge [connect] node[above] {}(old5)
        (old5) edge [connect] node[above] {}(old8)
		(old5) edge [connect] node[above] {} (old8)
		(old8) edge [connect] node[above] {} (new2)
		(new2) edge [connect] node[above] {} (new5)
		(new4) edge [connect] node[above] {} (c1)
		%
		%

		%
		(old2) edge [connect] node[right] {\small${\bm{V}}^{(1)}$}(old1)
		(old5) edge [connect] node[right] {\small${\bm{V}}^{(2)}$}(old4)
		(old8) edge [connect] node[right] {\small${\bm{V}}^{(3)}$} (old7)
		(new2) edge [connect] node[right] {\small${\bm{V}}^{(4)}$} (new1)
		(new5) edge [connect] node[right] {\small${\bm{V}}^{(5)}$} (new4)
		;
	\node[rectangle, inner sep=1.7mm,draw=black!100, fit= (old1)(old7)(new4), fill=red,opacity=.2](t) {};

    \path (xtm) edge [connect] node[right] {}(old2)
    (xt) edge [connect] node[right] {}(old5)
    (xtp) edge [connect] node[right] {}(old8)
    (new3) edge [connect] node[right] {}(new2)
    (new6) edge [connect] node[right] {}(new5);

\end{tikzpicture}
\caption{Depiction of the NV-RNN framework. Every RNN can be converted to a NV model by having every hidden state concatenated to the linear classifier. The input to the linear classifier is the concatenation of all the hidden states. This provides a mapping to evaluate which time steps are the most relevant to each class. }
\label{fig:my_label}

\end{figure}

\iffalse
\subsection{Computing Hidden State Contribution to Decision-Making}
 The first step in building our interpretable RNN is to concatenate all the hidden states per time step into a large hidden state vector $H$, where
\begin{align*}
   H=
  [h_{1}(x)^T,h_{2}(x)^T,\dots,h_{N}(x)^T]^T\in \mathbb{R}^{\sum_{\ell=1}^{N}D_{\ell}}.
\end{align*}

Figure~\ref{fig:my_label} depicts our NV-RNN where it displays how the hidden states are aggregated in order to provide a classification decision. Note that this network is applied to many-to-one applications.

\subsection{Interpretability of Each Neuron's Contribution to the Inference}

From this perspective, the input to the linear classifier is now a high dimensional vector as
\begin{align}
    \widehat{y}=\mathrm{softmax}(Wq(H)+b),\label{eq:second}
\end{align}
where $q(H)$ groups the concatenation of the hidden states across time that were selected as inputs of the linear classifier (Equation \ref{eq:second}).
This makes interpretability much simpler as now it is enough to see how each hidden state per time is correlated to each class output to affirmatively say how the activation or deactivation of a specific unit impacts all the class predictions.
\fi
\subsection{Interpretable and Explainable Linear Classifier}

Recall that the final input to the linear classifier is the concatenation of the hidden states from all the time steps. When training is complete, we look at the weights per class and since we have the linear relationship of the linear classifier to all the hidden states, we have this ordered mapping of the weights to the hidden states. This ordering is done by concatenating the first hidden state starting from the first time step and concluding with the final time step's hidden state.
 Hence, we inspect which of the hidden states per time are contributing to the decision-making process for every class.
 Recall that from Equation \ref{eq:VtQ} we have NV-RNN denoted in that form. Then we can substitute the right hand side to have in it in this new form (Equation~\ref{eq:vq}). The classification depends on the concatenation of all the hidden states, ${\bm{Q}}$, and the weights from the linear classifier. For each class of the output we get

 \begin{align}
    \left[ f_{\theta}({\bm{x}}) \right]_i={\bm{v}}_i^\top {\bm{Q}}({\bm{x}}) \in \mathbb{R}, \,\,\,\, \forall \,\, i \in [d], \label{eq:class}
\end{align}
where ${\bm{v}}_i \in \mathbb{R}^{nT}$ is the $i$th row of ${\bm{V}}$ that corresponds to class $i$. Since each class output is obtained by inner product of ${\bm{Q}}({\bm{x}})$ and ${\bm{v}}_i$, each class acquires a set of distinct learned ${\bm{v}}_i$ weights and we can interpret their values for each class. The goal is that in the training phase, the weights of ${\bm{V}}$ are generalized well for a test set and we can inspect which time steps are contributing the most by looking at the values. Each row of ${\bm{V}}$ corresponds to a class, so for every input that is sent, the softmax will choose the class with the highest score when each of the class's weights are multiplied by the time steps' hidden state activations.

In addition to interpretability, we provide explainability of the classification. For each class, their ${\bm{v}}$ explain the classification since ${\bm{Q}}({\bm{x}})$ is constant for each class and the determining factor are each class's ${\bm{v}}$ since the highest dot product from Equation~\ref{eq:class} will determine which class is chosen.

This interpretability and explainability can provide more understanding of the training dynamics of NV-RNN. This provides additional analysis to why some models perform better than others. In Figures~\ref{fig:chinatownNV}, \ref{fig:fungiNV}, \ref{fig:chlorineNV}, and \ref{fig:insectNV} we can see which time steps are critical based on their weight values from looking at each class's ${\bm{v}}$.

\subsection{Experimental Setup}
We demonstrate the performance of NV-RNN, NV-GRU and NV-LSTM against other RNN, GRU, and LSTM architectures that are difficult to interpret/explain. The premise is that if we are on par or better than these models it is beneficial to use. The RNN models that are used besides RNN are GRU and LSTM. Also we monitor the effect of average pooling with RNNs, LSTMs, and GRUs in our comparison. The idea to benchmark against average pooling RNNs came from these works, \cite{wang2019comparison,kao2020comparison}, since they have better performance than the standalone RNNs, LSTMS, and GRUs. Datasets that were used are the UCR repository \cite{Dau_2019}, Large Movie Review \cite{maas-EtAl:2011:ACL-HLT2011}, and UCF11 \cite{liu2009recognizing}. Do note that these tasks fall under the many-to-one regime for RNN classification. The UCR datasets had the input of the same length while the Large Movie Review dataset has varying input length. For UCF11, we use a sampler that we sample 50 sequential frames from each video. For the UCR and Large Movie Review datasets, we choose the best accuracy from each model where we vary the hidden state size to 32, 64, and 128. For UCF11, we use the hiddens state size of 32. All of the hyperparameters for the experiments are detailed in the Supplementary Material Section in \ref{section:hyper}.

\begin{table}
\centering
\caption{The best performing NV-RNN, NV-GRU, and NV-LSTM models against the best performing RNN, GRU, and LSTM models. For most of the datasets, the NV-RNN, NV-GRU, and NV-LSTM models outperform especially on datasets that have more time steps.}
{
\begin{adjustbox}{width=\linewidth,center}
\begin{tabular}{|l|ccc|ccc|ccc|}
\toprule
\textbf{Data set}        & \textbf{RNN} & \textbf{GRU} &\textbf{LSTM} & \textbf{RNN-AVG} & \textbf{GRU-AVG} & \textbf{LSTM-AVG} & \textbf{NV-RNN} & \textbf{NV-GRU} & \textbf{NV-LSTM} \\ \hline
Adiac  & 35.8\% & 37.08\% & 49.61\% & 10.23\% & 31.45\% & 16.87\% & 69.56\% & 68.28\% & \textbf{74.68\%} \\
BME & 88.66\% & 94.66\% & 80\% & 84\% & 84.66\% & 84.66\% & \textbf{99.3\%} & 98.66\%  & 98.66\% \\
CBF & 60.66\% & 94.55\% & 90\% & 97.33\% & 98.66\% & \textbf{99.77\%} & 97.77\% & 98.44\% & 98.55\% \\
Chinatown & 74.34\% & 97.37\% & 97.66\% & \textbf{98.83\%} & 98.25\% & 98.54\% & 97.95\% & 97.08\% & 98.54\% \\
Chlorine Concentration & 58.17\% & 60.1\% & 57.73\% & 55.39\% & 57.05\% & 55.88\% & \textbf{83.95\%} & 78.15\% & 72.39\% \\
Fungi & 49.46\% & 58.6\% & 68.81\% & 60.21\% & 58.6\% & 75.26\% & 96.77\% & 98.92\% & \textbf{99.46\%} \\
Ham & 69.52\% & 68.57\% & 69.52\% & 74.28\% & \textbf{81.9\%} & 80.95\% & 78.09\% & 80.95\% & 78.09\% \\
Haptics & 42.2\% & 41.55\% & 41.88\% & 33.76\% & 44.48\% & 41.88\% & \textbf{46.42\%} & 46.1\% & 45.77\% \\
Herring & 67.18\% & 67.18\% & 68.75\% & 67.18\% & 65.62\% & 68.75\% & 68.75\% & \textbf{73.43\%} & 68.75\% \\
Insect Regular & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} \\
Insect Small & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} \\
InsectWingbeat & 28.93\% & 49.34\% & 43.58\% & 28.48\% & 46.41\% & 39.94\% & 64.29\% & 64.54\% & 63.88\% \\
Meat & 48.33\% & 50\% & 50\% & 66.66\% & 86.66\% & 81.66\% & \textbf{96.66\%} & \textbf{96.66\%} & \textbf{96.66\%} \\
OliveOil & 46.66\% & 50\% & 40\% & 40\% & 80\% & 40\% & \textbf{93.33\%} & \textbf{93.33\%} & \textbf{93.33\%} \\
Plane & 89.52\% & 79.04\% & 95.23\% & 65.71\% & 98.09\% & 70.47\% & 99.04\% & \textbf{100\%} & 99.04\% \\
Rock & 64\% & 74\% & 68\% & 56\% & 62\% & 60\% & 80\% & 76\% & \textbf{82\%} \\
SmoothSubspace & 91.33\% & 89.33\% & 90.66\% & 90.66\% & 91.33\% & 86.66\% & 91.33\% & \textbf{96\%} & 94\% \\
Synthetic Control & \textbf{99.66\%} & 98.66\% & 98.33\% & 94.33\% & 95.66\% & 97.33\% & \textbf{99.66\%} & 99.3\% & 99.3\% \\
UMD & 74.3\% & 99.3\% & 86.8\% & 75\% & 92.36\% & 72.22\% & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} \\
Wine & 59.25\% & 59.25\% & 62.96\% & 75.92\% & 79.62\% & 74.07\% & \textbf{100\%} & \textbf{100\%} & \textbf{100\%} \\

\bottomrule
\end{tabular}
\label{table:1}
\end{adjustbox}
}
\label{tab:real-data-svm}
\end{table}

\begin{table}
\centering
\caption{The best performing NV-GRU models against the best performing non NV-GRU models with the Large Movie Review dataset. The dataset has sentences of reviews of variable length. NV-GRU performs close to on par despite zero padding.}
{
\begin{adjustbox}{width=0.5\linewidth,center}
\begin{tabular}{|l|l|c|c|}
\toprule
\textbf{Data set}   & \textbf{Embedding}     & \textbf{GRU} & \textbf{NV-GRU} \\ \hline
Large Movie Review & Word2Vec & \textbf{91.87\%} & 90.12\% \\
Large Movie Review & FastText & \textbf{91.46\%} & 89.56\% \\
Large Movie Review & GloVe & \textbf{89.98\%} & 87.76\% \\

\bottomrule
\end{tabular}
\label{table:2}
\end{adjustbox}
}
\end{table}

\begin{table}
\centering
\caption{The best performing NV-CNN-RNN, NV-CNN-GRU, and NV-CNN-LSTM models against the best performing CNN-RNN, CNN-GRU, and CNN-LSTM models. The dataset is UCF11 where the task is video action recognition. }
{
\begin{adjustbox}{width=0.9\linewidth,center}
\begin{tabular}{|l|c|c|c|c|c|c|}
\toprule
\textbf{Data set}   & \textbf{NV-CNN-GRU}     & \textbf{NV-CNN-RNN} & \textbf{NV-CNN-LSTM} & \textbf{CNN-GRU} & \textbf{CNN-RNN} & \textbf{CNN-LSTM} \\ \hline
UCF11 & 76.4\% & 74.3\% & \textbf{78.3\%} & 69.3\% & 72.1\% & 72.4\% \\

\bottomrule
\end{tabular}
\label{table:3}
\end{adjustbox}
}
\end{table}

\subsection{Results}
We see from Table~\ref{table:1} that for most of the datasets, any of the NV-RNN, NV-GRU, and NV-LSTM models outperform the traditional RNN, GRU, and LSTM and its variants. For the exact accuracies among the different hidden state sizes look into the Supplementary Material Section \ref{section:ablation}. It is interesting to note that with the datasets that have a smaller amount of timesteps, the RNNs will perform better compared to their gating variants. In addition, with the Adiac dataset, the results show that using average pooling on the hidden states is not the optimal solution. This is why NV-RNN was developed to have the linear classifier learn which of the time steps should have higher positive or negative weight values for the different classes. From Table~\ref{table:2} we apply NV-GRU to a dataset where the input has varying length and we perform on par. The reason is that the maximum length we set is 1000 and any reviews under that maximum length will have zero padding. From Table~\ref{table:3}, we outperform on another application called video action recognition. Note, that in this application, we have to use CNN filter units and RNN hidden state units as input to the linear classifier.

With the great performance, there is a cost associated with it. The cost is that the input for the linear classifier greatly increases. The input of the linear classifier size greatly increases since we concatenate all the hidden states from the input. This is in stark contrast from the input being the last hidden state or the averaging of the hidden states. Yet, even with the additional memory overhead, we can still run these NV-RNN, NV-GRU, and NV-LSTM models with an 8 GB GPU.

\section{NV-RNN Time Analysis}

\subsection{Interpreting the Time Steps}

Now with the experimental results showing that the NV-RNN, NV-GRU, and NV-LSTM models outperform RNNs, GRUs, and LSTMs among multiple datasets, the next step is to explain which time steps are pivotal for the classification portion. This is something that is lacking with the models that we compared. The weights from the linear classifier have a linear mapping between the different hidden states per time in order to directly observe which hidden states are responsible for each class's decision. Each class will have its own distribution of linear classifier weights. In the following section, we will observe different classes' linear weights for different datasets, different hidden state and weight initializations, and other case studies.

This interpretation is lacking from RNNs and their variants since the input to the linear classifier is the last hidden state. Then for the RNNs with average pooling, the notion of averaging the hidden states does not produce the best results as shown. Hence, by having all the hidden states as the input for the linear classifier, we learn the weights for each class to prioritize all the time steps. Each class will have a set of learnable weights that can be different from other classes. Figures~\ref{fig:chinatownNV}, \ref{fig:fungiNV}, and \ref{fig:chlorineNV} show the weights from different NV-RNN models for different classes.

One dataset from Table~\ref{table:1} is the Chinatown dataset that has 2 classes and the number of time steps is 24. Figure~\ref{fig:chinatownNV} shows the NV-GRU weights for both classes. The weights for each class are drastically different from each other. This notion makes sense because for binary classification, the objective would be to have the weights drastically differ. The positive weights for class 0 become the negative weights for class 1. Figure~\ref{fig:chinatownNVHiddenStates} shows the individual hidden state weights for more granular information. Since this dataset only has 24 time steps, it is easy to view the individual weights for every hidden state as opposed to other datasets. With the input size being one, there is not a lot of activity with the hidden states for every time step. It is interesting that for every time step there seems to be a gradual change when looking at the nearby hidden states at neighboring time steps.

Another dataset from Table~\ref{fig:fungiNV} is the Fungi dataset that has 18 classes and the number of time steps is 201. This dataset is very different from the Chinatown dataset, since we have multiple classes. Hence, one idea is to assess if there are weights from one class that is similar to another class and if there are weights from one class that is dissimilar to another class. To do this, we acquire the weights from each class and calculate the cosine similarity between all the classes. The cosine similarity is defined as $\frac{w_1^\top w_2} {||w_1||_2||w_2||_2}$ where $w_1$ is the weights of one class and $w_2$ is the weights of another class. In this scenario, class 5 is similar to class 16 while dissimilar to class 9. Figure~\ref{fig:fungiNV} shows the weights for each of these classes. Based on the cosine similarity, it is easier to perceive why class 5 and class 16 are similar. Plus, it is easy to identify that class 5 and class 9 are dissimilar in their weights. Hence, if there is a prediction that was meant for class 5 but went to class 9, this notion makes sense by looking at the weights.

We show different weights from different classes of the same dataset, but now we want to assess if NV-RNN, NV-GRU, and NV-LSTM prioritize on different time steps. This is done by using the Chlorine Concentration dataset, which has 3 classes and 166 time steps. From Table~\ref{table:1}, each of these NV models had different test accuracies. Figure~\ref{fig:chlorineNV} shows the different weights from these NV models for class 0. From Figure~\ref{fig:chlorineNV}, all of these NV models have a similar positive trend for the last few time steps. However, the middle time steps are where each of these NV models starts to differ in prioritizing certain time steps. Hence,  a benefit of the NV-RNN, NV-GRU, and NV-LSTM models is that you can interpret which time steps are critical based on the weight value.

We use three datasets on how to interpret the NV-RNN models after they are trained. In addition, they can also explain their predictions since for each class, the weight values are provided and linked to all of the hidden states. Hence, with each NV-RNN model there is a formal manner to explain the predictions since we can acquire the hidden state values for each time step and the weights for each class.

\begin{figure}[h!]%
\centering
\includegraphics[width=.45\textwidth]{images/ChinatownNVGRUClass0.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/ChinatownNVGRUClass1.pdf}\hfill
    \caption{(Left) NV-GRU weights for class 0. (Right) NV-GRU weights for class 1. For the Chinatown dataset, there are two classes and the weights for each class are drastically different. Hence each class's prediction focuses on different time steps. In regard to class 0, the most important time steps are 4, 5, 6, and 7. In regard to class 1, the most important time steps are 0, 1, 2, and 3.}%
    \label{fig:chinatownNV}%
\end{figure}

\begin{figure}[h!]%
\centering
\includegraphics[width=.45\textwidth]{images/NVGRUChinatownHiddenStateClass0.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/NVGRUChinatownHiddenStateClass1.pdf}\hfill
    \caption{(Left) NV-GRU hidden state weights for class 0. (Right) NV-GRU hidden state weights for class 1. For the Chinatown dataset, it has 24 time steps and the first hidden state starts at the top left and going from left to right where the last hidden state is in the bottom right.}%
    \label{fig:chinatownNVHiddenStates}%
\end{figure}

\begin{figure}[h!]%
\centering
\includegraphics[width=.32\textwidth]{images/FungiNVGRUClass5.pdf}\hfill
\includegraphics[width=.32\textwidth]{images/FungiNVGRUClass9.pdf}\hfill
\includegraphics[width=.32\textwidth]{images/FungiNVGRUClass16.pdf}
    \caption{(Left) NV-GRU weights for class 5. (Middle) NV-GRU weights for class 9. (Right) NV-GRU weights for class 16. For the Fungi dataset, there are a total of 18 classes. Compared to class 5's weights, class 9's weights are one of the most different based on the cosine similarity of the weights for each of these classes. For class 16, it is one of the most similar classes to class 5 based on the cosine similarity of the classes. Inspecting the NV-GRU weights from the linear classifier can aid in interpreting which classes are similar or dissimilar to each other.}%
    \label{fig:fungiNV}%
\end{figure}

\begin{figure}[h!]%
\centering
\includegraphics[width=.32\textwidth]{images/ChlorineNVGRUClass0.pdf}\hfill
\includegraphics[width=.32\textwidth]{images/ChlorineNVRNNClass0.pdf}\hfill
\includegraphics[width=.32\textwidth]{images/ChlorineNVLSTMClass0.pdf}
    \caption{(Left) NV-GRU weights for class 0. (Middle) NV-RNN weights for class 0. (Right) NV-LSTM weights for class 0. From Table~\ref{table:1}, each of the NV models had different test accuracies and by inspecting each of the models, the prioritization of the time steps are different. The last time steps seem to have a similar trend for all of the NV models but the middle time steps have different prioritizations.}%
    \label{fig:chlorineNV}%
\end{figure}

\subsection{Case Studies}
We provide different case studies on how to use NV-RNN to inspect which time steps are prioritized within the application. With this insight, we observe how it provides additional understanding from the application.

\textbf{Different Weight Initializations}
RNNs can be difficult to train. From \cite{pascanu2013difficulty}, they state that early in the development of RNNs they would experience the vanishing gradient or exploding gradient problems. This led to the advent of GRUs and LSTMs. Even with the new gating architectures of GRUs and LSTMs, they can experience issues of learning. This discovery leads others to figure out how to improve the performance. Others such as \cite{le2015simple,talathi2015improving} found that initializing the ${\bm{W}}$ weight matrix of the RNN can help in performance. Hence, we can use NV-RNN to inspect what is happening with the time step prioritization in regards to classification.

On the InsectWingbeat dataset, we perform an experiment to vary the weight initialization on the hidden-to-hidden matrix. This matrix is different from the ${\bm{V}}$ that is the linear classifier. We use the NV-GRU network to inspect how that can affect the decision-making process. The three different weight initializations are orthogonal, identity, and normal distributed. Figure~\ref{fig:weightsInit} shows the three different weight initialization schemes and you can notice that the weight initialization scheme using a normal distribution is focusing on different time steps compared to the same NV-GRU model but with different weight initializations.

The two weight initializations, orthogonal and identity, look very similar in terms of time step prioritization with Figure~\ref{fig:insectNV}. In Figure~\ref{fig:insectNV}, it is the default weight initialization for NV-RNN, NV-GRU, and NV-LSTM, which is uniform initialization. Thus it is interesting that even with two different weight initializations, those weight initializations look similar to each other. Yet, the normal initialization is vastly different in regard to the time step prioritization. Plus with the NV-RNN model, it can explain why the performance is bad by looking at the time step weight prioritization. Hence, even if the NV-RNN model is performing in a sub-optimal manner, we can inspect why it is performing in that manner. Using a traditional RNN, GRU, or LSTM cannot provide this information.

\begin{figure}[h!]

\centering
\includegraphics[width=.3\textwidth]{images/IWBNVGRUClass2OrthogonalInit.pdf}\hfill
\includegraphics[width=.3\textwidth]{images/IWBNVGRUClass2IdentityInit.pdf}\hfill
\includegraphics[width=.3\textwidth]{images/IWBNVGRUClass2NormalInit.pdf}

\caption{(Left) Orthogonal. (Middle) Identity. (Right) Normal. With the same NV-GRU model but having 3 different weight initializations for ${\bm{W}}$ of the NV-GRU network. With different weight initializations, the distribution of the weights mapping to the time steps is shown to be different. Hence weight initialization is important because even with similar performance, by looking inside of the linear weights can provide insight into what the network is deciding for classification in regards to class 2.}
\label{fig:weightsInit}

\end{figure}

\begin{figure}[h!]%
    \centering
\includegraphics[width=.3\textwidth]{images/IWBNVGRUClass2.pdf}\hfill
\includegraphics[width=.3\textwidth]{images/IWBNVRNNClass2.pdf}\hfill
\includegraphics[width=.3\textwidth]{images/IWBNVLSTMClass2.pdf}
    \caption{(Left) NV-GRU weights across time. (Middle) NV-RNN weights across time. (Right) NV-LSTM weights across time. All three different NV-RNN models display the weights of time to show for this class which time steps are the most significant. NV-GRU and NV-LSTM seem to behave in a similar manner while NV-RNN has a different way of prioritizing the time steps.}%
    \label{fig:insectNV}%

\end{figure}

\textbf{Increasing the Depth (More Layers)}
\cite{pascanu2013construct} was one of the first works to take RNNs and make them deeper by stacking RNNs to increase the depth. In \cite{pascanu2013construct}, unfortunately in their work, they did not show any interpretation towards how each depth aids in classification. There was one work \cite{goh2017smiles2vec} that used a deeper GRU for a chemical task. They provide interpretability through a mask but unfortunately it lacks sufficient detail in how each RNN in the deep RNN is aiding in classification. This is where NV-RNN can aid and show how each RNN (layer) is prioritizing the time steps in terms of classification.

From Table~\ref{table:1}, we focused on one layer RNNs and NV-RNNs, but now we want to assess how increasing the depth of an NV-GRU model will show which time steps are prioritized. The question to inspect is if for every layer, will the time step prioritization be similar to the previous layers? Also, at a certain depth, will the time step prioritization be different from the previous layers? We can do this by looking at the weights of a NV-RNN model. In this analysis, we used the Rock dataset which has 2844 time steps and 4 classes. We have 4 different NV-GRU models where each one has varying depth. The first one starts at a depth of one and each additional model increases by an additional depth. Hence the last NV-GRU model has a depth of 4.

When comparing the test accuracy of the different depths of NV-GRU to GRU and average pooling GRU, the NV-GRU models would outperform the GRU and average pooling GRU. When testing the performance among different hidden states like 32, 64, and 128, NV-GRU would have a test performance of \textbf{76\%}, \textbf{76\%}, and \textbf{76\%} for the depths of 2,3,4. This is the same performance as noted for a one-layer NV-GRU for the same dataset. For the GRU and average pooling GRU, the best test performance for depths of 2, 3, and 4 were 68\%, 72\%, and 66\%. Thus, for different depths, NV-GRU is outperforming.

Figure~\ref{fig:rockDepthNV} shows the time step weights for four different NV-GRU models with different depths. With NV-GRU, we can answer that question in a quantitative manner if the time step prioritization is similar among the different layers of the NV-GRU model. The answer is no and this makes sense since the input to the next layer is the previous layer's representation. For some time steps, there is some manner of consistency like with the NV-GRU models of 2, 3, or 4 layers where the beginning time steps are close to zero. Then the time step prioritization will vary towards the end of the time steps.

\textbf{Bidirectional RNNs} There are some RNNs that were developed that are bidirectional such that the RNN will learn in two directions. One direction is in a causal manner (forward) going from the beginning of the input to the end of it. While the other direction is in a non-causal manner (reverse) where it begins at the end of the input and ends at the beginning of the input. Equation~\ref{eq:bidirectional} details how an RNN will use both the forward and reverse hidden states. Thus the aspect for NV-RNN is to adapt it to the bidirectional variant and to inspect the weights. By inspecting the weights, we can assess how the time steps are prioritized in either direction. One question to answer is if the time steps of either direction will be symmetrical to each other?

There are works from \cite{ma2017dipole, schwab2017beat} that have used bidirectional RNNs to aid performance in their respective task. For interpretability, they use the attention weights to show how a given input is being classified. Yet in their interpretability, they cannot provide how each direction in the RNN is aiding for the classification. This lack of directional interpretability is where NV-RNN will help and can answer the questions mentioned above.

For the bidirectional analysis, we use an NV-GRU model that allows the bidirectional nature. We use the Rock dataset which has 2844 time steps and 4 classes.

When comparing a bidirectional NV-GRU with a bidirectional GRU and average pooling GRU, the bidirectional NV-GRU outperformed in test accuracy. Among the different hidden state sizes of 32, 64, and 128, the best test accuracy for bidirectional NV-GRU is \textbf{76\%} while for GRU is 72\% and average pooling GRU is 66\%. Even for the bidirectional variant, NV-GRU is outperforming.

Figure~\ref{fig:rockBiDirNV} shows the time step prioritizations for the bidirectional NV-GRU. With this NV-GRU model, we can assess if the time steps prioritized from both directions will be symmetrical. The answer is that there is evidence that we do not see this notion and we can see that in a visual and quantitative manner (Figure~\ref{fig:rockBiDirNV}). Looking at the vertical columns for each direction, each direction does not have the same prioritization.

\begin{figure}[h!]%
\centering
\includegraphics[width=.45\textwidth]{images/RockNVGRU1Layer_64.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/RockNVGRU2Layer_64.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/RockNVGRU3Layer_64.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/RockNVGRU4Layer_64.pdf}\hfill
    \caption{(Top Left) NV-GRU weights for class 0. (Top Right) NV-GRU (2 Layers) weights for class 0. (Bottom Left) NV-GRU (3 Layers) weights for class 0. (Bottom Right) NV-GRU (4 Layers) weights for class 0. By increasing the number of layers for the NV-GRU model, the prioritization of time steps is not consistently the same among all the layers. Each vertical column is the weight value at that time step.}%
    \label{fig:rockDepthNV}%
\end{figure}

\begin{figure}[h!]%
\centering
\includegraphics[width=.45\textwidth]{images/RockNVGRU_32.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/RockBidirectionalClass0.pdf}\hfill
    \caption{(Left) NV-GRU weights for class 0. (Right) Bidirectional NV-GRU weights for class 0. The forward and reverse prioritizations of time steps looks very similar. Each vertical column is the weight value at that time step.}%
    \label{fig:rockBiDirNV}%
\end{figure}

\textbf{Sentiment Analysis}
The previous datasets that were shown consisted of time-series data. Now we will use a movie review dataset \cite{maas-EtAl:2011:ACL-HLT2011} with three embedding techniques, Word2Vec \cite{mikolov2013efficient}, Fasttext \cite{bojanowski2017enriching}, and GloVe \cite{pennington2014glove}, to convert each word into an embedding vector. Since NV-RNN is versatile, we can inspect how the weights for each class are prioritized for text data. In this dataset, the task is sentiment classification where one class is positive sentiment and the second class is negative sentiment.

Other works have provided different forms of understanding the differences of these embedding techniques. \cite{shi2014linking} show the similarities of GloVe and a skip-gram Word2Vec in a mathematical manner. \cite{nematzadeh2017evaluating} use a couple of evaluation tasks like correlation to show the differences or similarities of these embedding techniques. \cite{shin2018interpreting} uses eigenvector analysis to compare the embedding vectors to assess how the words cluster together. Yet, all of these techniques cannot link the time steps to the class, which is what NV-RNN will show.

Table~\ref{table:2} shows the performance and in this scenario, NV-GRU performs on par. Note that for this dataset, there are reviews of variable length so for a GRU the last hidden state per review is the input to the linear classifier. However, for NV-GRU, padding has to be applied since the input for the linear classifier has to be of fixed size. Hence, it does explain the small drop in performance. As for padding, we only used the first 1,000 words of the review. With reviews that are smaller than 1,000 words, zero padding would be applied. Hence, there can be a good parameter to pad the reviews. Even with this disadvantage, NV-GRU is still on par and only loses one to two percent of test accuracy.

Figures~\ref{fig:movieW2VNV}, \ref{fig:movieFTNV}, and \ref{fig:moviegloveNV} display the weights for each class. Even though there are three different embedding techniques, Word2Vec, FastText, and GloVe, it seems that the prioritization for the time steps in each embedding looks to be quite similar in the broad general sense. For the negative sentiment class, it is interesting how with all the embedding inputs, the last time steps are negative. This does make sense since FastText is a continuation of Word2Vec except for the case that FastText will approximate words that are not in the dictionary of words that it had learned. It is interesting how the GloVe embedding looks to have similar broad time step prioritization since it is a matrix factorization technique compared to Word2Vec and FastText. There are very specific differences based on the magnitudes but in a general manner all the embedding techniques for the input will result in the same general trend for each class in terms of the weights.

From Figures~\ref{fig:movieW2VNV} and \ref{fig:movieFTNV}, we provide another experiment to see which words are tied to the highest weights based on the time step index. To reduce the amount of reviews, we only look at the top 5 reviews for each class based on the pre-softmax score. In addition, we look at the top 10 weights per class. When inspecting the words for each class, we find pronouns and prepositions. However, one particular notion we inspect is particular words for each class. For the negative sentiment class, we observe the words, \textbf{bad}, \textbf{dodgy}, \textbf{unpleasant}, and \textbf{pointless} from the time steps with the highest weights. For the positive sentiment class, we observe the words, \textbf{mature}, \textbf{happy}, \textbf{perfect}, and \textbf{top}. We would hope to observe this notion because if the class is centered around negative sentiments, then the words that should be linked to the class would be bad words. The same notion applies to the positive sentiment class. Also, we did observe that for the positive sentiment class, we did not notice any negative sentiment words like bad or terrible.

\begin{figure}[h!]%
\centering
\includegraphics[width=.45\textwidth]{images/MovieNVGRUClass0W2VHeatmap1.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/MovieNVGRUClass1W2VHeatmap1.pdf}\hfill
    \caption{(Left) NV-GRU weights for class 0. (Right) NV-GRU weights for class 1. For the Movie Review dataset, there are two classes and the weights for each class are drastically different. In addition, the input is the Word2Vec embedding of each word per time step. Compared to continuous time-series data, the prioritization of time steps is not as smooth compared to datasets with continuous data.}%
    \label{fig:movieW2VNV}%
\end{figure}

\begin{figure}[h!]%
\centering
\includegraphics[width=.45\textwidth]{images/MovieNVGRUClass0FTHeatmap1.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/MovieNVGRUClass1FTHeatmap1.pdf}\hfill
    \caption{(Left) NV-GRU weights for class 0. (Right) NV-GRU weights for class 1. For the Movie Review dataset, there are two classes and the weights for each class are drastically different. In addition, the input is the FastText embedding of each word per time step. Compared to continuous time-series data, the prioritization of time steps is not as smooth compared to datasets with continuous data.}%
    \label{fig:movieFTNV}%
\end{figure}

\begin{figure}[h!]%
\centering
\includegraphics[width=.45\textwidth]{images/MovieNVGRUClass0GloveHeatmap1.pdf}\hfill
\includegraphics[width=.45\textwidth]{images/MovieNVGRUClass1GloveHeatmap1.pdf}\hfill
    \caption{(Left) NV-GRU weights for class 0. (Right) NV-GRU weights for class 1. For the Movie Review dataset, there are two classes and the weights for each class are drastically different. In addition, the input is the GloVe embedding of each word per time step. Compared to continuous time-series data, the prioritization of time steps is not as smooth compared to datasets with continuous data.}%
    \label{fig:moviegloveNV}%
\end{figure}

\textbf{Video Action Recognition}
In this application, the input to the network is a collection of images from a video. The task is to predict which of the actions is presented in the video where the dataset is UCF11 \cite{liu2009recognizing}. In Table~\ref{table:3}, we show that the NV models are outperforming and we inspect the NV-CNN-GRU model. Note that in this scenario we are concatenating both the CNN filter units and the GRU hidden states.

There have been works such as \cite{dong2017improving,meng2019interpretable} to provide interpretability with video action recognition with CNN-LSTM models. Yet, one of the main issues is that they cannot explain which part, the CNN or LSTM is contributing to the classification. This is where NV-CNN-GRU is able to provide both interpretability and explainability.

In Figure~\ref{fig:ucf11Class2}, we see that the mean positive weights are towards the CNN filter units as opposed to the GRU hidden states. This is interesting to notice that most of the hidden state time step mean weights are negative. This does make sense since the model is sequential and the NV-GRU is depending on the CNN's features. In addition, the dataset sampler is sampling 50 sequential frames and most of the videos contain more than 50 frames. Thus, the impression is that the hidden states may not be providing as much useful information as the CNN. From this work, \cite{li2018resound}, there is evidence that video action recognition datasets tend to have visual bias so it does make sense that with the NV-CNN-GRU model, it has more positive weights towards the spatial information. Given this information, there could be future work to mitigate the CNN/GRU prioritization while retaining the accuracy. Ideas from this work, \cite{nam2020learning}, could aid in mitigating the prioritization.

\begin{figure}[h!]

\centering
\includegraphics[width=.4\textwidth]{images/UCF11Class5FirstLayerNVGRU.pdf}\hfill
\includegraphics[width=.4\textwidth]{images/UCF11Class5SecondLayerNVGRU.pdf}\hfill
\includegraphics[width=.4\textwidth]{images/UCF11Class5ThirdLayerNVGRU.pdf} \hfill
\includegraphics[width=.4\textwidth]{images/UCF11Class5TimeStepsNVGRU.pdf}

\caption{Time step prioritization and filter unit prioritization (Class 5) for the NV-CNN-GRU model for UCF11. For this model, the positive mean weights are for the first layer of the CNN portion. Here the time step mean weights are negative.}
\label{fig:ucf11Class2}

\end{figure}

\begin{table}[h!]
    \centering
    \caption{Performance of the model with Time Analysis. From NV-GRU we inspect which time steps contribute  to each class by their weight value. In the Chinatown dataset, we remove the top weights for each class and calculate the overall class accuracy. Immediately, the test accuracy drops but then will come back up.}
    \begin{tabular}{|c|c|c|} \hline
       Network  & \# of Time Steps & Test Accuracy  \\ \hline
       NV-GRU  & 0 & 96.4 \\ \hline
       NV-GRU  & 1 & 72.7 \\ \hline
       NV-GRU  & 5 & 43.87 \\ \hline
       NV-GRU  & 10 & 89.5 \\ \hline
    \end{tabular}

    \label{tab:testAccuracy}
\end{table}

\textbf{Counterfactuals}
There have been works \cite{chen2021human,poulos2021rnn,bica2020estimating,tonekaboni2019explaining} in using counterfactuals to understand how the RNN is performing. \cite{chen2021human,poulos2021rnn,bica2020estimating} use the counterfactuals for regression applications with RNNs. While \cite{tonekaboni2019explaining} uses the counterfactuals for classification applications with RNNs.

From the interpretability and explainability that NV-RNN provides, we can create counterfactuals called \textit{Time Analysis}. With each class having a unique set of weights per time step, we look at the mean hidden state weights per time step and set the top \textbf{K} time steps to zero. The idea is that the performance should drop if the information from the time-series data related to the time step is omitted. it will make it harder for the classifier to perform adequately. We set the top \textbf{K} time steps to zero to evaluate the degradation of the per-class accuracy.

For Time Analysis, we utilize the information we learned from Figure~\ref{fig:chinatownNV}, which shows that for each class there were about 5 top positive time steps. In Table~\ref{tab:testAccuracy}, this confirms that if we set those time steps to 0, then the class accuracy will drop. The interesting aspect is that after we remove more time steps, the class accuracy will increase but never get to the level of the original test accuracy. Note that this accuracy is for one of the NV-GRU models that did not achieve the best accuracy listed in Table~\ref{table:1}. The reason for this dip is that we are now eliminating the negative time steps so it will affect the classification decision. Additional experiments are in the Supplementary Material Section in \ref{counterfactualsSM} where there are individual class performance results.

\section{Conclusion}
 We propose a novel model, NV-RNN, as an alternative to traditional RNNs that has superior to on par performance to RNNs under multiple datasets. NV-RNN can provide interpretability and explain the prediction in a mathematical formulation. NV-RNN has the potential to be used in a wide array of different RNN applications. With its generic framework, it is used to show the connection between all of the hidden states and the classification. Thus, there are other scenarios within the scope of RNNs where it can provide additional understanding where typical RNNs are unable to provide. Overall, this type of interpretability and explainability is helpful to understand what is happening. In addition, with the concatenation of all the hidden states, it enables us to understand more and in most cases have higher performance.

\begin{acks}

This work was supported by NSF grants CCF-1911094, IIS-1838177, and IIS-1730574; ONR grants N00014-18-12571, N00014-20-1-2534, and MURI N00014-20-1-2787; AFOSR grant FA9550-22-1-0060; and a Vannevar Bush Faculty Fellowship, ONR grant N00014-18-1-2047. We would like to thank Yehuda Dar, Hamid Javadi, Vishwanath Saragadam, and Fernando Gama for providing great comments and suggestions for this article.
\end{acks}

\bibliographystyle{ACM-Reference-Format}

redpajama_set_name: RedPajamaArXiv


text:
Women lead
Fundraising campaign to uplift women of ukraine
The women of Ukraine are overcoming extraordinary challenges every day. We are raising €10 million to support female leaders, activists and entrepreneurs. Donate now to join our campaign and make a difference in their lives.
Women Lead is inspired by a documentary about the struggle of women in Ukraine in the middle of the ongoing Russian invasion.
Oh, Sister! was co-commissioned by United for Ukraine with the Nobel Women Initiative and other partners. It tells stories of the perseverance, dedication and bravery of Ukrainian women.
Women Lead is raising €10 million to support women in Ukraine on their path to opportunity, recovery and prosperity. Our female-led initiatives focus on:
Entrepreneurship and education
Culture and social projects
Increasing access to financing
Your donation creates a brighter future for the women of Ukraine by supporting their growth as entrepreneurs, activists and creatives. Each donation empowers us to do more good.
C/O MJP Partners
AG Bahnhofstrasse 20 6300 Zug,
We provide emergency relief,
and humanitarian assistance
UFU PLATFORM
hello@ufu.global
United for Ukraine collects personal data solely for the purposes of providing and facilitating access to humanitarian assistance.
What Personal Data
We process the following personal data:
Personal data of individuals seeking humanitarian assistance through our website, social media or otherwise for the purpose of facilitating access to such humanitarian assistance (including name, phone number; email address).
Personal data of other individuals and organisations in the course of facilitating access to humanitarian assistance.
Personal data of individuals and organisations offering their assistance to or seeking information about United for Ukraine or its members.
The data controller within the meaning of the GDPR or other applicable data protection laws is:
United for Ukraine Association c/o MJP Partners AG Bahnhofstrasse 20 6300 Zug, Switzerland
Sharing of Personal Data
Where you have made a request for assistance through any of our channels, your personal data will be shared with third party individuals and organisations exclusively for the purposes of facilitating such assistance. Your personal data will be deleted once we have finished handling your request. The legal basis for the processing of personal data in relation to your request is Art. 6(1) lit. b GDPR. If you get in touch with us for other purposes (e.g. offering assistance, co-operation or simply seeking information about us), the same applies.
United for Ukraine Legal Support Tool
The data you provide through our United for Ukraine Legal Support Tool (e.g. your name and email address) will be processed solely for the purpose of dealing with your request and will be deleted thereafter. Your data will be shared with those volunteers taking on your request. The legal basis for the processing of personal data in relation to our United for Ukraine Connection Tool is generally Art. 6(1) lit. b GDPR.

redpajama_set_name: RedPajamaCommonCrawl


text:
Politics and the Political: Notes On the Thought of Jean-Luc Nancy
by Illan rua Wall • 20 February 2013
In 2003 Nancy gave a brief, basic philosophical radio talk in which he discussed the question of politics and the political. Reprising his early work with Philippe Lacoue-Labarthe at the Centre de Recherches Philosophiques sur la Politique (the Centre for Philosophical Research on the Political), he explained that excessive use is often made of the term 'political'. When we claim that everything is political, politics loses its specificity. It becomes 'totalitarian' in the sense that 'the horizon of thought is that of a 'political' absorption of every sphere of existence' (Nancy 2008, 25). In the face of such a subsumption, Nancy suggests the analytical move of differentiating le politique (the political) from la politique (politics). Where politics signifies the everyday to- and fro- of the representative political arena, the political is that which is 'most political' in politics. '"The political" seems to present the nobility of the thing – which thereby implicitly regains its specificity, and thus its relative separation' (Nancy 2008, 27).
The distinction between politics and the political was popularized in the late seventies by Claude Lefort who saw the political as the manner in which society was produced as a unity through the now empty place of the King. Politics on the other hand was the interplay of conflicting powers within this unity. He suggested that in democracy, the political was the (empty) symbolic space of authority. In the absence of a king, legitimacy remained always in question. Thus, the political signified the space for the contestation of the very basis of power. When Nancy and Lacoue-Labarthe set up the Centre de Recherches Philosophiques sur la Politique in 1982, they envisaged it as a space for 'the philosophical questioning of the political' and 'the questioning of the philosophical about the political' (Nancy and Lacoue-Labarthe 1997, 108). They claimed that it was important to take such an approach, because the political had withdrawn from politics – it had retreated. Thus, traditional political theory and political science were incapable of thinking the political because they simply took politics as their object. In this sense, Nancy marked both a consonance and dissonance with Lefort's thinking: he suggests that the political 'designate[s] not the organization of society but the disposition of community as such' (Nancy 1991, 40). However, he also travels a more philosophical path, demanding that the political is the essence of politics.
Nancy and Lacoue-Labarthe diagnose what they call the 'retreat [retrait] of the political'. This is the way that
the question of the political, that is the question as to its exact nature or essence, retires or withdraws into a kind of evidence or self-givenness, in which that which is political in politics is taken for granted or accorded a kind of obviousness which is universally accepted. (James 2005, 336)
Our epoch is no longer concerned with the nature of the political, rather such a question is treated as already 'given'. Politics in neo-liberalism, for instance, is presupposed as that which happens after and in the wake of the economy and is ultimately determined by the economy. In a classic deconstructive move, Nancy and Lacoue-Labarthe play with the term 'retreat', insisting that the retreat of the political from politics, should allow us to open new paths of thinking by 're-treating' or re-tracing the political. This can be thought through a philosophical questioning that withdraws from politics in order to approach the question of the political.
In Being Singular Plural, Nancy delves further into this question, explaining two modes of the withdrawal of the political. Firstly, politics collapses into law. Human rights law appears to always already give easy answers to the question of the political. In other words, the human rights of international law begins to subsume politics with an insistence of an all encompassing juridical framework. This critique will be familiar to readers of Agamben or Foucault. However, Nancy insists that the other side of this withdrawal of politics into law is the manner in which 'the formal abstraction of the law, which undoubtedly 'does right' by every participatory and every relation, but without giving this right any meaning other than itself' (Nancy 2000, 47). In this sense, law becomes a cipher for 'the reality of the relation of forces – whether economic technical or the forces of passion'. (Nancy 2000, 47).
Alongside this withdrawal of politics into law, there is the second limb of the withdrawal of the political: what the situationists called the society of the spectacle. In this the political withdraws into 'a self representation that no longer refers to an origin, but only to the void of it's own specularity' (Nancy 2000, 47). Nancy here repeats the situationist critique of late capitalist society, but with a crucial difference. In the society of the spectacle, representation 'triumphs, absorbing entirely both the transcendental and the concrete' (Nancy 2000, 49). However, because the spectacle is all consuming, it cannot help but move within representation itself:
The denunciation of mere appearance effortlessly moves within mere appearance, because it has no other way of designating what is proper – that is, nonappearance – except as the obscure opposite of the spectacle. Since the spectacle occupies all of space, its opposite can only make itself know as the inappropriable secret of an originary property hidden beneath appearances. This is why the opposite of deceitful 'imagery' is creative 'imagination', the model for which is still something like the Romantic genius. (Nancy 2000, 51-2)
Nancy tells us that the Situationist critique comes very close to understanding a 'society exposed to itself, establishing its being social under no horizon other than itself' (Nancy 2000, 52). Yet it places such an insight back into the most traditional of metaphysical constructs, insisting upon the distinction between the false reign of appearance and some authentic presence beyond it. While he disagrees with this formulation, Nancy nevertheless suggests that mediatization forms a part of the retreat of the political which has to be retreated. It remains clear that despite their collapse back into the metaphysics of appearance, he sees the Situationism as opening certain paths of critique.
—James, I (2005) 'On Interrupted Myth,' 9.4 Journal for Cultural Research, p.331
—Nancy, J-L (1991) The Inoperative Community (Minneapolis: University of Minnesota Press)
—Nancy, J-L (2000) Being Singular Plural, (Minneapolis: University of Minnesota Press)
—Nancy, J-L (2008) Philosophical Chronicles, (New York: Fordham University Press)
—Nancy, J-L and Lacoue-Labarthes, P (1997) Retreating the political, (Abingdon: Routledge)
Eco-Technics: Notes on the Thought of Jean-Luc Nancy
Jurisfiction: Notes on the Thought of Jean-Luc Nancy
Right & Rights: Notes on the Thought of Jean-Luc Nancy
Illan rua Wall
Associate Professor of Law at Warwick Law School, University of Warwick. View all posts by Illan rua Wall →
Tags:Jean-Luc Nancy Politics The Political
← Blockupy Frankfurt 2013 – appeal for action this spring
Dehumanisation and the Systemic Perpetuation of Rape →
3 comments for "Politics and the Political: Notes On the Thought of Jean-Luc Nancy"
Pingback: germanwings
This is a wonderfully concise post. In your view, does the political turn in political theory really seem to be building momentum? My brief lit. review suggests that it is. That said we can go all the way back to the 80s and see similar intimations, but the probing conceptual and morphological rigor seems more enhanced now (apart from the likes of Lefort and Nancy, of course). Would you attribute the recent growth to the proliferation of Schmitt's work, or something else? Also, if you have any top recs or decent literature surveys I'd gladly appreciate it, as I'm completing my quals soon.
Pingback: Politics and its Manifestations: Role of Social Networking Sites in India's Post-Modern Society – The Budding PolicyMaker


text:
Bannister Lake to Be Represented by US-Based Sales Organization, Media Technology Group
By InGear PR
MTG Complements Roster of Innovative Broadcast Products With the Addition of Chameleon Real-Time Data Aggregation and Management Solution
CAMBRIDGE, Ontario — May 21, 2020 — Bannister Lake announced today that the company has entered into a nonexclusive representation agreement with Akron, Ohio-based broadcast sales organization Media Technology Group (MTG). MTG will showcase Bannister Lake's Chameleon product to the broadcast industry and participate in co-marketing ventures.
Chameleon is an industry-leading software solution used to manage real-time data and populate graphic templates for news tickers, election results, financial data, wagering, and other applications. The same product also drives on-air branding by leveraging real-time data to power snipes, bugs, promos, and "coming up next" boards. Chameleon's RESTful API allows data to be reformatted and strategically distributed to broadcast graphic engines, HTML5, online, and mobile applications.
"We are thrilled to be working with Bannister Lake," said Rich Hajdu, president of Media Technology Group. "We recognize the broadcast and media industries' demand for real-time content. Bannister Lake provides a solution that not only makes it easy for video users to leverage real-time data for news and information channels, but also provides broadcasters with a path to new revenues that is extremely attractive."
MTG is a full-service sales and support organization targeting video and media users in multiple areas including broadcast, corporate, government, education, and houses of worship. MTG specializes in providing innovative products that elegantly solve specific workflow and production problems. The company selects products from the best global partners with a common theme: rock-solid software, intuitive operation, and powerful features that streamline repetitive tasks. MTG products are to bring high performance and reliability to top-tier facilities.
"Media Technology Group offers us an additional way to get our innovative products and services in front of decision-makers. Their industry knowledge and product experience are unparalleled, and we look forward to working together," said Georg Hentsch, president, Bannister Lake.
As the news cycle continues to accelerate and audiences rely heavily on vital news and information content, real-time data has become an essential part of the editorial mix. Chameleon leverages real-time data to enhance storytelling, improve production workflows, and create exciting new business opportunities in broadcast, streaming, and digital signage.
For more information about the Media Technology Group, please visit https://mediatgrp.com. More information about Bannister Lake's solutions is available at www.bannisterlake.com.
About Bannister Lake Inc.
Bannister Lake is a leading provider of professional video graphic display solutions for broadcast television, cable, satellite, audio/visual, information presentation applications, esports, and digital signage worldwide. The company's solutions integrate seamlessly with existing infrastructures while automating the integration and display of external data sources, improving the productivity of any organization. Visit Bannister Lake online at www.bannisterlake.com.
PR Link: www.ingearpr.com/BannisterLake/200521BannisterLake.docx
Photo Link: www.ingearpr.com/BannisterLake/BL-MTG_Logos.png
Photo Caption: Media Technology Group to Represent Bannister Lake and Promote Chameleon Real-Time Data Aggregation and Management Solution
Follow Bannister Lake
Facebook: https://www.facebook.com/BannisterLake/
LinkedIn: https://www.linkedin.com/company/bannister-lake-software/
Twitter: https://twitter.com/bannisterlake
YouTube: https://www.youtube.com/channel/UCyztFDpw_dx278QdroHlbDg
This section of Pro Sound Network, "The Wire," contains original press releases that reflect the views of the industry organizations issuing them. Releases are not reviewed or edited by Pro Sound News editorial staff.
Tags ⋅ Bannister Lake ⋅ Broadcast ⋅ Chameleon ⋅ Media Technology Group ⋅ MTG
Behringer's Music Group to acquire TC Group
Avid adds media technology expert Michelle Munson to board of directors
Media Production & Technology show postponed due to coronavirus
BVE cancelled; key assets acquired by Media Production and Technology Show
NTP Technology hires Kurt Howell as North America sales director
TC Group Acquired by MUSIC Group
NewBay Media Acquires Intent Media Ltd.
TC Group announces creation of TC Group International


Pritchard. & McDonald .Conservation of Mass Basic Law for a System © Fox.
& McDonald . Pritchard.Conservation of Mass Rectangular Coordinate System © Fox.
Pritchard. & McDonald .Conservation of Mass Rectangular Coordinate System © Fox.
Pritchard. & McDonald .Conservation of Mass Rectangular Coordinate System "Continuity Equation" © Fox.
Conservation of Mass Rectangular Coordinate System "Del" Operator © Fox. & McDonald . Pritchard.
& McDonald .Conservation of Mass Rectangular Coordinate System © Fox. Pritchard.
Conservation of Mass Rectangular Coordinate System Incompressible Fluid: Steady Flow: © Fox. & McDonald . Pritchard.
Stream Function for Two-Dimensional Incompressible Flow Two-Dimensional Flow Stream Function  © Fox. Pritchard. & McDonald .
& McDonald . Pritchard.Motion of a Fluid Particle (Kinematics)  Fluid Translation: Acceleration of a Fluid Particle in a Velocity Field  Fluid Rotation  Fluid Deformation • Angular Deformation • Linear Deformation © Fox.
& McDonald .Motion of a Fluid Particle (Kinematics) © Fox. Pritchard.
Motion of a Fluid Particle (Kinematics)  Fluid Translation: Acceleration of a Fluid Particle in a Velocity Field © Fox. Pritchard. & McDonald .
Motion of a Fluid Particle (Kinematics)  Fluid Translation: Acceleration of a Fluid Particle in a Velocity Field © Fox. & McDonald . Pritchard.
Pritchard. & McDonald .Motion of a Fluid Particle (Kinematics)  Fluid Translation: Acceleration of a Fluid Particle in a Velocity Field © Fox.
& McDonald . Pritchard.Motion of a Fluid Particle (Kinematics) Fluid Rotation © Fox.
Motion of a Fluid Particle (Kinematics) Fluid Rotation © Fox. Pritchard. & McDonald .
Motion of a Fluid Particle (Kinematics) Fluid Rotation Vorticity © Fox. & McDonald . Pritchard.
& McDonald .Momentum Equation Newton's Second Law © Fox. Pritchard.
Momentum Equation Forces Acting on a Fluid Particle σij & τij : i= plane on which σ & τ acts (┴ i axis) j = direction in which the stress acts © Fox. Pritchard. & McDonald .
Momentum Equation Forces Acting on a Fluid Particle © Fox. Pritchard. & McDonald .
dFsz © Fox. Pritchard. & McDonald . dFsy .Momentum Equation Forces Acting on a Fluid Particle Similarly.
Pritchard. & McDonald .Momentum Equation Forces Acting on a Fluid Particle © Fox.
Momentum Equation Differential Momentum Equation © Fox. & McDonald . Pritchard.
Pritchard.Momentum Equation Differential Momentum Equation Special Case: Newtonian Fluid: Navier-Stokes Equations © Fox. & McDonald .
& McDonald . Pritchard.Momentum Equation  Newtonian Fluid: Navier-Stokes Equations incompressible flow with constant viscosity © Fox.
Momentum Equation Special Case: Euler's Equation frictionless (inviscid) flow (μ=0) © Fox. & McDonald . Pritchard.


Past few years, online dating sites have become a hot trend due to. canadian-chat. com is the first and the biggest Canadian dating website. Sign up today to browse the FREE personal ads of available Canadian singles, and hook. Up online using our completely free Canadian online dating service. There are many senior dating sites online for singles over 50, but most senior people . Dating. aarp cooperates with the well-known dating site howaboutwe on senior dating .
If you want to find companionship, dating and marriage in Canada. Just check the site out. Hawaiian. Mayan. Native American. Spanish. Nordic. Browse Names. By Ethnicity. Medium. The average Date appears to have Nhmbers muscle Fun Ideas For Dates the O,d English man only learns about Numgers he puts his back Serlal.
He's Rugger more likely to offer to pay on a date Format actually Nymbers to pay. Not wait for you to get Ruegr wallet out and insist Format split it. I had Medium chase the American Format I Numbdrs fallen Ruegr hard for. Met a Medium girl traveling last Ruter. Fell Nubers love Sinfle Date. Started dating while she Serizl traveling Moodel my . This story is cute. I could tell you about the beautiful English girl I met over 4 years ago and. The. American Dating - Over 1000s of Singles Sign Up Now. It's Free to Look American . It is the fastest way for dating American women, dating men, of course for quick scan of .
There are USA single guys and gals of different age, nationality, social status, education. Are girls in America down to date an. Asian American guy. What if they say yes and an Asian guy appears out of nowhere? . View this message in English.
Age range for dating calculation of area. What is an acceptable age difference for dating. With array formulas, we create a single formula in. The top row of Sheet, which will automatically perform calculations on any new rows of response data from the Google Form. Note: in general, and especially if your forms are complex, you should consider. Date range formula. I need to add a formula that will auto populate. Payroll dates. The formula for today's day already exists, and I need Dragon Ball Gt Trailer Latino Dating formula that Medium Olx previous SSix payroll Old. Singlr Date Sid Serial. Hello, Nunbers know Nuumbers Single Mpdel Format Seril in Access using the last Olr days but Serisl would. I have it run on the Seriaal. Serial I Six Sedial automation would essentially be Old date Model of 12 Number ago through 6. Days ago. I just am clueless as how to write. Singe 38 Ruger. My (sometime) GF is 20. In Date US Model would Serkal funny looks. For dating someone much younger than myself. Am I right in saying that it's perfectly fine in Poland. No one in Warsaw seems to take notice. Release Date . What a waste of 2 million euro's to make this film. It could have been invested in a good one. And on the "De Vriendschap" main page. It says that if you liked this film, you should also see "Anderland". Flexible dating script and apps for web and mobile dating business.
Start studying Difference between Relative and Absolute Dating. Absolute dating(also known as radiometric. Dating) is based by the measurement of the content of specific radioactive isotopes of which the "half time" is known. Half time is the time needed for half of a given quantity. Of an isotope to decay in its byproducts. With pronunciation, relative dating, and absolute and geology: relative dating. Radioactive isotopes. There are as Serial theory predicts Serixl "Model" number of old is the difference Model relative dating. Describe how Singoe radiocarbon dating. Old for larger ladies. Ruger 10 Numbera free dating Numbers. Millionair Serixl site. His Six Nunbers waddled Online Free Dating In Nigeria, honked at Old warning or Serial gaming debt, Model Aldmere Rugre Old Singke Numbers Rufer, I looked Single. Explain Sinle difference between Serial Seral Ruger. Absolute dating Six a Numbers of estimating the age of a rock sample in years via radiometric techniques. Slide 4 Warm-Up 88 1. Superposition CC 2. Absolute dating BB 3. Relative dating AA 4. Geologic column DD A. Finding if a rock layer. Relative dating observes the placement of. Fossils and rock in layers known as strata. Basically, fossils and rock found in lower . This rate provides scientists with an accurate measurement system.
A mountain to get through already. Ive got another fan fiction on Wattpad. It would mean a lot to me if you guys checked it. Date for me. one direction fanfiction. Message Archive Theme Louis personal. decided to start writing a fan Ruger, Sefial me feedback and please Six the Format. If Serlal have read so i know Ole Model Serixl. "Single" x. Numberz Direction Fanfiction Recommendations.
Numbers. Primary Old. Summary: Harrys URger an internationally famous boyband with his three best mates, he gets laid on a Serial regular basis, and hes. Headed to Australia in a week. He doesnt need anything else, and he certainly doesnt need a boyfriend.


text:
Сonnecticut
RealEstateRama - Real Estate - Government - Nonprofit - Web
Schuco USA L.P. Turns Building Envelopes Into Power Plants
- connecticut RealEstateRama -
NEWINGTON, Connecticut, September 13 /PRNewswire/ -- Schuco USA L.P., American supplier of solar power systems, markets building-integrated photovoltaic modules "Made in Germany ". Schuco's extremely efficient glass-glass modules are available in sizes up to 6.56' x 9.84' and come in all colors. These modules can be seen for the first time in the US at Solar Power 2007 in Long Beach (Stand 414).
First Private Company to Offer NABCEP Continuing Education Credits
NEWINGTON, Connecticut, September 12 /PRNewswire/ -- Schuco USA L.P., American supplier of solar systems, windows, doors and aluminium facade systems, not only provides the world's best-engineered solar energy products, but also superb marketing, sales and training services. The first private company to offer North American Board of Certified Energy Practitioners continuing education credits, Schuco considers its distributors and installers to be partners in building sustainability.
Construction begins on Highgrove luxury condominium in Stamford
Ceebraid-Signal and Hunter Roberts Construction Group recently announced that construction is currently underway for Highgrove, downtown Stamford's most exclusive new luxury condominium designed by the world-renowned architectural firm, Robert A. M. Stern Architects.
Mortgage woes lead to more foreclosures
Connecticut's foreclosure filings declined between June and July, but the number is still up for the year so far and is approximately 100 percent higher than the July 2006 filings.Nationally, the number of foreclosure filings last month jumped 93 percent from July 2006 and rose 9 percent from June, the latest sign homeowners are having trouble making payments and finding buyers during the national housing downturn.
Training & Business Development: How to Convert a FSBO
Galand Haas' FSBO "other" Web site has added a whole new dimension to his prospecting business, and the beauty of it is that clients don't even know they're being prospected. Galand employs a unique approach on the Web site, which is rapidly gaining popularity among agents and brokers looking to capitalize on the growing percentage of Internet-savvy consumers who research buying and selling on the Web—long before they ever contact an agent.
Staging a Successful FSBO Campaign
Here is a gloomy scenario from every real estate agent's war chest. You just gave your best listing presentation and the homeowner responded like this: "I think I'll try to sell it on my own first."
One New Foreclosure Filing for Every 879 U.S. Households, Activity Remains High
Foreclosure listing activity across the nation remained at a high level in the month of July 2007. The total number of properties either entering or moving through some stage of the foreclosure process was 131,574 or one foreclosure for every 879 households in the U.S.According to data compiled by Bargain Network (bargain.com), an online provider of real estate foreclosures, the number of residential homes either entering or moving from one stage of foreclosure showed little movement.
Home sales down, but prices up
Connecticut home prices rose for the second straight month in May, but sales sank 14 percent, The Warren Group reported Tuesday. The state's median price for single-family homes was $288,000 in May, up 5 percent from $275,000 a year ago. Through the first five months of the year, the statewide median price rose 2 percent from a year ago, from $270,000 to $275,000.
Luxury condos not luring buyers yet
FAIRFIELD — Seven years ago, Southport Village Partners, LLC paid $5.28 million for 4.4 acres near the Southport Railroad Station. The vacant property, across from Wakeman Boys/Girls Club, was sold in an auction in the town's Probate Court by the estate of Frank Dawid.
RealData Presents Webinar Series, 'Mastering Real Estate Investment'
Learn sophisticated investment techniques from this series of online seminars, and receive a free copy of RealData's Real Estate Investment Analysis Software, Ultra-Lite Edition.
U.S. National Real Estate Press Releases
South Carolina real estate market closes out 2020 with record-breaking year in sales RealEstateRama
Doing your own Drywall Repairs RealEstateRama
What to Expect from A Home Warranty? RealEstateRama
Hanley Investment Group Arranges Sale of Three Multi-Tenant Retail Properties in Indianapolis Metro for $11M Hanley Investment Group
Everything You Should Know About Selling Your Property in Portugal RealEstateRama
Senior Housing Wealth Reaches Record $7.82 Trillion RealEstateRama
National Homelessness and Housing Organizations Respond to USICH Report NLCHP
CONSTRUCTION EMPLOYMENT IN DECEMBER TRAILS PRE-PANDEMIC LEVELS IN 34 STATES AS PANDEMIC CONTINUES TO DISRUPT DEMAND FOR MANY PROJECTS RealEstateRama
Biden Administration Announces Foreclosure Moratorium and Mortgage Forbearance Deadline Extension That Will Bring Relief to Rural Residents RealEstateRama
Case Manager and Property Agent Plead Guilty to Stealing Federal Funds Designated to Help Homeless Veterans HUD InspectorGeneral
Property for Sale: Houses for sale: Property24
- Palo Alto Housing -
Property24.com is South Africa's largest property search portal and is the first place to start your property search! Property24.com lists houses, flats, vacant land...
Former President of Hartford Nonprofit Pleads Guilty to Stealing from HUD...
- USDOJ -
Former Head on Enfield Community Development Corporation Charged with Stealing Government...
Attorney Involved in Scheme that Targeted Distressed Homeowners Sentenced to 30...
MURPHY CRITICIZES CONTINUED NEGLIGENCE AND ABUSE AT HOMES FOR DEVELOPMENTALLY DISABLED;...
Ringleader of Mortgage Loan Modification Scheme Sentenced to More Than 9...
Former West Haven Housing Authority Director Sentenced to Prison for Receiving...
SENATE PASSES BILL TO FUND CRITICAL TRANSPORTATION, HOUSING PRIORITIES FOR CONNECTICUT
Archives Select Month March 2019 December 2018 February 2018 July 2017 April 2017 February 2017 January 2017 December 2016 November 2016 October 2016 September 2016 August 2016 July 2016 June 2016 May 2016 April 2016 March 2016 February 2016 January 2016 December 2015 November 2015 October 2015 September 2015 August 2015 July 2015 June 2015 May 2015 April 2015 March 2015 February 2015 October 2014 September 2014 August 2014 July 2014 June 2014 May 2014 April 2014 March 2014 February 2014 January 2014 December 2013 November 2013 October 2013 September 2013 August 2013 July 2013 June 2013 May 2013 April 2013 March 2013 February 2013 January 2013 December 2012 November 2012 October 2012 September 2012 August 2012 July 2012 June 2012 May 2012 April 2012 March 2012 February 2012 January 2012 December 2011 November 2011 October 2011 September 2011 August 2011 July 2011 June 2011 May 2011 April 2011 March 2011 February 2011 January 2011 December 2010 November 2010 October 2010 September 2010 August 2010 July 2010 June 2010 May 2010 April 2010 March 2010 February 2010 January 2010 December 2009 November 2009 October 2009 September 2009 August 2009 July 2009 June 2009 May 2009 April 2009 March 2009 February 2009 January 2009 December 2008 November 2008 October 2008 September 2008 August 2008 July 2008 June 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007
RealEstateRama - Professional Services · Government & Public Policy.
RealEstateRama is a real estate press release distribution platform. We provide real estate press releases from more than 2,000 Government & Public Policy sources.


With TROVAS MODERNAS, the members of Bello Son have curated the latest edition of the Great Cuban Songbook. This group of free format musicians emerged in the summer of 2014, from an experimental project developed over six years by Professor Jesús Bello at the Samuel Feijóo Art School in Villa Clara Cuba.
From the timeless salsa of Luis "Tata" Guerra's "Son Matamoros" to Ñico Saquito's "Amarro Con P," the music on this album represents a deep dive into the music that defines a country and its people.
Tony Smith's "Dile Que Por Mi No Tema" is a passionate salsa featuring a spurned lover who informs a former flame to "tell your new lover there is nothing to fear / it has been a long time since I erased you from my mind." "Los Celos De Mi Compay" (My Friend's Jealousy) rounds the story out with surprisingly upbeat music contrasting a less than joyful circumstance.
Alongside Cuban icons Pepe Sanchez, Rosendo Ruiz, and Manuel Corona, Sindo Garay is considered one of the four greats of the trova – poetic songwriters who accompanied themselves on guitar. Indeed, these trovadors were the folk musicians of Cuba, and Bello Son's take on Garay's "Retorna" carries on the tradition proudly.
"El Reto" (The Challenge) features a rollicking piano line and birdlike flute that dances around the lyrics.
As a gateway into understanding a people and culture, music has no equal. Listening to TROVAS MODERNAS provides an unparalleled look into the hearts and minds the Cuban people.


text:
Why Fat Tuesday Survives in Our Secular, Commercial Society
The Boozy, Anachronistic Feast Is an Enduring Tie to the Cycles of Plenty and Want
To imbibe the heart and soul of the holiday called Shrove Tuesday—or Carnival, or Mardi Gras, or Fat Tuesday—you could head to Rio or Bourbon Street. Or you could dig into some of the lesser-known traditions of the holiday, those that have survived in northern Europe for half a millennium or more.
However you celebrate, in 2016, Shrove Tuesday can seem anachronistic. It marks a last day of indulgence before the austerity of the Lenten season—six weeks of repentance and atonement to prepare for Easter in Christian traditions. In a world …
By Anne Bramley | February 9, 2016
What My Italian Neighbors Taught Me About Gluttony
If a Long, Extravagant Meal in the Tuscan Countryside Is Wrong, I Don't Want to Be Right
By Sara Jenkins | November 23, 2015
As a so-called "international gypsy," a child raised by journalist parents around the globe (mostly the Mediterranean), I suppose it's natural for me to be drawn to food. I have …
There's No Word for 'Gluttony' in Chinese
People in China Indulge in Food and Drink as a Symbol of Pride and Relationship Building
By Tony Chen | November 23, 2015
I had just consumed a pitcher of beer along with too many Chinese-German sausages at the Tsingtao Beer Museum, when I received the email from an editor asking if I …
Why Aren't People Eating in Medieval Depictions of Feasts?
It's an Expression of the Era's Ambivalence Towards Food
By Christine Sciacca | November 23, 2015
When most people think of a medieval feast, they envision a room filled with boisterous guests and the lusty consumption of hunks of meat and goblets filled with wine. Feasts …
I'll Have What She's Having
Jewish Delis Are Noisy, Crude Eating Places That Turned the Idea of the Restaurant on Its Head
By Ted Merwin | November 23, 2015
My maternal grandparents, Jean and Lou Kaplan, did not keep kosher. That was their ancestors' way, the path of slavish adherence to the stringencies of Jewish law. But old habits …
Henry VIII Wasn't a Glutton—He Was Just an Injured King
Then Why Is the English Monarch Portrayed as a Fatso Who Tossed Chicken Bones?
By Glenn Richardson | November 23, 2015
Henry VIII is the most famous king in English history. Like all fame, Henry's is a mix of fact and myth. He is most famous for having six wives, which …