text:
\section{Introduction}
Effectively identifying meaningful entities or entity mentions from the raw text plays a crucial part in understanding the semantic meanings of natural language. Such a process is usually known as Named Entity Recognition (NER) and it is one of the fundamental tasks in natural language processing (NLP). A typical NER system takes an utterance as the input and outputs identified entities, such as person names, locations, and organizations. The extracted named entities can benefit various subsequent NLP tasks, including syntactic parsing \cite{koo2010efficient}, question answering \cite{krishnamurthy2015learning} and relation extraction \cite{lao2010relational}. However, accurately recognizing representative entities in natural language remains challenging.

Previous works treat NER as a sequence labeling problem. For example, \citet{lample2016neural} achieve a decent performance on NER by incorporating deep recurrent neural networks (RNNs) with conditional random field (CRF) \cite{lafferty2001conditional}.
However, a critical problem that arises by treating NER as a sequence labeling task is that it only recognizes non-overlapping entities in a single, sequential scan on the raw text; it fails to detect nested named entities which are embedded in longer entity mentions, as illustrated in Figure \ref{fig:multi}.

\begin{figure}[bpth!]
    \centering
    \includegraphics[width=0.8\linewidth]{figure/Nested_Sen.pdf}
    \caption{An example from the ACE-2004 dataset \cite{doddington2004automatic} in which two GPEs (Geographical Entities) are nested in a Facility Entity. }
    \label{fig:multi}
\end{figure}

Due to the semantic structures within natural language, nested entities can be ubiquitous: {\it e.g.} 47\% of the entities in the test split of ACE-2004 \cite{doddington2004automatic} dataset overlap with other entities, and 42\% of the sentences contain nested entities.
Various approaches \cite{alex2007recognising, lu2015joint, katiyar2018nested, lu2017mention, wang2018neural} have been proposed in the past decade to extract nested named entities.
However, these models are designed explicitly for recognizing nested named entities. They usually do not perform well on non-overlapping named entity recognition compared to sequence labeling models.

To tackle the aforementioned drawbacks, we propose a novel neural framework, named {\textsc{MGNER}}, for Multi-Grained Named Entity Recognition. It is suitable for tackling both Nested NER and Non-overlapping NER.
The idea of {\textsc{MGNER}} is natural and intuitive, which is to first detect entity positions in various granularities via a Detector and then classify these entities into different pre-defined categories via a Classifier. {\textsc{MGNER}} has five types of modules: Word Processor, Sentence Processor, Entity Processor, Detection Network, and Classification Network, where each module can adopt a wide range of neural network designs.

In summary, the contributions of this work are:
\begin{itemize}[leftmargin=*]
    \setlength\itemsep{0mm}
    \item We propose a novel neural framework named {\textsc{MGNER}} for Multi-Grained Named Entity Recognition, aiming to detect both nested and non-overlapping named entities effectively in a single model.

    \item {\textsc{MGNER}} is highly modularized.
    Each module in {\textsc{MGNER}} can adopt a wide range of neural network designs. Moreover, {\textsc{MGNER}} can be easily extended to many other related information extraction tasks, such as chunking \cite{ramshaw1999text} and slot filling \cite{mesnil2015using}.

    \item Experimental results show that {\textsc{MGNER}} is able to achieve new state-of-the-art results on both Nested Named Entity Recognition tasks and Non-overlapping Named Entity Recognition tasks.

\end{itemize}
\section{Related Work}
Existing approaches for recognizing non-overlapping named entities usually treat the NER task as a sequence labeling problem. Various sequence labeling models achieve decent performance on NER,
including probabilistic graph models such as Conditional Random Fields (CRF) \cite{ratinov2009design}, and deep neural networks like recurrent neural networks or convolutional neural networks (CNN). \citet{hammerton2003named} is the first work to use Long Short-Term Memory (LSTM) for NER.
\citet{collobert2011natural} employ a CNN-CRF structure, which obtains competitive results to statistical models.
Most recent works leverage an LSTM-CRF architecture.
\citet{huang2015bidirectional} use hand-crafted spelling features; \citet{ma2016end} and \citet{chiu2016named} utilize a character CNN to represent spelling characteristics; \citet{lample2016neural} employ a character LSTM instead. Moreover, the attention mechanism is also introduced in NER to dynamically decide how much information to use from a word or character level component \cite{rei2016attending}.

External resources have been used to further improve the NER performance.
\citet{peters2017semi} add pre-trained context embeddings from bidirectional language models to NER.
\citet{peters2018deep} learn a linear combination of internal hidden states stacked in a deep bidirectional language model, ELMo, to utilize both higher-level states which capture context-dependent aspects and lower-level states which model aspects of syntax.
These sequence labeling models can only detect non-overlapping entities and fail to detect nested ones.

Various approaches have been proposed for Nested Named Entity Recognition.
\citet{finkel2009nested} propose a CRF-based constituency parser which takes each named entity as a constituent in the parsing tree.
\citet{ju2018neural} dynamically stack multiple flat NER layers and extract outer entities based on the inner ones. Such model may suffer from the error propagation problem if shorter entities are recognized incorrectly.

Another series of approaches for Nested NER are based on hypergraphs.
The idea of using hypergraph is first introduced in \citet{lu2015joint}, which allows edges to be connected to different types of nodes to represent nested entities.
\citet{lu2017mention} use a multigraph representation and introduce the notion of mention separator for nested entity detection.
Both \citet{lu2015joint} and \citet{lu2017mention} rely on the hand-crafted features to extract nested entities and suffer from structural ambiguity issue. \citet{wang2018neural} present a neural segmental hypergraph model using neural networks to obtain distributed feature representation.
\citet{katiyar2018nested} also adopt a hypergraph-based formulation and learn the structure using an LSTM network in a greedy manner.
One issue of these hypergraph approaches is the spurious structures of hypergraphs as they enumerate combinations of nodes, types and boundaries to represent entities. In other words, these models are specially designed for the nested named entities and are not suitable for the non-overlapping named entity recognition.

\citet{xu2017local} propose a local detection method which relies on a Fixed-size Ordinally Forgetting Encoding (FOFE) method to encode utterance and a simple feed-forward neural network to either reject or predict the entity label for each individual text fragment \cite{luan2018multi, lee2017end, he2018jointly}. Their model is in the same track with the framework we proposed whereas the difference is that we separate the NER task into two stages, {\it i.e.}, detecting entity positions and classifying entity categories.
\section{The Proposed Framework}
An overview of the proposed {\textsc{MGNER}} framework for multi-grained entity recognition, is illustrated in Figure \ref{fig:EPN}.
Specifically, {\textsc{MGNER}} consists of two sub-networks: the Detector and the Classifier.
The Detector detects all the possible entity positions while the Classifier aims at classifying detected entities into pre-defined entity categories.
The Detector has three modules: 1) Word Processor which extracts word-level semantic features, 2) Sentence Processor that learns context information for each utterance and 3) Detection Network that decides whether a word segment is an entity or not.
The Classifier consists of 1) Word Processor which has the same structure as the one in the Detector, 2) Entity Processor that obtains entity features and 3) Classification Network that classifies entity into pre-defined categories.
In addition, a self-attention mechanism is adopted in the Entity Processor to help the model capture and utilize entity-related contextual information.

Each module in {\textsc{MGNER}} can be replaced with a wide range of different neural network designs. For example, BERT \cite{devlin2018bert} can be used as the Word Processor and a capsule model \cite{sabour2017dynamic, xia2018zero} can be integrated into the Classification Network.

It is worth mentioning that in order to improve the learning speed as well as the performance of {\textsc{MGNER}}, the Detector and the Classifier are trained with a series of shared input features, including the pre-trained word embeddings and the pre-trained language model features.
Sentence-level semantic features trained in the Detector are also transferred into the Classifier to introduce and utilize the contextual information.
We present the key building blocks and the properties of the Detector in Section 3.1 and the Classifier in Section 3.2, respectively.

\subsection{The Detector}
The Detector is aimed at detecting possible entity positions within each utterance. It takes an utterance as the input and outputs a set of entity candidates.
Essentially, we use a semi-supervised neural network inspired by \cite{peters2017semi} to model this process. The architecture of the Detector is illustrated in the left part of Figure \ref{fig:EPN}. Three major modules are contained in the Detector: Word Processor, Sentence Processor and Detection Network. More specifically, pre-trained word embeddings, POS tag information and character-level word information are used for generating semantically meaningful word representations. Word representations obtained from the Word Processor and the language model embeddings---ELMo \cite{peters2018deep}, are concatenated together to produce context-aware sentence representations. Each possible word segment is then examined in the Detection Network and to be decided whether accepted it as an entity or not.

\subsubsection{Word Processor}
Word Processor extracts semantically meaningful word representation for each token.
Given an input utterance with $K$ tokens $(t_{1}, ..., t_{K})$, each token $t_k (1\leq k \leq K)$ is represented as
\begin{equation*}
    \mathbf{x}_{k}= [ \mathbf{w}_{k}; \mathbf{p}_{k}; \mathbf{c}_{k}],
\end{equation*}\vspace{-1.5em}

\noindent by using a concatenation of a pre-trained word embedding $\mathbf{w}_{k}$, POS tag embedding $\mathbf{p}_{k}$ if it exists, and a character-level word information $\mathbf{c}_k$.
The pre-trained word embedding $\mathbf{w}_k$ with a dimension $D_w$ is obtained from GloVe \cite{pennington2014glove}.
The character-level word information $\mathbf{c}_k$ is obtained with a bidirectional LSTM \cite{hochreiter1997long} layer to capture the morphological information. The hidden size of this character LSTM is set as $D_{cl}$.
As shown in the bottom of Figure \ref{fig:EPN}, character embeddings are fed into the character LSTM.
The final hidden states from the forward and backward character LSTM are concatenated as the character-level word information $\mathbf{c}_k$.
Those POS tagging embeddings and character embeddings are randomly initialized and learned within the learning process.

\subsubsection{Sentence Processor}
To learn the contextual information from each sentence, another bidirectional LSTM, named word LSTM, is applied to sequentially encode the utterance.
For each token, the forward hidden states ${\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightarrow$}}\over {\mathbf{h}}} }_{k}$ and the backward hidden states ${\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\leftarrow$}}\over {\mathbf{h}}} }_{k}$ are concatenated into the hidden states $\mathbf{h}_{k}$.
The dimension of the hidden states of the word LSTM is set as $D_{wl}$.
\begin{align}
\begin{split}
     {\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightarrow$}}\over {\mathbf{h}}} }_{k}  &= {\rm LSTM}_{fw} (\mathbf{x}_{k}, {\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\leftarrow$}}\over {\mathbf{h}}} }_{k-1} ), \\
     ~~{\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\leftarrow$}}\over {\mathbf{h}}} }_{k} &= {\rm LSTM}_{bw} (\mathbf{x}_{k},  {\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\leftarrow$}}\over {\mathbf{h}}} }_{k+1}), \\
     ~~\mathbf{h}_{k} &= [ {\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightarrow$}}\over {\mathbf{h}}} }_{k}; {\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\leftarrow$}}\over {\mathbf{h}}} }_{k}].
\end{split}
\end{align}
Besides, we also utilize the language model embeddings pre-trained in an unsupervised way as the ELMo model in \citep{peters2018deep}. The pre-trained ELMo embeddings and the hidden states in the word LSTM $\mathbf{h}_{k}$ are concatenated.
Hence, the concatenated hidden states $\mathbf{h}_{k}$ for each token can be reformulated as:
\begin{equation}
    \mathbf{h}_{k} = [ {\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightarrow$}}\over {\mathbf{h}}} }_{k}; {\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\leftarrow$}}\over {\mathbf{h}}} }_{k}; \mathbf{ELMo}_{k}],
\end{equation}
where $\mathbf{ELMo}_{k}$ is the ELMo embeddings for token $t_k$. Speficially, a three-layer bi-LSTM neural network is trained as the language model. Since the lower-level LSTM hidden states have the ability to model syntax properties and higher-level LSTM hidden states can capture contextual information, ELMo computes the language model embeddings as a weighted combination of all the bidirectional LSTM hidden states:
\begin{equation}
    \mathbf{ELMo}_{k} = \gamma \sum\nolimits_{l = 0}^L {{u_j}} \mathbf{h}_{k,l}^{LM},
\end{equation}
where $\gamma$ is a task-specified scale parameter which indicates the importance of the entire ELMo vector to the NER task. $L$ is the number of layers used in the pre-trained language model, the vector $\mathbf{u} = [u_0, \cdots, u_L]$ represents softmax-normalized weights that combine different layers. $\mathbf{h}_{k,l}^{LM}$ is the language model hidden state of layer $l$ at the time step $k$.

A sentence bidirectional LSTM layer with a hidden dimension of $D_{sl}$ is employed on top of the concatenated hidden states $\mathbf{h}_{k}$. The forward and backward hidden states in this sentence LSTM are concatenated for each token as the final sentence representation $\mathbf{f}_{k}$ $\in \mathbb{R}^{2D_{sl}}$.

\subsubsection{Detection Network}
Using the semantically meaningful features obtained in $\mathbf{f}_{k}$, we can identify possible entities within each utterance. The strategy of finding entities is to first generate all the word segments as entity proposals and then estimate the probability of each proposal as being an entity or not.

To enumerate all possible entity proposals, different lengths of entity proposals are generated surrounding each token position.
For each token position, R entity proposals with the length varies from 1 to the maximum length R are generated.
Specifically, it is assumed that an input utterance consists of a sequence of $N$ tokens $(t_{1}, t_{2}, t_{3}, t_{4}, t_{5}, t_{6}, ..., t_{N})$.
To balance the performance and the computational cost, we set $R$ as 6.
We take each token position as the center and generate 6 proposals surrounding it. All the possible $6N$ proposals under the max-length of 6 will be generated.
As shown in Figure \ref{fig:entity_proposal}, the entity proposals generated surrounding token $t_3$ are: $(t_{3})$, $(t_{3}, t_{4})$, $(t_{2}, t_{3}, t_{4})$, $(t_{2}, t_{3}, t_{4}, t_{5})$, $(t_{1}, t_{2},  t_{3}, t_{4}, t_{5})$, $(t_{1}, t_{2},  t_{3}, t_{4}, t_{5}, t_{6})$.
Similar entity proposals are generated for all the token positions and proposals that contain invalid indexes like ($t_{0}$,$t_{1}$,$t_{2}$) will be deleted. Hence we can obtain all the valid entity proposals under the condition that the max length is R.

\begin{figure}[htbp!]
    \centering
    \includegraphics[width=0.9\linewidth]{figure/EntityProposal.pdf}\vspace{-2mm}
    \caption{All possible entity proposals generated surrounding token $t_{3}$ when the maximum length of an entity proposal $R$ is set as 6.}
    \label{fig:entity_proposal}
\end{figure}

For each token, we simultaneously estimate the probability of a proposal being an entity or not for R proposals.
A fully connected layer with a two-class softmax function is used to determine the quality of entity proposals:
\begin{equation}
    \mathbf{s}_{k} = {\rm softmax}\left (\mathbf{f}_{k}\mathbf{W}_{p} + \mathbf{b}_{p}\right ),
\end{equation}
where \( \mathbf{W}_{p} \in \mathbb{R}^{2D_{sl} \times 2R}\) and \( \mathbf{b}_{p} \in \mathbb{R}^{2R}\) are weights and the bias for the entity proposal layer; $\mathbf{s}_{k}$ contains $2R$ scores including $R$ scores for being an entity and R scores for not being an entity at position $k$. The cross-entropy loss is employed in the Detector as follows:
\begin{equation}
   {L_p} =  - \sum\nolimits_{k = 1}^K {\sum\nolimits_{r = 1}^R {{\bf{y}}_{k}^{r}\log {\bf{s}}_k^r} },
\end{equation}
where ${\bf{y}}_{k}^{r}$ is the label for proposal type $r$ at position $k$ and ${\bf{s}}_{k}^{r}$ is the probability of being an entity for proposal type $r$ at position $k$.
It is worth mentioning that, most entity proposals are negative proposals. Thus,
to balance the influence of positive proposals and negative proposals in the loss function, we keep all positive proposals and use down-sampling for negative proposals when calculating the loss $L_p$.
For each batch, we fix the number of the total proposals, including all positive proposals and sampled negative proposals, used in the loss function as $N_b$.
In the inference procedure of the Detection Network, an entity proposal will be recognized as an entity candidate if its score of being an entity is higher than score of not being an entity.

\subsection{The Classifier}
The Classifier module aims at classifying entity candidates obtained from the Detector into different pre-defined entity categories. For the nested NER task, all the proposed entities will be saved and fed into the Classifier. For the NER task which has non-overlapping entities, we utilize the non-maximum suppression (NMS) algorithm \citep{neubeck2006efficient} to deal with redundant, overlapping entity proposals and output real entity candidates.
The idea of NMS is simple but effective: picking the entity proposal with the maximum probability, deleting conflict entity proposals, and repeating the previous process until all the proposals are processed. Eventually, we can get those non-conflict entity candidates as the input of the Classifier.

To understand the contextual information of the proposed entity, we utilize both sentence-level context information and a self-attention mechanism to help the model focus on entity-related context tokens.
The framework of the Classifier is shown in the right part of Figure \ref{fig:EPN}. Essentially, it consists of three modules: Word Processor, Entity Processor and Classification Network.

\begin{table*}[!ht]
\centering
\resizebox{\linewidth}{!}
\begin{tabular}{ll|ccc|ccc|ccc}
 &&\multicolumn{3}{c|}{\textbf{ACE-2004}}  & \multicolumn{3}{c|}{\textbf{ACE-2005}} & \multicolumn{3}{c}{\textbf{CoNLL-2003}}   \\
&& \textbf{TRAIN}     & \textbf{DEV}   & \textbf{TEST}      & \textbf{TRAIN}     & \textbf{DEV}   & \textbf{TEST}      & \textbf{TRAIN}     & \textbf{DEV}   & \textbf{TEST}      \\
 \hline
\multirow{2}{*}{sentences}&\#total          & 6,799    &829     & 879          & 7,336     &958     & 1,047       &14,987   &3,466&3,684   \\
&\#overlaps               & 2,683(39\%) &293(35\%) & 373(42\%)   & 2,683 (37\%) &340(35\%) & 330 (32\%)  & - &-& - \\\hline
\multirow{4}{*}{entities}&\#total           & 22,207    &2,511     & 3,031        & 24,687    &3,217    & 3,027       & 23,499      &5,942 &5,648      \\
&\#overlaps                & 10,170 (46\%) & 1,091(43\%) & 1,418 (47\%) &        9,937 (40\%)    &1,192(37\%)   &    1,184 (39\%)         & - & - &- \\
&length \textgreater 6 & 1,439 (6\%) &179(7\%)  & 199 (7\%)    & 1,343 (5\%)  &148(5\%) & 160 (6\%)   & 23(0.1\%) &8(0.1\%) & 0 (0\%)    \\
&max length            & 57        &35    & 43           & 49      &30      & 27          & 10 &10     & 6     \\
\end{tabular}
}
\caption{Corpora Statistics for the ACE-2004, ACE-2005 and CoNLL-2003 datasets.}
\label{data_statistics}
\end{table*}

\subsubsection{Word Processor}
A same Word Processor as in the Detector is used here to get the word representation for the entity candidates obtained from the Detector.
The word-level embedding, which is the concatenation of pre-trained word embedding and POS tag embedding if it is exists, is transferred from the Word Processor in the Detector to improve the performance as
well as to speed up the learning process. The character-level LSTM and character embeddings are trained separately in the Detector and the Classifier.

\subsubsection{Entity Processor}
The word representation is fed into a bidirectional word LSTM with hidden size $D_{wl}$ and the hidden states are concatenated with the ELMo language model embeddings as the entity features.
A bidirectional LSTM with hidden size $D_{el}$ is applied to the entity feature to capture sequence information among the entity words. The last hidden states of the forward and backward Entity LSTM are concatenated as the entity representation $\mathbf{e} \in \mathbb{R}^{2D_{el}}$.

The same word in different contexts may have different semantic meanings.
To this end, in our model, we take the contextual information into consideration when learning the semantic representations of entity candidates.
We capture the contextual information from other words in the same utterance. Denote $\mathbf{c}$ as the context feature vector for these context words, and it can be extracted from the sentence representation $\mathbf{f}_{k}$ in the Detector.
Hence, the sentence features trained in the Detector is directly transferred to the Classifier.

An easy way to model context words is to concatenate all the word representations or average them. However, this naive approach may fail when there exists a lot of unrelated context words. To select high-relevant context words and learn an accurate contextual representation, we propose a self-attention mechanism to simulate and dynamically control the relatedness between the context and the entity.
The self-attention module takes the entity representation $\mathbf{e}$ and all the context features $\mathbf{C} =  [\mathbf{c_{1}}, \mathbf{c_{2}}, ...,  \mathbf{c_{N}}]$ as the inputs, and outputs a vector of attention weights $\mathbf{a}$:
\begin{equation}
    \mathbf{a} = softmax( \mathbf{C} \mathbf{W} \mathbf{e}^T),
\end{equation}
where $\mathbf{W} \in \mathbb{R}^{2D_{sl} \times 2D_{el}}$ is a weight matrix for the self-attention layer, and $\mathbf{a}$ is the self-attention weight on different context words. To help the model focus on entity-related context, the attentive vector $\mathbf{C}^{att}$ is calculated as the attention-weighted context:
\begin{equation}
\mathbf{C}^{att} = \mathbf{a} * \mathbf{C}.
\end{equation}
The lengths of the attentive context $\mathbf{C}^{att}$ varies in different contexts. However, the goal of the Classification Network is to classify entity candidates into different categories, and thus it requires a fixed embedding size. We achieve that by adding another LSTM layer. An Attention LSTM with the hidden dimension $D_{ml}$ is used and the concatenation of the last hidden states in the forward and backward LSTM layer as the context representation $\mathbf{m} \in \mathbb{R}^{2D_{ml}}$. Hence the shape of the context representation is aligned.
We concatenate the context representation and the entity representation together as a context-aware entity representation to classify entity candidates: $ \mathbf{o} = [\mathbf{m}; \mathbf{e}]$.

\subsubsection{Classification Network}
A two-layer fully connected neural network is used to classify candidates into pre-defined categories:
\begin{equation}
   \mathbf{p}= {\rm softmax}\left({ \mathbf{W}_{c2}\left( {\sigma \left (\mathbf{o}\mathbf{W}_{c1} + \mathbf{b}_{c1}\right )}\right)}  + \mathbf{b}_{c2} \right),
\end{equation}
where $\mathbf{W_{c1}} \in \mathbb{R}^{(2D_{ml}+2D_{el}) \times D_h}$, $\mathbf{b_{c1}} \in \mathbb{R}^{D_h}$ , $\mathbf{W_{c2}} \in \mathbb{R}^{D_{c1} \times (D_{t}+1)}$, $\mathbf{b_{c2}} \in \mathbb{R}^{D_t+1}$ are the weights for this fully connected neural network, and $D_t$ is the number of entity types. Actually, this classification function classifies entity candidates into $(D_{t}+1)$ types. Here we add one more type as for the scenario that a candidate may not be a real entity.
Finally, the hinge-ranking loss is adopted in the Classification Network:
\begin{equation}
L_{c} =  \sum\nolimits_{y_w \in Y_w} max \left \{  0,\Delta +\mathbf{p}_{y_w} - \mathbf{p}_{y_r}  \right \},
\end{equation}
where $\mathbf{p}_{w}$ is the probability for the wrong labels $y_w$, $\mathbf{p}_{r}$ is the probability for the right label $y_r$, and $\Delta$ is a margin.
The hinge-rank loss urges the probability for the right label higher than the probability for the wrong labels and improves the classification performance.
\section{Experiments}
To show the ability and effectiveness of our proposed framework, {\textsc{MGNER}}, for {Multi-Grained} Named Entity Recognition, we conduct the experiments on both Nested NER task and traditional non-overlapping NER task.

\subsection{Datasets}
We mainly evaluate our framework on ACE-2004 and ACE-2005 \cite{doddington2004automatic} with the same splits used by previous works \cite{luo2015joint, wang2018neural} for the nested NER task. Specifically, seven different types of entities such as person, facility, weapon and vehicle, are contained in the ACE datasets.
For the traditional NER task, we use the CoNLL-2003 dataset \cite{tjong2003introduction} which contains four types of named entities: location, organization, person and miscellaneous.
An overview of these three datasets is illustrated in Table \ref{data_statistics}. It can be observed that most entities are less or equal to 6 tokens, and thus we select the maximum entity length $R=6$.

\subsection{Implementation Details}
We performed random search \cite{bergstra2012random} for hyper-parameter optimization and selected the best setting based on performance on the development set.
We employ the Adam optimizer \cite{kingma2014adam} with learning rate decay for all the experiments. The learning rate is set as 0.001 at the beginning and exponential decayed by 0.9 after each epoch.
The batch size of utterances is set as 20. In order to balance the influence of positive proposals and negative proposals, we use down-sampling for negative ones and the total proposal number $N_b$ for each batch is 128.
To alleviate over-fitting, we add dropout regularizations after the word representation layer and all the LSTM layers with a dropout rate of 0.5.
In addition, we employ the early stopping strategy when there is no performance improvement on the development dataset after three epochs.
The pre-trained word embeddings are from GloVe \cite{pennington2014glove}, and the word embedding dimension $D_w$ is 300. Besides, the ELMo 5.5B data\footnote{https://allennlp.org/elmo} is utilized in the experiment for the language model embedding.
Moreover, the size of character embedding  $\mathbf{c}_k$ is 100, and the hidden size of the Character LSTM $D_{cl}$ is also 100.
The size of POS tag embedding $\mathbf{p}_{k}$ is 300 for the ACE datassets and no POS tag information is used in the CoNLL-2003 dataset.
The hidden dimensions of the Word LSTM layer $D_{wl}$, the Sentence LSTM layer $D_{sl}$, the Entity LSTM layer $D_{el}$ and the Attention LSTM layer $D_{ml}$ are all set to 300.
The hidden dimension of the classification layer $D_h$ is 50. The margin $\Delta$ in the hinge-ranking loss for the entity category classification is set to 5.
The ELMo scale parameter $\gamma$ used in the Detector is 3.35 and 3.05 in the Classifier, respectively.

\begin{table}[htbp!]
\centering
\resizebox{\linewidth}{!}
\begin{tabular}{l|ccc|ccc}
\multirow{2}{*}{\textbf{MODEL}} & \multicolumn{3}{c|}{\textbf{ACE-2004}} & \multicolumn{3}{c}{\textbf{ACE-2005}}  \\
                  & \textbf{P}       & \textbf{R}         & \textbf{F1}   & \textbf{P}          & \textbf{R}         & \textbf{F1}            \\\hline
\citet{lu2015joint}&70.0&56.9&62.8&66.3&59.2 &62.5\\
\citet{lample2016neural}& 71.3 &50.5 & 58.3&   64.1 & 52.4& 57.6  \\
\citet{lu2017mention}&72.7&58.0&64.5
&69.1&58.1&63.1\\
\citet{xu2017local}    &68.2&54.3&60.5&67.4 & 55.1 & 60.6  \\
\citet{katiyar2018nested} & 73.6 &71.8&72.7 &70.6 & 70.4 & 70.5  \\
\citet{ju2018neural}  &-&-&-&74.2&70.3&72.2 \\
\citet{wang2018neuralshort}&74.9 & 71.8& 73.3
&74.5 &71.5 &73.0 \\
\citet{wang2018neural}  & 78.0& 72.4 &75.1          & 76.8     & 72.3   & 74.5  \\ \hline
{\textsc{MGNER}} w/o context &79.8&76.3&78.0& \textbf{79.6}&75.6&77.5\\
{\textsc{MGNER}} w/o attention &81.5&76.5&78.9& 79.4 &76.0 & 77.7 \\
{\textsc{MGNER}} &\textbf{81.7}&\textbf{77.4}&\textbf{79.5}&79.0&\textbf{77.3}&\textbf{78.2}\\
\end{tabular}
}
\caption{Performance on ACE-2004 and ACE-2005 test set for the Nested NER task.}
\label{multi_result}
\end{table}

\subsection{Results}
{\bf Nested NER Task}.
The proposed {\textsc{MGNER}} is very suitable for detecting nested named entities since every possible entity will be examined and classified. In order to validate this advantage, we compare {\textsc{MGNER}} with numerous baseline models: 1) \citet{lu2015joint} which propose the mention hypergraphs for recognizing overlapping entities; 2) \citet{lample2016neural} which adopt the LSTM-CRF stucture for sequence labelling; 3) \citet{lu2017mention} which introduce mention separators to tag gaps between words for recognizing overlapping mentions; 4) \citet{xu2017local} that propose a local detection method; 5) \citet{katiyar2018nested} which propose a hypergraph-based model using LSTM for learning feature representations; 6) \citet{ju2018neural} that use a layered model which extracts outer entities based on inner ones; 7) \citet{wang2018neuralshort} which propose a neural transition-based model that constructs nested mentions through a sequence of actions; 8) \citet{wang2018neural} which adopt a neural segmental hypergraph model.

Experiment results of the Nested NER task on the ACE-2004 and ACE-2005 datasets are reported in Table \ref{multi_result}.
We can observe from Table \ref{multi_result} that, our proposed framework {\textsc{MGNER}} outperforms all the baseline approaches.
For both datasets, our model improves the state-of-the-art result by around 4\% in terms of precision, recall, as well as the F1 score.

To study the contribution of different modules in {\textsc{MGNER}}, we also report the performance of two ablation variations of the proposed {\textsc{MGNER}} at the bottom of Table \ref{multi_result}.
{\textsc{MGNER}} w/o attention is a variation of {\textsc{MGNER}} which removes the self-attention mechanism and {\textsc{MGNER}} w/o context removes all the context information.
To remove the self-attention mechanism, we feed the context feature $\mathbf{C}$ directly into a bi-directional LSTM to obtain context representation $\mathbf{m}$, other than the attentive context vector $\mathbf{C}^{att}$. As for  {\textsc{MGNER}} w/o context, we only use entity representation $\mathbf{e}$ to do classification other than the context-aware entity representation $\mathbf{o}$.
By adding the context information, the F1 score improves 0.9\% on the ACE-2004 dataset and 0.7\% on the ACE-2005 dataset. The self-attention mechanism improves the F1 score by 0.6\% on the ACE-2004 dataset and 0.5\% on the ACE-2005 dataset.

\begin{table}[htbp!]
\centering
\resizebox{\linewidth}{!}
\begin{tabular}{l|ccc|ccc}
\multirow{2}{*}{\textbf{MODEL}} & \multicolumn{3}{c|}{\textbf{OVERLAPPING}} & \multicolumn{3}{c}{\textbf{NON-OVERLAPPING}} \\
 & \textbf{P}        & \textbf{R}        & \textbf{F1}        & \textbf{P}          & \textbf{R}         & \textbf{F1}       \\ \hline
 \citet{lu2015joint}&   68.1       &     52.6     &  59.4        &   64.1        &  65.1 & 64.6          \\
\citet{lu2017mention} &     70.4     &    55.0      &     61.8      &   67.2         &     63.4      &       65.2     \\
 \citet{wang2018neuralshort} &     77.4    &    70.5      &    73.8       &      76.1    &   69.6        &     72.7       \\
 \citet{wang2018neural} &    80.6     &    73.6      &    76.9       &       75.5     &     71.5      &   73.4     \\ \hline
{\textsc{MGNER}} &      \textbf{82.6}    &     \textbf{76.0}    &    \textbf{79.2}       &   \textbf{77.8}     &    \textbf{79.5}      &  \textbf{78.6}   \\
\end{tabular}
}
\caption{Results on different types of sentences (ACE-2005).}
\label{overlap}
\end{table}

To analyze how well our model performs on overlapping and non-overlapping entities, we split the test data into two portions: sentences with and without overlapping entities (follow the splits used by \citet{wang2018neural}). Four state-of-the-art nested NER models are compared with our proposed framework {\textsc{MGNER}} on the ACE-2005 dataset. As illustrated in Table \ref{overlap}, {\textsc{MGNER}} consistently performs better than the baselines on both portions, especially for the non-overlapping part. This observation indicates that our model can better recognize non-overlapping entities than previous nested NER models.

The first step in {\textsc{MGNER}} is to detect entity positions using the Detector, where the effectiveness of proposing correct entity candidates immediately affects the performance of the whole model. To this end, we provide the experiment results of detecting correct entities in the Detector module here. The precision, recall and F1 score are 85.23 , 91.84, 88.41 for the ACE-2004 dataset and 84.95, 89.35, 87.09 for the ACE-2005 dataset.

\begin{table}[htbp!]
\centering
\resizebox{\linewidth}{!}
\begin{tabular}{l|cc}
\multicolumn{1}{l|}{\multirow{2}{*}{\textbf{MODEL}}} & \multicolumn{2}{c}{\textbf{CoNLL-2003}}  \\
\multicolumn{1}{c|}{}      & \textbf{DEV}      &\textbf{TEST} \\ \hline
\citet{lu2015joint} & 89.2 & 83.8 \\
\citet{lu2017mention} & - & 84.3 \\
\citet{xu2017local} &  -  & 90.85 \\
\citet{wang2018neural} &-&90.2\\\hline
\citet{lample2016neural}    &  -  &  90.94\\
\citet{ma2016end}   &94.74&  91.21\\
\citet{chiu2016named} &$94.03\pm 0.23$ & $91.62 \pm 0.33$ \\
\citet{peters2017semi}  & - & $91.93 \pm 0.19$\\
\citet{peters2018deep} &- & $92.22 \pm 0.10$ \\
\hline
{\textsc{MGNER}} w/o context &$ 95.21 \pm 0.12$& $92.23 \pm 0.06 $\\
{\textsc{MGNER}} w/o attention&$ 95.23 \pm 0.06$& $92.26 \pm 0.09 $\\
{\textsc{MGNER}} &$ \textbf{95.24} \pm \textbf{0.13} $&$ \textbf{92.28} \pm \textbf{0.12} $\\
\end{tabular}
}
\caption{F1 scores on CoNLL-2003 devlopement set (DEV) and test set (TEST) for the English NER task. Mean and standard deviation across five runs are reported. Pos tags information are not used.}
\label{result}
\end{table}

{\bf NER Task}. We also evaluate the proposed {\textsc{MGNER}} framework on the NER task which needs to reorganize non-overlapping entities. Two types of baseline models are compared here: sequence labelling models which are designed specifically for non-overlapping NER task and nested NER models which also provide the ability to detect non-overlapping mentions. The first type of models including 1) \citet{lample2016neural} which adopt the LSTM-CRF structure; 2) \citet{ma2016end} which use a LSTM-CNNs-CRF architecture; 3) \citet{chiu2016named} which propose a CNN-LSTM-CRF model; 4) \citet{peters2017semi} which add semi-supervised language model embeddings; and 5) \citet{peters2018deep} which utilize the state-of-the-art ELMo language model embeddings. The second types include four Nested models mentioned in the Nested NER section: 1) \citet{luo2015joint}; 2) \citet{lu2017mention}; 3) \citet{xu2017local}; 4) \cite{wang2018neural}.

Table~\ref{result} shows the F1 scores of different approaches on CoNLL-2003 devlopement set and test set for the English NER task. Mean and standard deviation across five runs are reported.
It can be observed from Table~\ref{result} that the proposed {\textsc{MGNER}} model outperforms all the baselines. The models designed for non-overlapping entity detection usually performs better than Nested NER models for the NER task. Our proposed framework outperforms state-of-the-art results both on the NER and Nested NER task. \citet{xu2017local} is the best baseline model among the Nested models since it shares a similar idea of our proposed framework by individually examining each entity proposal.
From the ablation study, we can observe that by purely adding the context information, the F1 score on the CoNLL-2003 test set improves from 92.23 to 92.26, and by adding the attention mechanism, the F1 score improves to 92.28.

We also provide the performance of detecting non-overlapping entities in the Detector here. The precision, recall and F1 score are 95.33, 95.69 and 95.51 on the CoNLL-2003 dataset.
\section{Conclusions}
\vspace{-2mm}
In this work, we propose a novel neural framework named {\textsc{MGNER}} for {Multi-Grained} Named Entity Recognition where multiple entities or entity mentions in a sentence could be non-overlapping or totally nested.
{\textsc{MGNER}} is framework with high modularity and each component in {\textsc{MGNER}} can adopt a wide range of neural networks.
Experimental results show that {\textsc{MGNER}} is able to achieve state-of-the-art results on both nested NER task and traditional non-overlapping NER task.
\section*{Acknowledgments}
We thank the reviewers for their valuable comments.
Special thanks go to Lu Wei from Singapore University of Technology and Design for sharing the datasets split details.
This work is supported in part by NSF through grants IIS-1526499, IIS-1763325, and CNS-1626432.

redpajama_set_name: RedPajamaArXiv


I was part of a sangha where the teacher sexually abused several women and financially and emotionally abused all of us. The trust we placed in this teacher was compete and the devastation tragic for so many, including myself. It has taken me several years to get to the point where I would consider joining a sangha again. Unfortunately, there are still members of this group that continue to support this "roshi" and bring him into the community for "teachings" Not only does this violate the agreements he made to discontinue teaching, it violates the credibility of everyone he has harmed. His socio pathic personality involves more than "making a mistake", as this was not his first violation.
Thank you Kristin U. for bringing this up.
When communities fail to follow through on agreements or boundaries made in the wake of abuses, this can have a devastating effect. This magnifies an experience of abuse into institutional betrayal. It is a form of secondary wounding that, in my experience both personally and professionally, is very difficult to repair and often has some very long and very negative ripple effects. It destroys the bonds of trust that communities may be trying carefully to re-establish after having been shattered once already.
It is extremely important for communities' safety and well being that communities prioritize ensuring that such agreements are fulfilled. One way that can happen is just by caring more and understanding the importance, as well as being more precise and accountable. Another way can be to have less secrecy about agreements and boundaries, so that the whole community is involved in follow through and in creating a healthy atmosphere.
Abuse does not have to be sexual for abuse to be happening. Like the sexual, sometimes the abuse isn't even seen. Emotional, verbal, psychological, and physical violence are quite effective, too. The old phrase "street angel, house devil" can be applied to any perpetrator of the crazy-making situations which can blast a child's, or anyone's, self esteem and trust so much so that navigating the world is a horrendous experience, until that person's life is self-understood and work is done so that the person becomes self-rearing and in much better heartmind space, no longer identifying as "victim" but as someone who went through a particular set of experiences.
Change to the happier IS possible, after all kinds of abuse. I believe that and I do my work on my Self, and I am here, in some kind of recovery of the life I believe the Universe means me to be having. A good one, with joy. I now make the choices as to what I do with and how I respond to–rather than react to–the emotionally-laden memories and to what were the actual facts of my life.
Not one of the close-to-one-thousand people who came to my father's funeral, who told me how much they loved him, or who knew my mother, would believe what went on in our house every day of my formative years. But I know. And I am stronger because of who I AM, and what I do with me, today. I practice forgiveness (though I do not condone), and I AM grateful for the love, humour, intelligence, and authenticity with which I fill my life today, which seemed so lacking in the past, before I experienced real awareness. I continue to learn and apply useful tools for living, instead of what I had absorbed in that first and most unconscious, angry, wounded, and sorrowful home. Life was always good; NOW I know it, and I am worthy of it and I get better in it.
Patience, forgiveness, gratitude–This is my practice.
Such a strong message and so eloquently put! Thank you Pam. This is a recurring issue in many communities and it is particularly hard to address in large scale volunteer-based organizations.
On the occasions when I have brought up concerns about this issue, the response has often been a kind of blankness, and almost a bewilderment as to why I would be troubled by such things. Like, "When did you get to be such a prude?" Beyond simply denial,there was a sense of just not getting that this is a problem. Bringing all this into the public discourse is an essential first step. There is a need for a cultural shift as well as clearer policies and follow up.


text:
William Morison
䋼
Andrew Melville, by William Morison
The Project Gutenberg EBook of Andrew Melville, by William Morison This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org
Title: Andrew Melville Famous Scots Series
Author: William Morison
Release Date: July 30, 2007 [EBook #22174]
Character set encoding: ISO-8859-1
*** START OF THIS PROJECT GUTENBERG EBOOK ANDREW MELVILLE ***
Produced by Jordan, Josephine Paolucci and the Online Distributed Proofreading Team at http://www.pgdp.net
FAMOUS SCOTS: SERIES
PUBLISHED BY OLIPHANT ANDERSON FERRIER EDINBVRGH AND LONDON
The designs and ornaments of this volume are by Mr. Joseph Brown, and the printing from the press of Messrs. T. and A. Constable, Edinburgh.
Transcriber's notes: Minor typos have been corrected. Footnotes have been placed at the end of the paragraph to which they refer. Greek has been changed to Latin letters and placed in brackets.
PREFATORY NOTE
Let it be understood that the quotations in Scots, where the author is not mentioned, are from the Autobiography and Diary of James Melville.
INTRODUCTORY 9
BIRTH--EDUCATION--YEARS ABROAD 15
SERVICES TO SCOTTISH EDUCATION--PRINCIPALSHIP OF GLASGOW AND ST. ANDREWS 23
THE 'DINGING DOWN' OF THE BISHOPS--MELVILLE AND MORTON 31
THE 'BIGGING UP' OF THE BISHOPS UNDER LENNOX AND ARRAN--MELVILLE'S FLIGHT TO ENGLAND 43
THE KING'S SURRENDER TO THE CHURCH 56
THE POPISH LORDS--MELVILLE AND THE KING AT FALKLAND PALACE 71
THE KING'S GREEK GIFT TO THE CHURCH 93
MELVILLE AT HAMPTON COURT 116
THE KING'S ASSEMBLIES 134
THE TOWER: SEDAN 140
While Andrew Melville has other claims on the lasting honour of his countrymen than the part he took in securing for Scotland the ecclesiastical system which has been the most powerful factor in her history, it may be held as certain that where this service which filled his life is disesteemed, his biography, if read at all, will be read with only a languid interest. It will be our first endeavour, therefore, to show that such a prejudice in regard to our subject is mistaken and misleading.
Melville, and all from first to last who joined in the Scottish resistance to Episcopacy, were persuaded that the controversy in which they were engaged was one not academic merely but vital, and that, as it was settled one way or the other, so would the people be left in a position in which they would be able to develop their religious life with freedom and effect, or in one which would incalculably cripple it. That is a contention which history has amply vindicated.
The best justification of the struggle carried on during the period from Melville to the Revolution (1574-1688) to preserve the Presbyterian system in the Church is to be found in the benefits which that system has conferred upon the country. It has penetrated the whole Christian people with a sense of their individual responsibility in connection with the principles and government of the Church; it has saved the Church from being dwarfed into a mere clerical corporation; it has laid for it a broad and strong basis by winning to it the attachment of its common members, and by exercising their intelligence, sympathy, and interest in regard to all its institutions and enterprises. It may be truly said of the Scottish people that their highest patriotism has been elicited and exercised over the religious problems of the nation; that they have shown more sensitiveness concerning their religious rights, liberties, and duties than concerning any other interest of their life; and that they have been more readily and deeply touched when the honour and efficiency of their Church was at stake than by any other cause whatever. How should an ecclesiastical system better vindicate its claim? Nothing so ennobles a people as the care of matters of high concern--such a care as Presbyterianism has laid on the Scottish people.
But it was not only the conviction of the excellence of their own economy that led the Presbyterians to maintain it at all hazards--it was also their fear of many tendencies in the rival system. They dreaded that the imposition of Episcopacy would ultimately undo the work of the Reformation, and bring the nation once more under the yoke of Rome. Here, too, history has justified them. Had it not been for the conjunction of the forces of the Scottish Presbyterians and the English Puritans during the reign of Charles the First, the designs of that monarch against the Protestantism of both kingdoms could not probably have been checked. The least that can be said with truth on this matter is, that the Protestantism of the country was gravely imperilled in his reign and in the reigns of his two immediate successors, and that the resolute attitude of Scotland counted

redpajama_set_name: RedPajamaCommonCrawl


text:
Support the Underground nightclub during Covid 19
Margaret Schlamp
Joe Solinap
Tyler Brilinski
Mark Alway is organizing this fundraiser on behalf of Marc Pospisil.
Seattle Trance Family (STF) time to make PLUR a reality!
The State of Washington has restricted all large gatherings for the safety of the entire population because of the COVID19 outbreak. Your favorite nightclub, the Underground , at 122 S Washington St. in Seattle WA has just announced we've had to cancel at least 4 events with world class talent and DJs.
Marc Pospisil, the promoter and STF "father figure" has been an inspiration and God for trance and electronic music in Seattle. He has branded the Seattle Trance Alliance , Elements Seattle, and Sunset Sessions. He has brought talent from around the world to the Number 1 Ranked EDM club in Seattle .
He has had to cancel 4 shows for the month of March (to be rescheduled) and employs a staff team that no longer has a source of income.
We are seeking to raise $12,000 to help cover the costs and expenses while not being able to run events. And as many of you, our family, know the staff at the Underground is amazing. Think of Jason who works security, Malissa at the door, and Donny who keeps everything runny smoothly. DJ contracts have already been paid and we are thankful to the agencies for allowing us to reschedule the talent, however, the club does not get a refund or a way to recoup that operating capital until the shows are rescheduled and ticket sales begin again. In the mean time, without the operating capital, the Underground would struggle to book new talent. Additionally, the staff that normally work these shows won't be getting a paycheck. In cooperation with Marc, we would like to pay all the staff what they would normally receive for working the 4 cancelled shows and re-build the capital cash necessary to book new talent while that money is locked away in the rescheduled contracts. Any funds raised above the goal amount could be used for a special STF "Family night" discounted show to support the scene and family.
While all tickets are being refunded, we are asking you to donate the amount of your ticket to help ensure the cancelations don't have a negative impact on Marc's ability to acquire world class talent and to ensure the staff can be cared for. Let's all pitch in a little bit so that no one person takes the brunt of the financial impact.
Made by Mark Alway, known as the "Dancing Photographer" and staff member of the Underground for the last 3.5 years.
Organizer and beneficiary
Mark Alway
Marc Pospisil
Created March 12, 2020


text:
Congresswoman Demands Answers Over LASD Rough Arrests Caught on Video
By City News Service Jun 5, 2020
COMPTON (CNS) - Videos of recent rough arrests in Compton and Lynwood involving Los Angeles County sheriff's deputies are under investigation by the department and drew criticism today from the congresswoman who represents both communities.
In one video, a deputy pulls over a man in Lynwood for a traffic violation, then pulls him out of the vehicle after the man defiantly refuses to comply with the deputy's orders.
The deputy can be seen punching the driver while inside the vehicle, then pulling him out and taking him to the ground where he was taken into custody.
The driver was arrested on suspicion of assault with a deadly weapon on a police officer, resisting arrest and driving without a license. A passenger was arrested for failure to comply with a lawful order and another passenger was arrested for possession of methamphetamine for sales.
A second deputy suffered a broken arm while carrying out the arrest, Sheriff Alex Villanueva said.
In the Compton incident, two deputies are seen in a short video pinning down an alleged looter and, while trying to take him into custody, a third deputy comes in and begins kneeing the suspect in the ribs. What led up to the suspect being on the ground was not shown in the video.
Rep. Nanette Diaz Barragán, whose 44th District includes Compton and Lynwood, called the videos "disturbing'' and demanded answers from the department.
"While we do not yet have all of the information or know the full extent of the circumstances around either incident, the videos suggest that there was unnecessary and excessive use of force used in both incidents by law enforcement officers,'' said Diaz Barragán, D-San Pedro.
She said she has spoken with the heads of both the Compton and Lynwood stations and asked for more information and answers regarding each incident.
Both incidents are under an expedited review by the department, Diaz Barragán said.
"I made it clear that I will continue to demand answers and accountability as these investigations continue,'' she said.
In response to the Compton video, Villanueva said on Wednesday "This is another example of why we so desperately need the (Los Angeles County Board of Supervisors) to fully fund (the sheriff's department's) body-worn camera project. The public deserves full transparency.''
He also urged anyone with footage from the Compton incident to contact the department.


text:
Polish Falcon Society of Depew, New York, Inc.
On November 15, 1903 eight young men organized the Polish Falcon Society Under its first President Anthony Krzywicki, these men had as their goal the formation of the Poles into a strong, effective group for the preservation of the highest ideals of their national heritage.
Initially, the Society met at Matthew Dworzanowski's Hall and, in 1906, moved their meetings to SS. Peter and Paul's School Hall. On June 25, 1911, the blessing of the Society's Banner by the Rev. Stanley Fimowicz, Pastor of SS. Peter and Paul, took place and in September of that year, the Zlot or Convention of the Third Circuit of the Polish Falcon Alliance, was held in Depew. In 1919 the Falcons purchased their own clubrooms at 45 Crane Street by pooling their financial resources. The Society was incorporated in 1922 and by 1925, the building was debt free.
Through the years the Falcons sponsored a program of physical training and were well known for their baseball, basketball, bowling and wrestling teams. Dramatics, social events and pinochle tournaments have also been a part of a vigorous program of members participation.
At one time a fifty-four member ladies' auxiliary called the Polish Falconette's existed, but have since disbanded. A male choir, under the direction of John Nowak, and the Polish Falcon Band, under the leadership of Walter Witkowski, enlivened many picnics and provided entertainment for church functions as well as civic affairs.
In 1951 the clubrooms on Crane Street were enlarged and remodeled and renovation of the hall and upstairs rooms completed in 1953. The 1960's saw an increase in membership and a growing need for a larger and more modern facility. In late 1970 ground was broken for the present clubroom at 445 Columbia which were formally opened on September 11, 1971. This move has enabled the Society to further its involvement in the community.
The establishment of a regular bingo night has enabled the Falcons to fund scholarship awards and lend financial support to area fire departments, senior citizen centers, youth and other various cultural and charitable organizations.
The Falcons have continued to preserve and promote their Polish heritage. They started one of the original Dyngus Day celebrations in 1973 and host numerous dances throughout the years. The Falcons have also reached out to their non-polish neighbors through such social events as singles dances, car cruise night, chicken BBQ dinner, tailgate parties, golf outings, and their popular Friday night fish fry dinners. We continue to celebrate Dyngus Day as a traditional Polish event to keep with our Polish Heritage.
On March 5, 1984 forty-five women, all wives of Society members had their first official meeting as the Polish Falcons Ladies Auxiliary. Their purpose was to assist the Society in all its endeavors. Through its numerous fund raisers, such as the annual chicken BBQ, steak dinner, Vera Bradley Bingo, Chinese auction, they have contributed to the upkeep and remodeling of the club rooms.
Additionally, the Auxiliary has donated its time and support to Religious and Civic organizations, charity events as well as needy individuals and families.
The past has seen growth in membership and increased community involvement. In order to better serve this commitment, the size of the hall was doubled in 1978 and the interior completely remodeled and updated in 1994. These improvements have allowed the Falcons to welcome their membership and the public to a modern and attractive facility.
We thank all the officers and members whose time and dedication have guided us. With that in mind, the future success and growth of the Society is ensured.


Here you'll find the best way to contact us.
Should you wish to contact us for any reason, please feel free to email us at the below address.
We are a financial services group focused on retail and commercial customers.
As a responsible business we can differentiate ourselves from our competitors.
Before you apply, take a look at the hints and tips provided.


myappsforpc.org - On this article we recommend you 14 images about Hausfassade Beige Taupe that we have collected from any source about Wohndesign. Traumhaus Design, and of course what we provide is the most best of images for Fassade Des Hauses Beige Und Taupe Farben 20121004164440 Hausfassade With. If you like images on our website, please do not hesitate to visit again and get inspiration about Traumhaus Design of our images collection.
And if you want to see more images more we recommend the gallery below, you can see the picture as a reference image about Hausfassade Beige Taupe.
Thank you for seeing gallery of Hausfassade Beige Taupe, we would be very happy if you come back.


Dancer Bio. Please enter your dancing history and details.
Bridal Bio. Please enter your bridal history and details.
Lingerie Bio. Please enter your lingerie history and details.
Action Bio. Please enter your action history and details.
Actor Bio. Please enter your acting history and details.
Performer Bio. Please enter your performing history and details.
Model Talents. Please let us know about your special abilities.


plus size wedding guest dresses and cheap mother of the bride dresses designers ideas on Newly Engaged Use Our Wedding Dress Finder To Locate Your Dream Elegant Present 0 in high resolution for little black dress discount online rental used trends 2018 plus size wedding dresses. This wallpaper women dresses for weddings was added in Sunday, October 21, 2018 and 21 users has viewed and downloaded this dresses for women. Newly Engaged Use Our Wedding Dress Finder To Locate Your Dream Elegant Present 0 have and approximately Unknown Size bandwith was consumed. If you like it, all you need to do is help us grow by sharing this plus size prom dresses wallpapers Newly Engaged Use Our Wedding Dress Finder To Locate Your Dream Elegant Present 0.


Our facilitators including our Creative Director create and deliver inspiring and engaging workshops. Here's the thoughts of some people who we've had the pleasure of working with.
"Thank you for providing engaging activities which were enjoyed by all age groups. Your enthusiasm rubbed off onto the kids and they all commented on how much fun they had.
"Creative Spotlight is all about empowering youth and young people and taking them out of their comfort zone and introducing or enhancing their skills. Monique has never done any performing arts or been in the spotlight before, she's quiet and quite reserved. They took the children and completely turned them around, my daughter had totally come out of her comfort zone by the end of the weeks summer camp. It ran for five days where the children are propelled into numerous different roles, such as stage hands, actors production and lighting, I was amazed at the level of professionalism considering it was just a week. The show was awesome, they'd learnt all their lines and cues to the music, it ran like clockwork, I can't believe it had all been achieved in five days. Monique was on stage acting, dancing and singing, I never thought I'd see that, she normally wouldn't be in the spotlight! Monique loves it; she really looks forward to doing it in the summer. All I can do is sing their praises, they're a really good team of professionals who and are excellent with the kids.
"The smaller number in each group allowed each pupil to access the information that was being delivered. It gives a practical approach for young people to understand the importance of careers and in presenting themselves to prospective employers in the correct manner.
" I thought the content and delivery were superb. The pupils were engaged throughout and clearly thrived in the task oriented sessions. The facilitators were excellent and kept the pace high throughout the sessions. The content was pitched perfectly for our pupils and was accessible yet challenging. I was really impressed.
"Thank you so much for taking part in our Careers Immersion days for Year 10. Your Work-It workshop was well received by our students, and the positive feedback on the evaluation sheets confirmed this.
Our intention was to make these two days aspirational so that our students would be encouraged to look ahead and set themselves ambitious goals. Your contribution certainly helped them to draw some useful conclusions about how they need to approach the next stage of their education, in preparation for future employment.


text:
03 - Introduction to C++ Variables
S C++ Tutorial
01 - Introduction to Object-Oriented Programming
02 - First C++ Program
04 - Introduction to C++ Functions
05 - C++ Classes
06 - C++ Objects
07 - C++ Constructors
08 - C++ Inheritance
09 - C++ Polymorphism
10 - C++ Control Flow Structures
11 - C++ Arrays
12 - C++ Pointers
01 Variables Introduction
In computing, you need to store and retrieve data. Data is stored in the memory of the computer. To retrieve data, you need to know where the data is stored in the memory. Variables help you access data stored in the memory. You can think of them as placeholders.
Let's look at an example to understand the concept of variables:
In arithmetic, let's consider you are given the following equation:
z=x+y; where x=10 and y=5
In the expression z=x+y, you substitute 10 in place of x and 5 in place of y.
z= 10+5, which is 15
Thus, x, y, and z are place holders.
In programming, these place holders are referred to as variables.
In C++ programming, a variable is a name given to a memory location. The value assigned toa variable is stored in the memory allocated to that variable.
A diagrammatic representation of the concept of variables stored in memory is as follows:
Another example, in programming, is of an employee database. An employee database contains data such as, Employee Name, Age, Salary, and Date of Joining and so on. In C++, to store this data, you would declare the following as variables:
EmployeeName
DateofJoining
1.1 Declaration Statement
A declaration statement specifies the variable name and its data type. In C++, you must declare a variable before you can use it in your program. Based on the data type of the variable, memory is allocated.
Let's say you want to add two whole numbers and display the result onthe screen. In this case, you need to store three values – the two numbers that you want to add and their sum. Thus, you need three variables; sum= x+y
In C++, you declare the variables as follows:
int x; // declaration statement
int y; //declaration statement
int sum; //declaration statement
1.1.1 Declaring Multiple Variables
You can declare multiple variables in the same line as follows:
int x, y, sum;
1.2 Assignment Statement
An assignment statement is used to initialize variables. For example,
x=5;
y=7;
Let's say that for an employee database application, you need to set the basic salary to specific value. In C++, you declare a variable that stores the basic salary, and assign it the required value as follows:
int basicsalary; //declaration Statement
basicsalary = 5000; //assignment statement
Alternatively, you can also write the following:
int basicsalary = 5000;
In the above statement, we have declared a variable and assigned a value to it the same line.
1.2.1 Unassigned Variables
A variable that has been declared but not assigned a value is known as an unassigned variable. If you use such a variable in an expression, then the compiler throws an error.
Let's look at an example:
int basicsalary;
int bonus;
int totalsalary
basicsalary = 5000;
totalsalary= basicsalary+bonus;
In the above statements, the variable bonus is an unassigned variable, and will throw a compile-time error.
Thus, it's a good practice to assign values to variables whenever they are declared.
1.3 Variable Scope
In C++, you can declare variables anywhere in the program. The scope of a variable determines for how long the variable exists in the memory of the computer. If a variable doesn't exist, you obviously can't use it in your program.
Based on scope, variables are divided into two categories:
1.3.1 Local Variables
A variable that is declared within a function is known as a local variable. Such a variable exists in the memory as long as the function is being executed. Once the execution of the function is complete, the variable no longer exists in the memory.
If you try to reference a local variable outside the function it was declared in, it generates an error.
Let's look at C++ program in which you create a function that prints your name.
void displayname()
string name;         // local variable
name = Julia;
cout<<"My name is<<name<<endl;
#include <iostream.h>
displayname();
The output of the above program is as follows:
My name is Julia
In this the variable name is a local variable because it has been declared within the function displayname.
Now, let's try to use the variable name in the main function:
displyname();
cout<<"Access local variable<<name<<endl;
The cout statement will throw an error. This is because, after the execution of the displayname function, the variable name no longer exists in the memory. Hence, the compiler cannot locate the variable name and throws an error.
1.3.2 Global Variables
A variable that is declared in main() is known as global variable. Such a variable exits throughout the execution of a program. In addition, other functions within main() can reference a global variable.Let's look at C++ program in which you create a function that prints your name. However,in this program, you will declare the variable name as a global variable.
{    string name;
cout<<"Accessing global variable"<<name<<endl;
In the above program, the variable name is a global variable. This is because the variable has been declared in main and not within the displayname() function. The function displayname() can access the variable name. Also the cout statement does not generate any error, because the variable exists in the memory even after the execution of the displayname() function.
1.4 C++ Program Example
A program in C++, to add two integers (whole numbers) is as follows:
int x;        //variable declaration
x=5;            //assignment statement
y=19;
sum= x+y;        //adding the values stored and storing                 the result
cout<<sum<<endl;
Understanding the statements in the above program:
The above statements are declaration statements.
For more information, see Declaration Statement.
sum=x+y;
The above statements are assignment statements. The variable sum stores the result of addition. The value stored in sum is 24 (9+15).
cout<<sum<<endl
The cout statement displays the value stored in the variable sum on the screen.
1.5 Variable sizes and the size of operator


text:
Das George Mason Memorial ist ein National Memorial im West Potomac Park in Washington, D.C., der Hauptstadt der Vereinigten Staaten.

Es gedenkt des Wirkens von George Mason, einem wichtigen Gründervater der Vereinigten Staaten, der die Virginia Declaration of Rights schrieb und als Delegierter an der Constitutional Convention im Jahr 1787 in Philadelphia teilnahm, aber oft als "vergessener Gründer" angesehen wird.

Mason, ein Anti-Föderalist, hat die Verfassung der Vereinigten Staaten nicht unterschrieben, da diese den Sklavenhandel nicht abgeschafft hat und, seiner Meinung nach, den einzelnen Bürger nicht ausreichend vor der Regierung schützte. Er war auch als der "zögernde Politiker" bekannt, so war auch der Titel der von Robert A. Rutland verfassten Biographie.

Das Denkmal wurde durch das Gesetz 101-358 am 10. August 1990 beschlossen. Ein Grundstück nahe dem Jefferson Memorial wurde ausgewählt. Das Denkmal besteht aus einer 22 m langen Mauer mit einer überlebensgroßen Statue, die Mason sitzend mit übergeschlagenen Beinen zeigt und einem runden Wasserbecken. Der Architekt war Faye B. Harwell und der Bildhauer war Wendy M. Ross. Der erste Spatenstich war am 18. Oktober 2000. Das Denkmal wurde am 9. April 2002 eingeweiht.

Das George Mason Memorial wird vom National Park Service verwaltet.

Weblinks

 Offizielle NPS Webseite
 Mason Spirit Article, Winter 2001 (GMU Alumni magazine): George Mason Memorial Garden to Be Built near the National Mall

National Memorial der Vereinigten Staaten
Denkmal in Washington, D.C.
Potomac River

redpajama_set_name: RedPajamaWikipedia


text:
Help Me With My Order
Ticketing For Your Event
G Jones - Minneapolis, MN
Saturday, 1 December 2018, 8:00pm . The Cabooze , Minneapolis	[All ages]
GetTickets
US$ 28.25	(incl. Fees)
Merch Only
Sales for this event are closed.
'The Ineffable Truth'
G Jones 2018 - 2019 Tour
Greg Jones is the artist behind the enigmatic G JONES moniker- a California native who cut his teeth in the underground bass music scene in Santa Cruz and the Bay Area in the early 10's. A lauded master of electronic music production and highly sought after performer, G Jones is a frequent collaborator of legendary artists including Bassnectar, Eprom and DJ Shadow, who called Jones "the most gifted Ableton beatmaker I've ever seen." Rolling Stone touted him a "studio wizard." Billboard Magazine had this to say about his album's first single, "understanding the possibility": "That's what listening to G Jones will do. It's a mental minefield of sudden drops and stutters, bass lines that open holes under your feet and send you shooting sideways to some warped mystery land."
Pre-sale tickets and exclusive merchandise available from this page only, starting 10am Pacific time, Tuesday 24 July 2018, exclusively for those who are pre-registered.
All Tour Dates:
Help Privacy Policy Refund Policy Use us for your Event Ticketing
Events in Minneapolis
©2021 TheTicketFairy®. Live for Live™.


At DCD we offer all aspects of decor and floral for any event. Linens, Chair Decor, Wedding Floral, Centerpieces, Backdrops, and more. You name it and your wish is our command! Click here to view our rates. While our rates for centerpieces and floral do vary, please contact us to inquire an estimate of your vision for your event.
Copyright 2013. Divine Creations. All Rights Reserved.


Hotel Debowy Biowellness & SPA offers a wide range of taste and catering inspirations for your wedding.
You can start with a cocktail party with canapé and end with a grand party the day after the ceremony.
Your happiness is our ultimate priority that is why we will prepare your wedding night in the luxury suite for newlyweds as a wedding gift from the Hotel. Another present is a tasting wedding menu for free while you are considering our venue for your wedding and we are sure it will confirm that Dębowy Hotel is your best choice.
Your Guests satisfaction will complement your happiness!
Upon arrival, your Guests will discover a special present from the hotel waiting for them in the room and plenty of attractions during the wedding and the after-party.
A stress-free wedding? Only in Hotel Debowy Biowellness & SPA!
Trust us with your wedding ceremony and be a guest at your own reception.
We are a team of experienced professionals with over 1000 weddings carried out so far!
We will do the ultimate so that on your great day your wishes are fulfilled.
Our wedding consultant will work with you every step of the way to plan your day to the last details so that you can fully ease out and focus on each other and your guest on the day.
You will be in the spotlight all the time!
You will get a stand for your dress, a separate room on the wedding day for make-up, styling hair for the bride and her maids. Why not indulge on some relaxation before the wedding? For you, maid of honor or mother or mother-in-law?
We will make your wishes come true!
Even greatest heroes need some support from friends on this special day!
A stag night in an interesting setting, for real men a sauna treatment in a barrel the day after the ceremony, or a whisky cocktail for Groom's friends – to name a few of our suggestions!
Breathtaking views and unforgettable ceremony in natural surrounding are at your fingertips!
If you dream of a wedding in a park, we will help you make it happen. Just name your dreams and we will find a way to put them into practice.


When you purchase a dedicated server, you are actually leasing a server box that is configured and set up according to your preferences, but remains at our data center. A dedicated server account provides you with a dedicated IP address and full control of server usage and software installation, with system admin or root access to the server.
1. This type of hosting is more reliable. The website will have more up-time because it is not sharing its bandwidth with any other websites. Therefore, the website owner can be sure that the website's visitors will never have any problem with bandwidth.
4. Dedicated hosting offers the client better security. A dedicated server is not shared with any other website. Therefore, it is more difficult for any other person to gain access to files, passwords or information. This fact, combined with security patches and regular updates, makes this one of the most secure hosting options.
Contact our sales team for dedicated server pricing to meet your needs. We can accomodate many scenarios to deliver you a seamless solution.


text:
Indian shadow bank Altico says evaluating options after default
MUMBAI (Reuters) - India's Altico Capital said on Sunday it was evaluating options to resolve its liquidity issues, following its recent default on an interest payment that was due earlier this month.
Altico, backed by Clearwater Capital Partners and Abu Dhabi Investment Council, said it defaulted on the interest payment after a rating downgrade prompted several of its lenders to recall a substantial amount of debt causing it to face a liquidity squeeze.
"This lack of capacity led Altico to pause further debt repayments in order to maintain itself as a going concern and to protect the interests of all stakeholders," the company said in a statement on Sunday.
Altico said it needed time from stakeholders to craft a plan that might "maximize asset value" and called on all stakeholders to "refrain from actions that exacerbate the liquidity problem."
"Altico has held a series of meetings with its creditors, with a view to agreeing a way forward that the company expects will allow it to meet its financial commitments," it said, adding that it had appointed financial and legal advisors to support the discussions.
Non-banking financial companies and housing finance companies have been stung by liquidity crisis after the collapse of behemoth Infrastructure Leasing & Financial Services Ltd late last year.
That has led to a contagion effect on players in the shadow banking sector with funding drying up and borrowing costs surging, and, experts say, is partly responsible for the ongoing economic slowdown in the country.
Reporting by Abhirup Roy; editing by David Evans


text:
Au cœur de l'orage est un documentaire français, réalisé en 1944-45 par Jean-Paul Le Chanois, sorti en 1948.

Le film évoque l'histoire de la Résistance dans le maquis du Vercors.

Conditions de tournage
Le film a été réalisé avec du matériel (prise de vues, pellicule) qu'il a dérobé, en août 1943, à la société Continental-Films dirigée par Alfred Greven, où il travaillait comme scénariste. Il avait constitué, dès 1941, au sein de cette société allemande de production de films français, un noyau du Comité de libération. Sur dénonciation, il était recherché par la police française et avait donc décidé de disparaître de Paris. Il a utilisé ce matériel pour filmer des maquisards du Vercors (ce qui deviendra son film, Au cœur de l'orage), ainsi que des événements de la libération de Paris.

Fiche technique
 Titre : Au cœur de l'orage
 Réalisation : Jean-Paul Le Chanois
 Photographie : Jean Bourgoin et Jacques Lemare
 Son : Louis Hochet et René Sarazin
 Montage : Emma Le Chanois
 Musique : Tibor Harsányi
 Production : Coopérative générale du cinéma français
 Pays d'origine :
 Format : Noir et blanc - 1,37:1
 Genre : documentaire
 Date de sortie :

Distribution
Narrateurs
 Jean Chevrier
 Jean Daurand
 Christiane Sertilange
 Hubner
 Kroneger
 Schiray

Notes et références

Bibliographie
 Gilles Vergnon, Le Vercors. Histoire et mémoire d'un maquis, Éditions de l'Atelier, 2002, p. 162-163

Liens externes

 Au cœur de l'orage sur Autour du  mai.fr

 Affiche

Film français sorti en 1948
Film français en noir et blanc
Film documentaire français
Film documentaire sur la Seconde Guerre mondiale
Film de la Coopérative générale du cinéma français


STUDY DESIGN: A 3-year prospective, cohort study. OBJECTIVES: To compare the incidence and risk factors of dysphagia after anterior cervical (AC), posterior cervical (PC), and posterior lumbar (PL) spine procedures. SUMMARY OF BACKGROUND DATA: Dysphagia is a known risk of AC surgery; however, comprehensive postoperative swallow evaluations have not been performed for a comparative cohort of AC, PC, and PL surgery patients. METHODS: Eighty-three patients were enrolled in the study, including 38 undergoing AC, 19 PC, and 26 PL procedures. Preoperative and postoperative swallowing evaluations were performed by questioning for subjective swallowing complaints and performing objective radiographic examination. Patients with severe dysphagia leading to an increased risk of aspiration were identified and treated until recovery or for 3 to 9 months. RESULTS: Comparison of preoperative and postoperative swallowing complaints revealed a significant increase for AC patients (P < 0.01) and a trend for PC (P = 0.06) and PL (P = 0.09) patients. Eighteen (47%) AC, 4 (21%) PC, but no PL patients demonstrated dysphagia on postoperative videofluoroscopic swallow evaluation. Age (>60 years, P < 0.01) was associated with increased risk of radiologic evidence of dysphagia. Surgical level, instrumentation, operative time, and presence of myelopathy or other comorbidities were not. Over 70% (12 of 17) of AC patients with dysphagia followed recovered within 2 months, while 23% (4 of 17) required some level of compensatory swallowing behavior up to 10 months following surgery. CONCLUSION: Dysphagia is a common occurrence after AC procedures but was also found after PC procedures. Intubation alone was not a risk factor for postoperative dysphagia in this cohort.


"Overall very positive. Robbie knew he was going to be running behind schedule and let me know ahead of time so there were no surprises. He was able to identify the issue I was having quickly and use the parts/material I had already purchased to complete the job. Very satisfied with his work and will be requesting him if services are needed in the future. "
"I called because I had a leak from my upstairs shower through the ceiling below. After cutting a hole in my ceiling, he told me it was a tile issue, not a plumbing issue. I paid $400 to have my tile caulked and sealed only to find that there was still a leak. My tile guy came back and found the leak was due to a hair clog in the shower drain that was causing water to pool and run over the lip and leak. Now I have to pay to have my ceiling repaired due to the hole the plumber put in it. Checking the drain should have been the first step. Even my tile guy said "I'm disappointed in your plumber". My hair clog has cost me over $1000"
"all went well. thanks for your help. "
"The technician was very courteous and knowledgeable and I didn't like he was trying to sell me something I didn't need. I will call your company for future hvac needs."
"All went very well. Thanks for your service."
"This was the second visit for Jacob Wright (for different problems) and again, he was able to determine the problem and get it fixed."
"Jacob Wright showed up right on time. He was able to determine the problem and get it fixed in a very timely manner. He was very pleasant and has excellent communication skills."