text:
The Fix Analysis
BET founder says Democrats are going too far left to beat Trump, but some politicos say he's out of touch
Robert Johnson, founder of the Black Entertainment Television network, became the first African American billionaire after he sold the company to Viacom in 2001. (Amanda Voisard for The Washington Post)
By Eugene Scott
Eugene Scott
Reporter covering identity politics for The Fix
Few demographic groups are more faithful to the Democratic Party than black Americans. But one of the first black billionaires is suggesting that the party has moved so far to the left, it risks alienating them.
While Robert Johnson played a significant role in shaping black culture when it comes to entertainment, some politicos say he is out of touch about the politics of black Americans.
"The party in my opinion, for me personally, has moved too far to the left," Johnson, the founder of the Black Entertainment Television cable network, told CNBC's Hadley Gamble on Tuesday. "And for that reason, I don't have a particular candidate [I'm supporting] in the party at this time."
"I think at the end of the day, if a Democrat is going to beat Trump, then that person, he or she, will have to move to the center, and you can't wait too long to do that," the lifelong Democrat added.
Johnson's belief that a centrist Democrat would have the best chance of beating President Trump is a common one, even among some black voters. Former vice president Joe Biden has been leading the Democratic field, including with black voters, for multiple reasons; among them is the belief that he has the best shot at defeating Trump.
Black Americans overwhelmingly back Democrats, even though most black Americans don't consider themselves liberal. According to a 2018 Associated Press/NORC Center poll, more than a quarter (27 percent) of black voters identify as conservative, and 44 percent consider themselves moderate. Only 26 percent identify as liberal.
While Johnson backed Democratic nominee Hillary Clinton in 2016, he's since made headlines for praising Trump's economic policies. And it's his view that the current economy is helping black Americans.
"I think the economy is doing great, and it's reaching populations that heretofore had very bad problems in terms of jobs and employments and the opportunities that come with employment," Johnson told CNBC, commenting on the black unemployment rate reaching its lowest level earlier this year.
"I give the president a lot of credit for moving the economy in a positive direction that's benefiting a large amount of Americans," he added. "I think the tax cuts clearly helped stimulate the economy. I think business people have more confidence in the way the economy is going."
But what Johnson fails to recognize is that most black Americans are not business people, said LaTosha Brown, co-founder of Black Voters Matter, an organization focused on increasing black voter turnout.
"Bob Johnson is not working class. He does not reflect the issue, nor does he even seem like he has the ability to speak to the issues of the working class," Brown told The Fix. "For him, to make a statement that this tax break has been helpful for black people — where has he been? Under a rock? There's all kind of reports that have come out that this tax benefit disproportionately benefited the wealthy and not the working class. In and of itself, to make that statement says to me that he's simply out of touch."
While many black voters disapprove of Trump's job performance because they disagree with his behavior and his views on issues related to race, many also view his presidency poorly because the economic policies Johnson praises have not helped black Americans as much as the president claims, said Adrianne Shropshire, executive director of the Black Pac, a group working to increase black Americans' political engagement.
"Mr. Johnson is an outlier within the African American community when it comes to the points that he raises," Shropshire said.
"Frankly the policy positions of the current Democratic field of candidates is aligned with the issues and priorities of those voters. So while Mr. Johnson may share the interests of millionaires and billionaires, he's out of step with black voters."
Ultimately, the 73-year-old Johnson may have views of the Democratic Party that are consistent with those of black Americans like himself: affluent, older and moderate. However, black Americans, and thus black voters, are far more ideologically diverse than they are often portrayed to be.
"We do know that older demographics in the community tend to support kind of the establishment Democratic Party," Eddie Glaude, chair of Princeton University's African American studies department, told The Fix. "Some are centrist. In 2016 in South Carolina, the broad support was for Hillary Clinton. We see that in the data.
"But we do know that even in South Carolina in 2016, we saw younger black voters breaking differently and voting differently. We see them talk about Medicare-for-all, a livable wage, not just criminal justice reform, but in some cases [prison] abolition. We see folks talking about the Green New Deal and environmental racism. When we look at the Movement for Black Lives, it's … to the left of what we might see establishment black Democrats backing."
Johnson may not yet know which candidate he's backing in 2020. If he rejects the Democratic nominee for Trump, he will almost certainly be an outlier among black voters. Republicans have won only about 10 percent of the black vote on average in the past 12 presidential elections, according to exit polls.
Most Read Politics
'This is the agenda of white nationalists': Four minority congresswomen condemn Trump's racist remarks
Trump says they 'hate our country.' The Democrats he attacked say the country 'belongs to everyone.'
Analysis Pelosi is forcing House Republicans to go on the record on Trump's 'go back' tweets
Trump tells White House counselor Kellyanne Conway to ignore congressional subpoena
Behind Trump's 'go back' demand: A long history of rejecting 'different' Americans
Opinion We need someone who can lift us up again. Enter President Obama.


NYS 4-H Shooting Sports Banquet & Auction…Update!
Reservations for the 2015 New York State 4-H Shooting Sports Banquet & Auction on Saturday, April 25th at Fort William Henry Hotel & Conference Center in Lake George are now closed. If you are still interested in attending, and your Registration is not already in hand at the Saratoga CCE Office as of today, please contact them to check on Wait List availability.
NYS4-HSS looks forward to a great afternoon and evening of celebration with our attendees as we pay special tribute to our Veterans and celebrate the 4-H Shooting Sports! Many photos and more detailed event information will be posted on the New York State 4-H Shooting Sports website this weekend – please stay tuned to http://www.nys4hshootingsports.org.
Our 2015 event promises to have something in it for everyone with almost too many outstanding items to even count with some amazing additions still rolling in. Campers and adventurers, take note – Gander Mountain, through their store in Plattsburg, is assembling a special package of Gander Mountain brand camping items for a party of two. Keep an eye on the website for more information on this great package and all the other great items up for grabs on April 25th!
NYS4-HSS Live Auction – Kayak!
This Dagger Kayak, complete with Paddle and Life Jacket, from the generous folks at Mountainman Outdoor Supply Company will be up for grabs to the highest bidder in the Live Auction at the New York State 4-H Shooting Sports Fundraiser on April 26, 2014 at Fort William Henry Resort, Lake George.
You'll be all set for the upcoming Saratoga Paddlefest in May, with this great kayak package.
Our sincere thanks to the Mountainman Outdoor Supply Company in Saratoga Springs, NY for their support!
The 2013 New York State 4-H Shooting Sports Banquet & Auction will be held on Saturday, April 20, in the spacious historic Towers Hall building of the Fort William Henry Hotel & Conference Center in Lake George, New York.
Thank you to each and everyone who attended and contributed to our 2012 event making it our most successful undertaking to date. We hope to reach even greater goals with this year's event with your continued support!
Reservations are due by Thursday, April 11, 2013. Please visit our website at http://nys4hshootingsports.org for more information on this great annual event!


So today we are talking about Dom Vulnerability and how it is actually quite Domly to be vulnerable. ReverendTodd takes a turn in the hot seat on this one. Since he posted a writing recently that was a very blunt and personal writing on the personal struggle within as he steps into PolyAmory territory. We think you'll enjoy this show and look forward to hearing from you.


text:
\section{Introduction}
Abnormal sleep is increasingly recognized as a crucial factor in many illnesses \cite{wulff2010sleep}. Specifically, sleep physiology recorded via polysomnography (PSG) provides a rich source of information about the brain and cardiovascular health \cite{lajnef2015learning}. Therefore, automated sleep staging and diagnostics of sleep disorders are likely to play a crucial role in large-scale epidemiological research linking sleep patterns to disease and wellness applications \cite{wulff2010sleep}. The ground truth for sleep staging remains the multi-lead electroencephalogram (EEG), where sleep experts use the standard rules (e.g., Rechtschaffen and Kales (R\&K) \cite{wolpert1969manual} and the American Academy of Sleep Medicine (AASM) \cite{iber2007aasm}) for annotating sleep stages. The rules focus on 30-sec windows of data (or `epochs') and manual labeling of epochs into five stages: Wake (W), Rapid Eye Movement (REM), Non-REM 1-3 (N1, N2, and N3).  In addition to the time and cost involved in manual sleep staging, the significant inter-expert subjectivity may lead to a noisy-labeling issue \cite{jia2020graphsleepnet}.

To address the aforementioned issues, neural networks have received much attention for developing an automated sleep staging network using physiological time series. In particular, convolutional neural networks (CNNs) have been exploited to handle the multimodality of PSG signals \cite{chambon2018deep,perslev2021u} and to extract subject-invariant representation across the population. Due to the time-series nature of PSG signals, hybrid networks, the combination of convolutional and recurrent layers, have been shown to be the most successful networks for sleep staging task \cite{seo2020intra, sun2019hierarchical, dong2017mixed}. These networks extract informative features to capture the dynamics of sleep stage transitions \cite{supratak2017deepsleepnet, phan2019seqsleepnet}. More specifically, the key advantage of hybrid networks is that they do the sleep staging task in an end-to-end learning framework, removing (almost) all pre-processing steps, which include cleaning data, extracting hand-crafted features, and selecting important features. For example, U-Sleep \cite{perslev2021u}, DeepSleepNet \cite{supratak2017deepsleepnet} and SeqSleepNet \cite{phan2019seqsleepnet} were proposed to handle varying combinations of the PSG signals, different lengths sleep epochs, and capture the temporal dependencies between neighboring epochs. However, since the inputs of these networks are on regular grids (e.g., image-like), these networks ignore the connections among brain regions. Therefore, such networks cannot capture the brain's geometrical information, which can be best described in a non-Euclidean space \cite{jia2020graphsleepnet, barachant2011multiclass}, and they disregard the connected sequential nature of PSGs encoding information about the dynamics of sleep stage transitions. These limitations make it difficult for clinicians to interpret the network's outputs.

To handle and better interpret graph structured data, graph neural network (GNN) and graph convolutional network (GCN) models have been proposed, which provide performing node-level, edge-level, and graph-level prediction tasks \cite{wu2020comprehensive, velivckovic2017graph, zhang2018link}, and have been used for sleep staging tasks \cite{jia2020graphsleepnet, ahmedt2021graph}. Graph convolution operations can be performed in a spatial domain (the space of the nodes) \cite{velivckovic2017graph,niepert2016learning,hamilton2017inductive} or a spectral domain (eigenvalue decomposition of a Graph Shift Operator, e.g. adjacency or Laplacian matrix) \cite{bruna2013spectral,defferrard2016convolutional,levie2018cayleynets}. Wu et al. \cite{wu2020comprehensive} has provided a survey on GCNs using spatial and spectral approaches. Note that although GCN-based models can achieve excellent performance in different domains, they usually use prior known or fixed graphs to perform graph convolution operations \cite{wu2020comprehensive}. This assumption is not optimal in many time-series classification tasks, such as sleep stage classification, since the connectivity graph can change adaptively with the stage transitions \cite{jia2020graphsleepnet}. To the best of our knowledge, GraphSleepNet \cite{jia2020graphsleepnet} is the only application of adaptive graph learning for performing automated sleep staging. However, the authors ignored the weights between the sequential epochs' interactions. In addition, the \textit{joint} attentive spatio-temporal information connections was not captured, and the extracted information from neighbor (sequential) temporal epochs were simply concatenated. To address these issues, we propose an adaptive GCN, named ProductGraphSleepNet, which exploits Graph Signal Processing (GSP) \cite{ortega2018graph} and Product Graph Learning (PGL) \cite{kadambari2020learning} concepts to \textit{jointly} learn the sparsest possible spatio-temporal graph representations.

Training and testing of our method is performed in a cross-subject manner on two well-known and public sleep datasets Montreal Archive of Sleep Studies (MASS)-SS3 \cite{o2014montreal} and SleepEDF database \cite{goldberger2000physiobank, kemp2000analysis}, which contain full night PSG recordings of 62 and 20 healthy subjects, respectively. Specifically, we present three contributions in this work as:

\begin{itemize}
	\item Jointly interpretting and learning of temporal and spatial graphs for each sleep stage to provide a more medically interpretable sleep staging network and model any possible connection between the temporal information. Note that in the GraphSleepNet work \cite{jia2020graphsleepnet}, the spatial graphs are learned using the information of the target epochs only, and temporal information is simply concatenated with the same weight, therefore, ignoring the importance of connection weights between sequential epochs.
	\item Utilizing bi-directional gated recurrent units (BiGRU) to learn the transition rules between the sleep stages and produce feature vectors for the temporal graphs' nodes.
	\item Adaptively learning the importance of temporal weights for sequential epochs by modifying the graph attention network (GAT) \cite{velivckovic2017graph} module, resulting to proposing a Graph-wise Attention Network (GwAT).
\end{itemize}

\section{Preliminaries}
\label{Prel}

\textbf{Notation}. For indexing the data elements or row/column arrays, we use the MATLAB indexing system, that is, \(\bold{A}(i,j)\), \(\bold{A}(i,:)\) and \(\bold{A}(:,i)\) denote the \((i,j)\)th element, the \(i\)th row and the \(i\)th column of \(\bold{A}\), respectively. In a 3D array \(\bold{A}\), \(\bold{A}(i,:,:)\) means the \(i\)th slice of the first dimension of \(\bold{A}\). The all-zero, all-one vectors, and also the trace operator of \(\bold{A}\) are stated as \(\bold{0}\), \(\bold{1}\), and \(tr(\bold{A})\), respectively.

Let \(\bold{S}_i \in \mathbb{R}^{Q\times T_s}\) represent the \(i\)th 30-sec sleep epoch of the neighbor (sequential) epochs \(\bold{S}\), and the PSG signals of the sequential sleep epochs are stated as \( \bold{S}=(\bold{S}_{t-d},…,\bold{S}_{t},…,\bold{S}_{t+d}) \in \mathbb{R} ^ {P \times Q \times T_s }\), where \(t\) indicates the current target epoch. \(P=2d+1\), \(Q\) and \(T_s\) denote the number of sequential epochs, number of PSG channels and number of PSG temporal samples, respectively. In this work, similar to \cite{jia2020graphsleepnet}, \(F_{de}\) is the number of the extracted differential entropy (DE) features from all PSG channels of each \(\{\bold{S}_i\}_{i=t-d}^{t+d}\). Therefore, the shape of the input data is described as \(\bold{X}=(\bold{X}_{t-d},\cdots,\bold{X}_{t},\cdots,\bold{X}_{t+d} ) \in \mathbb{R}^{P\times Q \times F_{de}}\) by concatenating the \(P\) sequential feature extracted epochs, where \(\bold{X}_i \in \mathbb{R}^{Q\times F_{de}}\) denotes the \(i\)th extracted feature epoch from the sequential epochs \(\bold{X}\).

\textbf{Graph Signal Processing}. A graph \(\mathcal{G}_N\) can be stated as \(\{\mathcal{V},\mathcal{E},\bold{W}_N\} \), where \(\mathcal{V}\) denotes the set of graph nodes with cardinality of \(|\mathcal{V}|=N\), \(\mathcal{E}\) is the set of graph edges interpreted as node connections and \(\bold{W}_N \in \mathbb{R}^{N\times N}\) is the adjacency matrix of \(\mathcal{G}_N\) containing the edge weight \(\{\bold{W}_N (i,j)\}_{i,j=1}^N\geq0\) between the \(i\)th and \(j\)th nodes. Due to considering undirected graphs in this work, \(\bold{W}_N\) is a symmetric matrix. A signal, i.e., \(\bold{y}\in\mathbb{R}^{N\times1}\), is called a graph signal if its samples are assigned to the nodes of \(\mathcal{G}_N\). A degree matrix is a diagonal matrix \(\bold{D}_N\), which have the nodes' degrees on its diagonal. The graph Laplacian matrix \(\bold{L}_N=\bold{D}_N-\bold{W}_N\) is used to describe many useful properties of a graph; e.g., connectedness and centrality \cite{stankovic2019vertex}. The smoothness of a graph signal \(\bold{y}\) on \(\mathcal{G}_N\) is measured by the total variation (TV) of \(\bold{y}\) on \(\mathcal{G}_N\) defined as \({\textrm{TV}}_N (\bold{y})=\bold{y}^T \bold{L}_N \bold{y}= \sum_{i,j=1}^{N} {\left(\bold{y}(i)-\bold{y}(j)\right)^2 \bold{W}_N (i,j)}\). Precisely, the less the \(\textrm{TV}_N (\bold{y})\), the more similar values of \(\bold{y}\) are on connected nodes of \(\mathcal{G}_N\). The feature matrix \(\bold{Y} \in \mathbb{R}^{N \times M}\) can be considered as \(M\) graph signals \(\{\bold{y}_m \in \mathbb{R}^{N\times 1}\}_{m=1}^M\)  on \(\mathcal{G}_N\). Besides, in \(\bold{Y}\), the feature vector of the \(i\)th node can be considered as \(\Tilde{\bold{y}}_n \in \mathbb{R}^{1\times M}\), where these feature vectors can again be gathered together as \(\bold{Y}=[\Tilde{\bold{y}}_1^T|\Tilde{\bold{y}}_2^T|...|\Tilde{\bold{y}}_N^T]^T\). To investigate the overall smoothness of \(\{\bold{y}_m\}_{m=1}^M\) on \(\mathcal{G}_N\), one can measure the overall TV of these graph signals on \(\mathcal{G}_N\) as:
\begin{equation}
	\label{eq:eq1}
	\begin{split}
		\textrm{TV}_N(\bold{Y}) & =\sum_{m=1}^M{\textrm{TV}_N(\bold{y}_m)} =\sum_{i,j}^N{||\Tilde{\bold{y}}_i-\Tilde{\bold{y}}_j||_2^2\bold{W}_N(i,j)}\\
		& =2\:tr(\bold{Y}^T\bold{L}_N\bold{Y})
	\end{split}
\end{equation}

\textbf{Graph Learning}. To learn a graph from data with assumption of being smooth on the graph, Dong et al. \cite{dong2016learning} proposed to minimize Eq (\ref{eq:eq1}) w.r.t the corresponding valid graph Laplacian \(\bold{L}_N\) in a convex optimization as:

\begin{equation}
	\label{eq:eq2}
	\bold{L}_N=\underset{\bold{L}\in \mathcal{F}_N}{\arg\min}\:{tr(\bold{Y}^T\bold{L}_N\bold{Y})+\lambda||\bold{L}_N||_F^2}
\end{equation}

\noindent where \(\lambda\) balances the sides of the optimization (i.e.  the sparsest graph can be inferred with \(\lambda=0\) \cite{dong2016learning}), and  \(\mathcal{F}_N\) is the set of the valid graph Laplacians defined as:
\begin{equation}
	\label{eq:eq3}
	\begin{split}
		\mathcal{F}_N & =\{ \bold{L} \in \mathbb{R}^{N\times N} | \bold{L1}=\bold{0}, tr(\bold{L})=N, \\
		& (\forall i\neq j) \bold{L}(i,j)=\bold{L}(j,i) \leq 0 \}
	\end{split}
\end{equation}

\textbf{Product Graph Learning}. Due to the joint interaction between different domains, Kadambari et al. \cite{kadambari2020learning} showed that it is more accurate, computationally efficient and interpretable to consider the graph of interest as the Cartesian product of two significantly smaller factor graphs  \(\mathcal{G}_P\) and \(\mathcal{G}_Q\) with \(\bold{L}_P \in \mathbb{R}^{P\times P}\) and \(\bold{L}_Q \in \mathbb{R}^{Q\times Q}\) as their Laplacians, where \(PQ=N\), relevant to their corresponding domains (in this work temporal and spatial), and learn the factor graphs \(\mathcal{G}_P\) and \(\mathcal{G}_Q\) using the Kronecker sum (i.e., \(\bigoplus\)) as in \cite{hammack2011handbook}:
\begin{equation}
	\label{eq:eq4}
	\bold{L}_N=\bold{L}_P \oplus \bold{L}_Q=\bold{L}_P \otimes  \bold{I}_Q + \bold{I}_P \otimes \bold{L}_Q
\end{equation}
\noindent where \(\bigotimes\) is the Kronecker product and \(\bold{I}_n \in \mathbb{R}^{n\times n}\) stands for the identity matrix.

Therefore, instead of having the multidomain graph signals \(\{\bold{y}_m\in \mathbb{R}^{N\times 1} \}_{m=1}^M\), the (product) graph matrices \(\{\bold{Y}_m\in \mathbb{R}^{P\times Q}\}_{m=1}^M\) represent the factor graph signals. Kadambari et al. \cite{kadambari2020learning} showed the product graph learning can be defined by \(\textrm{TV}_P (\bold{Y}_m)+\textrm{TV}_Q (\bold{Y}_m^T)\) as:

\begin{equation}
	\label{PGL_cost1}
	\begin{split}
		&\{\bold{L}_P,\bold{L}_Q\} \\
		&=\underset{\bold{L}_P\in \mathcal{F}_P, \bold{L}_Q\in \mathcal{F}_Q}{\arg\min}{\sum_{m=1}^{M}{[tr(\bold{Y}_m^T\bold{L}_P\bold{Y}_m)+tr(\bold{Y}_m\bold{L}_Q\bold{Y}_m^T)]}}\\
		&+\lambda(||\bold{L}_P||_F^2+||\bold{L}_Q||_F^2)
	\end{split}
\end{equation}

\noindent Since the goal is to learn  the sparsest graphs, therefore, using Eq. (\ref{eq:eq1}) and considering \(\lambda=0\), Eq. (\ref{PGL_cost1}) can be rewritten as:
\begin{equation}
	\label{eq:eq6}
	\{\bold{W}_P,\bold{W}_Q\}=\underset{\bold{W}_P\in \mathcal{W}_P, \bold{W}_Q\in \mathcal{W}_Q}{\arg\min}{\mathcal{L}_{\mathcal{G}_P,\mathcal{G}_Q}(\bold{W}_P,\bold{W}_Q)}
\end{equation}

\noindent where

\begin{equation}
	\footnotesize
	\label{PGL_cost2}
	\begin{split}
		\mathcal{L}_{\mathcal{G}_P,\mathcal{G}_Q} (\bold{W}_P,\bold{W}_Q )
		= & \dfrac{1}{2}  \sum_{m=1}^{M}{[\sum_{r,s}^{P}{ ||\bold{Y}_m(r,:)-\bold{Y}_m(s,:)||_2^2 \bold{W}_P(r,s)}} \\ & + \sum_{r',s'}^{Q}{||{\bold{Y}_m(:,r')-\bold{Y}_m(:,s')||_2^2\bold{W}_Q(r',s')}}]
	\end{split}
\end{equation}

\noindent where \(\mathcal{W}_n\) is the set of valid adjacency matrices for the undirected graphs of size \(n\) defined as:

\begin{equation}
	\label{eq:eq8}
	\mathcal{W}_n=\{\bold{W}\in \mathbb{R}^{n\times n}|\bold{W}(i,j)=\bold{W}(j,i)\geq0\}
\end{equation}

\par To jointly optimize the GL and classification tasks, the product graph learning (PGL) loss function \(\mathcal{L}_{PGL} (\bold{W}_P,\bold{W}_Q)=\mathcal{L}_{\mathcal{G}_P,\mathcal{G}_Q} (\bold{W}_P,\bold{W}_Q)\) is added to the classification loss function, i.e., Cross entropy loss.

\par \textbf{Graph Convolution}. To account the local connectivities between spatial nodes in the convolution process, the (spatial) graph signal \(\bold{s}\in \mathbb{R}^{Q\times 1}\) can be convolved with a graph Laplacian \(\bold{L}_Q \in \mathbb{R}^{Q\times Q}\) of the interested graph \(\mathcal{G}_Q\) in a more efficient manner using the Chebyshev graph convolution expansion of order \(K-1\) \cite{defferrard2016convolutional} as:

\begin{equation}
	\label{eq:eq9}
	g_{\bold{\theta}}*_\mathcal{G}\bold{s}=g_{\bold{\theta}}(\bold{L}_Q)\bold{s}=\sum_{k=0}^{K-1}{\theta_kT_k(\Tilde{\bold{L}}_Q)\bold{s}}
\end{equation}

\noindent where \(g_{\bold{\theta}}\) is the convolution kernel and \(*_\mathcal{G}\) denotes the graph convolution operator. \(\boldsymbol{\theta} \in \mathbb{R}^K\) stands for a vector containing the trainable Chebyshev coefficients. \(\Tilde{\bold{L}}_Q\) is the normalized Laplacian defined as \(\Tilde{\bold{L}}_Q=2/{\lambda_{max}}  \bold{L}_Q-\bold{I}_Q\), where \(\lambda_{max}\) is the maximum eigenvalue of \(\bold{L}_Q\). In addition, \({T}_k (x)=2x{T}_{k-1} (x)-{T}_{k-2} (x)\) denotes for the recursive Chebyshev polynomials with \(T_0 (x)=1\) and \(T_1 (x)=x\). Exploiting the approximate Chebyshev expansion, the aggregation of the information over \(K-1\) neighbor spatial nodes (i.e., \((K-1)\)-hop connections) is obtained.

\section{The Proposed ProductGraphSleepNet}
The detailed architecture if our proposed ProductGraphSleepNet is provided in Figure \ref{fig:Fig1}. The step-wise description of the different modules is detailed as:

\begin{figure*}[t!]
	\centering

	\includegraphics[width=18cm]{Figures2_v22.pdf}
	\caption{The ProductGraphSleepNet. After DE feature extraction of the neighbor sleep epochs, Spatio-temporal graphs and spatial attention coefficients are learned. Then, the attentive GC layer along with BiGRU produces temporal nodes' features. Finally, using GwAT and the learned temporal graph, the final sleep staging is performed.}
	\label{fig:Fig1}
\end{figure*}

\par \textbf{Spatial Attention Layer}. To capture the dynamic spatial attention, which adaptively changes during the transitions of the sleep stages, the spatial attention mechanism proposed by Guo et al. \cite{guo2019attention} is exploited. As shown in Figure \ref{fig:Fig1}, the input to this layer is \(\bold{X}^{(0)} \in \mathbb{R}^{P\times Q\times F_{de}}\), and the spatial attention weights, i.e., \(\bold{P}\in \mathbb{R}^{Q\times Q}\), is obtained by learning the weights \(\bold{V} \in \mathbb{R}^{Q\times Q}\), \(\bold{b}_p\in \mathbb{R}^{Q\times Q}\), \(\bold{Z}_1\in \mathbb{R}^{P\times 1}\), \(\bold{Z}_2 \in \mathbb{R}^{F_{de}\times P}\), \(\bold{Z}_3 \in \mathbb{R}^{F_{de}\times1}\) as:
\begin{equation}
	\label{eq:eq10}
	\bold{P}=\text{Softmax}(\bold{V}.\: \text{Sigmoid}(\bold{X}_{Lhs}\bold{Z}_2\bold{X}_{Rhs}^T+\bold{b}_p))
\end{equation}
\noindent where \(\bold{X}_{Lhs}=\sum_{p=1}^{P}{\bold{X}^{(0)}(p,:,:)\bold{Z}_1(p)}\in \mathbb{R}^{Q\times F_{de}}\) and \(\bold{X}_{Rhs}=\sum_{f=1}^{F_{de}}{\bold{X}^{(0)}(:,:,f)^T\bold{Z}_3(f)}\in \mathbb{R}^{Q\times P}\).

\textbf{Neural Network Modeling}. Let \(\bold{a}_{f,P}^{(i,j)}=|\bold{X}^{(0)}(i,:,f)-\bold{X}^{(0)}(j,:,f)|\in \mathbb{R}^{1\times Q}\) modeling the difference between the \(i\)th and \(j\)th sleep epochs for specified \(f\)th DE feature, and stack \(\{\bold{a}_{f,P}^{(i,j)}\}_{f=1}^{F_{de}}\) in a difference matrix \(\bold{A}_P^{(i,j)}=[{\bold{a}_{1,P}^{(i,j)}}^T |…| {\bold{a}_{F_{de},P}^{(i,j)}}^T]^T \in \mathbb{R}^{F_{de}\times Q}\). Then, the empirical difference vector \(\bold{d}_P^{(i,j)}\in \mathbb{R}^{F_{de}\times1}\) can be obtained as:

\begin{equation}
	\label{eq:eq11}
	\bold{d}_P^{(i,j)}=\dfrac{1}{Q}\sum_{q=1}^{Q}{\bold{A}_P^{(i,j)}(:,q)}
\end{equation}

\noindent  The \((i,j)\)th graph connection edge of the \(\bold{W}_P\) (i.e., \(\bold{W}_{P}(i,j)\)), which models the temporal connection between the \(i\)th and \(j\)th sleep epochs, is learned as:

\begin{equation}
	\label{eq:eq12}
	\bold{W}_{P}(i,j) = \dfrac{exp(ReLU(\bold{w}_P^T\bold{d}_P^{(i,j)}))}{\sum_{j'=1}^{P}{exp(ReLU(\bold{w}_P^T\bold{d}_P^{(i,j')}))}}
\end{equation}

\noindent with the neural network's learnable weights \(\bold{w}_P\in\mathbb{R}^{F_{de}\times1}\). Similarly, the formulation for modeling  \(\bold{W}_Q\), which contains the spatial connections between PSG electrodes and can be used for analysis of the brain connectivity, is summarized:

\begin{equation}
	\label{eq:eq16}
	\bold{a}_{f,Q}^{(i,j)}=|\bold{X}^{(0)}(:,i,f)-\bold{X}^{(0)}(:,j,f)|\in \mathbb{R}^{P\times 1}
\end{equation}

\begin{equation}
	\label{eq:eq15}
	\bold{A}_Q^{(i,j) }=[\bold{a}_{1,Q}^{(i,j)} |…| \bold{a}_{F_{de},Q}^{(i,j)}]^T \in \mathbb{R}^{F_{de}\times P}
\end{equation}

\begin{equation}
	\label{eq:eq14}
	\bold{d}_Q^{(i,j)}=\dfrac{1}{P}\sum_{p=1}^{P}{\bold{A}_Q^{(i,j)}(:,p)}
\end{equation}

\begin{equation}
	\label{eq:eq13}
	\bold{W}_{Q}(i,j) = \dfrac{exp(ReLU(\bold{w}_Q^T\bold{d}_Q^{(i,j)}))}{\sum_{j'=1}^{Q}{exp(ReLU(\bold{w}_Q^T\bold{d}_Q^{(i,j')}))}}
\end{equation}

\noindent with the learnable weights \(\bold{w}_Q\in \mathbb{R}^{F_{de}\times1}\). The output of the PGL layer are the learned spatial and temporal graph Laplacians \(\bold{L}_Q\) and \(\bold{L}_P\) based on the PGL cost function \(\mathcal{L}_{PGL}\) (\ref{PGL_cost1}) or (\ref{PGL_cost2}) mentioned in Section \ref{Prel}, as shown in Figure \ref{fig:Fig1}.

\par \textbf{Attentive Graph Convolutional Layer.} To account the spatial importance weights, local spatial connectivites in sequential epochs and as a generalization to the graph convolution of one graph signal, in this work, the \(i\)th sequential epoch \(\bold{X}^{(0)}(i,:,:)\in\mathbb{R}^{Q\times F_{de}}\), which contains \(F_{de}\) graph signals, is spatially convolved using the mentioned Chebyshev expansion and the learned spatial graph \(\bold{L}_Q\), and gets the output \(\bold{X}^{(1)}\in\mathbb{R}^{P\times Q\times F'}\), as shown in Figure \ref{fig:Fig1}, as:

\begin{equation}
	\label{eq:eq17}
	\begin{split}
		& \bold{X}^{(1)}(i,:,:) =g_{\bold{\theta}}*_{\mathcal{G}}\bold{X}^{(0)}(i,:,:)=g_{\bold{\theta}}(\bold{L}_Q)\bold{X}^{(0)}(i,:,:) \\
		& =\sum_{k=0}^{K}{{\left[T_k(\Tilde{\bold{L}}_Q)\odot \bold{P}\right]}\bold{X}^{(0)}(i,:,:)\bold{\Theta}^{(k)}};  \:\:i=1,2,..,P
	\end{split}
\end{equation}

\noindent where \(\bold{\Theta}^{(k)}\in\mathbb{R}^{F_{de}\times F'}\) is the \(k\)th trainable Chebyshev coefficients matrix, \(\odot\) denotes the Hadamard (element-wise) product, and \(\bold{P}\) is learned from the spatial attention layer.

\textbf{BiGRU.} The output \(\bold{X}^{(1)}\in\mathbb{R}^{P\times Q\times F'}\) is then flattened to obtain feature vectors \(\bold{X}^{(2)}\in\mathbb{R}^{P\times (QF')}\) in Figure \ref{fig:Fig1}, and fed to a BiGRU module with \(\beta\) units to learn the sleep stage transition rules between these \(P\) sequential epochs. Therefore, the BiGRU module gets output \(\bold{X}^{(3)}\in\mathbb{R}^{P\times\beta}\). This obtained \(\bold{X}^{(3)}\) in Figure \ref{fig:Fig1} is considered as the feature matrix of the nodes of the learned temporal graph \(\bold{L}_P\) and is fed to the GwAT module.

\par \textbf{Graph-wise Attention Network (GwAT).} To account the importance weights of the different neighbor sleep epochs and also learn attentive temporal feature vectors, we modify the celebrated Graph Attention Network (GAT) so it can be used in graph classification scenarios. Considering \(\bold{X}^{(3)}\in\mathbb{R}^{P\times\beta}\) and \(\bold{W}_P\) be the feature matrix and adjacency matrix of the learned temporal graph \(\mathcal{G}_P\) to our network, the goal here is learning attention coefficients \(\pmb{\alpha} \in\mathbb{R}^{P\times P}\), and obtaining new attentive feature vectors \(\bold{X}^{(4) }\in\mathbb{R}^{P\times F_{\textit{GwAT}}}\) as:

\begin{equation}
	\label{eq:eq18}
	\pmb{\alpha}(r,s)=\dfrac{\hat{\pmb{\alpha}}(r,s)}{\sum_{s'=1}^{P}{\hat{\pmb{\alpha}}(r,s')}}
\end{equation}

\begin{equation}
	\label{eq:eq19}
	\begin{split}
		& \hat{\pmb{\alpha}}(r,s)= \bold{W}_P(r,s)\times\\
		& \exp\left(\text{LeakyReLU}\left(\pmb{\gamma}^T\left[\bold{X}^{(3)}(r,:)\bold{W}||\bold{X}^{(3)}(s,:)\bold{W}\right]\right)\right)
	\end{split}
\end{equation}

\noindent where \(\boldsymbol{\gamma}\in\mathbb{R}^{2F_{\textit{GwAT}}\times1}\) and \(\bold{W}\in\mathbb{R}^{\beta\times F_{\textit{GwAT}}}\) are the learnable weights of the GwAT module, and \(||\) stands for the concatenation operator. Moreover, the new attentive feature vectors of the temporal graph nodes can be obtained as:

\begin{equation}
	\label{eq:eq20}
	\bold{X}^{(4)}=\pmb{\alpha}\bold{X}^{(3)}\bold{W}
\end{equation}

To increase the learning capacity, one can have \(K_{\textit{GwAT}}\) attention heads in which the mentioned equations can be briefly indexed w.r.t the \(k\)th (\(k=1,...,K_{\textit{GwAT}}\)) attention head:

\begin{equation}
	\label{eq:eq18}
	\pmb{\alpha}^{(k)}(r,s)=\dfrac{\hat{\pmb{\alpha}}^{(k)}(r,s)}{\sum_{s'=1}^{P}{\hat{\pmb{\alpha}}^{(k)}(r,s')}}
\end{equation}

\begin{equation}
	\label{eq:eq22}
	\begin{split}
		& \hat{\pmb{\alpha}}^{(k)}(r,s)= \bold{W}_P(r,s)\times\\
		& \exp\left(\text{LeakyReLU}\left(\pmb{\gamma}^T\left[\bold{X}^{(3)}(r,:)\bold{W}^{(k)}||\bold{X}^{(3)}(s,:)\bold{W}^{(k)}\right]\right)\right)
	\end{split}
\end{equation}

\begin{equation}
	\label{eq:eq23}
	{\bold{X}^{(4)}}^{(k)}=\pmb{\alpha}^{(k)}\bold{X}^{(3)}\bold{W}^{(k)}
\end{equation}

\noindent Finally, the GwAT's classification output \(\bold{X}_o\) is defined as:

\begin{equation}
	\label{eq:eq25}
	\bold{X}_o=\sigma(\dfrac{1}{K_{\textit{GwAT}}}\sum_{k=1}^{K_{\textit{GwAT}}}{(\dfrac{1}{P}\sum_{p=1}^{P}{{\bold{X}^{(4)}}^{(k)}(p,:)})}) \in \mathbb{R}^{1\times {F_{\text{\textit{GwAT}}}}}
\end{equation}

\noindent where \(\sigma\) is the \textit{softmax} function. Note that in this work, \(F_{\textit{GwAT}}=5\) is equal to the number of sleep stages.

\section{Experimental Setup}

\noindent This work focuses on developing an automated interpretable sleep staging classifier. To evaluate the proposed ProductGraphSleepNet, two well-known public datasets are used.

\textbf{Dataset}. 1) The MASS-SS3 database \cite{o2014montreal} contains PSG signals (with sampling frequency 256 Hz) of 62 healthy subjects recorded on 20 EEG, 3 electromyograms (EMG), 2 electrooculograms (EOG), and 1 electrocardiogram (ECG) electrodes. The sleep epochs were staged based on five sleep stages, i.e., W, REM, N1-3, by sleep staging specialists following the AASM standard \cite{iber2007aasm}. 2) The SleepEDF dataset \cite{goldberger2000physiobank,kemp2000analysis} includes data from 20 healthy subjects, 2 EEG, 1 EOG, 1 EMG, and 1 oro-nasal respiration channels with the sampling frequency of 100 Hz, where are scored by R\&K standard. To have similar labeling space (five-stage), the sleep stages of SleepEDF are converted to the AASM standard. For SleepEDF dataset, only the available EEG and EOG channels are used.

\textbf{Preprocessing}. Firstly, each of the PSG channels was decomposed into sub-frequency bands 0.5-4 Hz, 2-6 Hz, 4-8 Hz, 6-11 Hz, 8-14 Hz, 11-22 Hz, 14-31 Hz, 22-40 Hz, and 31-50 Hz. Then, without any further pre-processing or denoising procedures, DE features were extracted from them. Thus, in this work, \(Q=26\) and \(Q=3\) for the MASS-SS3 and SleepEDF datasets, respectively, and \(F_{de}=9\). Besides, due to taking into account four epochs before and after the target sleep epoch, we have \(d=4\) and \(P=2d+1=9\).

\textbf{Training Setup.} We conduct our experiment on the MASS-SS3 dataset using a 16-fold cross-validation scheme, in which each of the first 15 folds contains four subjects' data, while the last fold consists data from two subjects. As validation data for each test fold, one fold is randomly selected, and the best-trained model on this validation data is used to apply on the unseen test fold. For SleepEDF, we use the Leave One Subject Out (LOSO) cross validation (with the randomly selected one subject as validation data for each test subject). The values of the hyperparameters in the model are as follow: $F'=10$, $\beta=256$, $K_{\text{GwAT}}=20$, $\text{Number of training epochs} = 100$, $\text{Chebyshev Order K} = 3$, $\text{Dropout probability}= 0.6$, $\text{Batchsize}=1024 $, $\text{Learning rate} = 0.001$, $\text{Optimizer=Adam}$ \cite{kingma2014adam}. All the training steps are implemented using Tensorflow \cite{abadi2016tensorflow}.

\section{Experimental Results and Discussion}
To evaluate the effectiveness of ProductGraphSleepNet, some ablation baselines and also some recent relevant studies that reported their performance metrics on the MASS-SS3 and SleepEDF datasets are considered.

\textbf{Ablation study.} To investigate the effectiveness of the proposed modules in our ProductGraphSleepNet, we implemented two major baselines on the MASS-SS3 dataset, named Baseline 1 and 2 in Table \ref{MASS-SS3} with the details as follows: Baseline 1: Jia et al. \cite{jia2020graphsleepnet} proposed a graph neural network for performing sleep staging tasks. In the Baseline 1, the temporal information are simply concatenated, and the BiGRU is not exploited. Baseline 2: To investigate the impact of the GwAT in the proposed method, the GwAT is replaced with the simple concatenation of the outputs of the BiGRU module.

\textbf{Comparison with State-of-the-Art Methods.} The considered studies for MASS-SS3 are: 1) Dong et al. \cite{dong2017mixed} provides a comprehensive study in which the performance of the traditional Random Forest (RF) and Support Vector Machine (SVM) classifiers were compared with a mixed neural network, which is a hybrid of Multi-Layer Perceptron (MLP) and Long Short Term Memory (LSTM) modules; 2) Supratak et al. \cite{supratak2017deepsleepnet} considered a hybrid model of the CNN and BiLSTM modules, which take into account the neighbor sleep epoch data to learn the transition rules between the sleep stages; 3) Chambon et al. \cite{chambon2018deep} performed a study that handles the multimodality of the PSG signals in a temporal sleep staging scheme;  4) Jiang et al. \cite{jiang2019robust} proposed a Hidden Markov Model (HMM)-based robust classifier that handles multimodal PSG learning; 5) Sun et al. \cite{sun2019hierarchical}: Learning comprehensive and sequence feature learning using a hierarchical neural network based on a hybrid combination of the CNN and BiLSTM networks; 6) Phan et al. \cite{phan2019seqsleepnet} proposed an attentive hierarchical recurrent neural network to process the sequences of the sleep epochs. Similarly, for SleepEDF evaluation, the considered studies (as well as the described \cite{supratak2017deepsleepnet} study), are: 1) Tsinalis et al. \cite{tsinalis2016automatic} investigated the impact of the different time-frequency features for using as input data to a DL network. 2) Vilamala et al. \cite{vilamala2017deep} analyzed the combination of the Multitaper spectral with a CNN. 3) Seo et al. \cite{seo2020intra} exploited BiLSTM network for learning Intra/Inter-epoch temporal dependencies. Table \ref{MASS-SS3} and Table \ref{SleepEDF} provide the comparisons among these techniques on the MASS-SS3 and SleepEDF datasets, respectively, where bold and underlined metrics are corresponds to the best and second best performances, respectively. Based on these results, one can conclude that our proposed network has competitive performance compared to the state-of-the-art methods with not significant performance difference rather than the best performances (even with the small number of channels in the SleepEDF dataset), and more importantly our proposed method provides a medically interpretable automated sleep staging algorithm. Moreover, results of the baseline 1 show that capturing the dynamics of the sleep stage transition using a temporal module (e.g., BiGRU, PGL, and GwAT) plays a critical role in the sleep staging task. The results obtained using the baseline 2 show that GwAT increases the discriminability and transferability of the model. Area Under Curve (AUC), Area Under Precision-Recall Curve (AUPRC) plots, and the confusion matrix using ProductGraphSleepNet on the MASS-SS3 and SleepEDF datasets, admitting the great discriminability of the proposed method, are shown in Figures \ref{AUC_AUPRC_Confusion} and \ref{ConfusionMatrix}, respectively.

\begin{table*}[hbt!]
	\centering
	\caption{Performance (i.e., Overall Accuracy, F1-score and Kappa, as well as per class F1-score) comparison between the ProductGraphSleepNet and state-of-the-art on MASS-SS3, where bold and underlined metrics are corresponds to the best and second best performances, respectively.}
	\begin{tabular}{||c |c| c| c| c| c| c| c| c|c||}
		\hline
		& Method & Accuracy & F1-score & Kappa & Wake & REM & N1 & N2 & N3  \\ [0.5ex]
		\hline\hline
		Baseline 1 & Modified GraphSleepNet & 0.845 & 0.792 & 0.773 & 0.878 & 0.874 & 0.527 & 0.889 & 0.788  \\ \hline
		Baseline 2 &  Simple concatenation & 0.853 & 0.801 & 0.782 & 0.889 & 0.887 & 0.555 & 0.894 & 0.782 \\ \hline
		Dong et al. \cite{dong2017mixed} & SVM & 0.797 & 0.750 & - & 0.786 & 0.792 & 0.487 & 0.861 & \textbf{0.825} \\ \hline
		Dong et al. \cite{dong2017mixed} & RF & 0.817 & 0.724 & - & 0.782 & 0.794 & 0.351 & 0.880 & \underline{0.815} \\ \hline
		Dong et al. \cite{dong2017mixed} & MLP+LSTM & 0.859 & 0.805 & - & 0.846 & 0.861 & 0.563 & \textbf{0.907} & 0.848 \\ \hline
		Supratak et al. \cite{supratak2017deepsleepnet} & DeepSleepNet & 0.862 & 0.817 & 0.800 & 0.873 & 0.893 & \textbf{0.598} & 0.903 & \underline{0.815}\\ \hline
		Chambon et al. \cite{chambon2018deep} & CNN & 0.739 & 0.673 & 0.640 & 0.730 & 0.764 & 0.294 & 0.812 & 0.765 \\ \hline
		Jiang et al. \cite{jiang2019robust} & RF+HMM & 0.808 & 0.793 & 0.710 & - & - & - & - & - \\ \hline
		Perslev et al. \cite{perslev2021u} & U-Sleep & - & 0.800 & - & \textbf{0.930} & \textbf{0.910} & 0.540 & 0.870 & 0.750 \\ \hline
		Phan et al. \cite{phan2019seqsleepnet} & SeqSleepNet & \textbf{0.871} & \textbf{0.833} & \textbf{0.815} & - & - & - & - & - \\ \hline
		Our Method & ProductGraphSleepNet & \underline{0.867} & \underline{0.818} & \underline{0.802} & \underline{0.894} & \underline{0.898} & \underline{0.583} & \underline{0.904} & 0.813 \\ \hline
		\hline
	\end{tabular}
	\label{MASS-SS3}
\end{table*}

\begin{table*}[hbt!]
	\centering
	\caption{Performance (i.e., Overall Accuracy, F1-score and Kappa, as well as per class F1-score) comparison between the ProductGraphSleepNet and state-of-the-art on SleepEDF, where bold and underlined metrics are corresponds to the best and second best performances, respectively.}

	\begin{tabular}{||c |c| c| c| c| c| c| c| c|c||}
		\hline
		& Method & Accuracy & F1-score & Kappa & Wake & REM & N1 & N2 & N3  \\ [0.5ex]
		\hline\hline
		Tsinalis et al. \cite{tsinalis2016automatic} & Time-frequency features & 0.748 & 0.698 & 0.65 & 0.437 & 0.654 & 0.806 & 0.849 & 0.745\\ \hline
		Supratak et al. \cite{supratak2017deepsleepnet} & DeepSleepNet & 0.820 & 0.769 & 0.760 & 0.847 & 0.824 & \underline{0.466} & \textbf{0.898} & 0.848 \\ \hline
		Vilamala et al. \cite{vilamala2017deep} & Multitaper spectrals + CNN & 0.813 & 0.765 & 0.740 & 0.809 & 0.819 & \textbf{0.474} & 0.862 & \underline{0.862} \\ \hline
		Seo et al. \cite{seo2020intra} & Intra-/inter-epoch BiLSTM & \textbf{0.839} & \textbf{0.776} & \textbf{0.78} & \underline{0.877} & \underline{0.825} & 0.434 & \underline{0.877} & \textbf{0.867}\\ \hline
		Our Method & ProductGraphSleepNet & \underline{0.838} & \underline{0.774} & \underline{0.775} & \textbf{0.886} & \textbf{0.834} & 0.426 & 0.874 & 0.847 \\ \hline
		\hline
	\end{tabular}
	\label{SleepEDF}
\end{table*}

\begin{figure}[t!]
	\centering
	\subfloat[]{{\includegraphics[width=.43\textwidth, trim=4.5cm 1cm 5.5cm 2.5cm]{MASS_SS3_AUC_AUPRC_plot.pdf} }\label{fig:mycaption-a}}

	\subfloat[]{{\includegraphics[width=.43\textwidth, trim=4.5cm 1cm 5.5cm 2cm]{SleepEDF20_AUC_AUPRC_plot.pdf} }\label{fig:mycaption-b}}

	\caption{AUC/AUPRC of the sleep staging task obtained using the proposed method: a) MASS-SS3 and b) SleepEDF.}
	\label{AUC_AUPRC_Confusion}
\end{figure}

\begin{figure}[t!]
	\centering
	\subfloat[]{{\includegraphics[width=6.5cm, trim=0cm 0cm 0cm 0cm]{MASS_SS3_Confusion_matrix_MASS-SS3.pdf} }}%
	\qquad
	\subfloat[]{{\includegraphics[width=6.5cm, trim=0cm 0cm 0cm 0cm]{SleepEDF20_Confusion_matrix_SleepEDF20.pdf}}}%
	\caption{Confusion matrix for the sleep staging task using the proposed method: a) MASS-SS3 and b) SleepEDF.}
	\label{ConfusionMatrix}%
\end{figure}

\textbf{Analysis of the Learned Spatial/Temporal Graphs on
	the MASS-SS3 dataset.} Analysis of the functional brain connectivity through obtained spatial brain graphs, e.g., variation of the functional connectivities within sleep stage transitions \cite{nguyen2018exploring}, has been addressed in pioneer neuroscientific literature as an active research area \cite{alper2013weighted, huang2018graph}. In the following, the learned temporal and spatial graphs from the proposed ProductGraphSleepNet corresponding to EEG vs. EEG and EEG vs. Non-EEG electrodes and averaged over class samples are analyzed to gain insights about the brain pattern transitions and illustrate the interpretability aspect of the proposed network. Firstly, the learned graphs need to be binarized according to a well-specified threshold, which we devise and adopt a statistical approach to obtain a specific threshold discriminating the obtained graphs the most. In this way, we considered each pair of graphs corresponding to the sleep stages and vary the edge values of these graphs across the span of  \(\{0, 0.2, 0.4, 0.6, 0.8, 1\}\). Then, the \(p\)-values obtained from the \(t\)-test approach are calculated as depicted in Figure \ref{PairwisePValues}, and the lowest (non-zero) \(p\)-value is selected to illustrate the most discriminating patterns, i.e., the most statistical significance, in the obtained graphs. Here, \(Thr=0.4\) is selected and used for the binarization of the obtained binary graphs.

\begin{figure*}[t!]
	\centering

	\includegraphics[width=18cm]{PairwisePValues.pdf}
	\caption{The calculated \(p\)-values from \(t\)-test statistical approach for EEG vs. EEG (EEG in this figure), EEG vs. Non-EEG (Non-EEG in this figure), and also temporal sleep epochs graph connectivities (Temp in this figure), across the values of \(\{0, 0.2, 0.4, 0.6, 0.8, 1\}\) for each pair of sleep stages.}
	\label{PairwisePValues}
\end{figure*}

\begin{figure}[t!]
	\centering

	\includegraphics[width=9cm]{AvgPValues.pdf}
	\caption{The \(p\)-values obtained from \(t\)-test and averaged over the pairwise sleep stage plots for EEG vs. EEG (EEG in this figure), EEG vs. Non-EEG (Non-EEG in this figure), and also temporal sleep epochs graph connectivities (Temp in this figure), in the span of \(\{0, 0.2, 0.4, 0.6, 0.8, 1\}\).}
	\label{AvgPValues}
\end{figure}

The binarized learned spatial graphs implying the connections across the EEG electrodes have been depicted in Figure \ref{Fig3} (a). It can be seen that the overall functional connectivities increase in the Wake state compared to other Non-REM ones. This observation is compatible with mentioned neuroscientific findings implying the reduction in hypothalamic functional connectivity of the Non-REM stages for (probably) stabilizing sleep \cite{kaufmann2006brain, tarun2021nrem}. Besides, it can be observed that the occipital region is considerably more active during the REM stage compared to the other Non-REM ones, which is supported by the mentioned and observed higher occipital metabolism in REM compared to Non-REM ones leading to more connectivity \cite{kjaer2002regional, kaufmann2006brain}.

Figure \ref{Fig3} (b) shows the binarized averaged learned connections between non-EEG channels (i.e., EOG Left, EOG Right, EMG chin1-3, and ECG) and EEG channels. For clarity, the connections between the EEG channels are ignored in this figure. Based on these learned connections, one can conclude that: 1) There are significantly more connections between ECG and EEG channels during REM compared to the other stages, especially NREM. This is consistent with \cite{varoneckas1986components, penzel2003dynamics}, where they showed that the increased activity of the heart rate also its connections stem from the increased influence of the brain on the autonomous nervous system. 2) The connection between EOG and EEG channels has a discriminative pattern between the sleep stages of NREM, REM, and wake, which aligns with the results of \cite{virkkala2007automatic}, where they detected discriminative synchronized EEG activity in the Wake and REM vs. the other stages by measuring the peak-to-peak amplitude and cross-correlation in the 0.5-6 Hz band between two EOG channels.

The binarized learned temporal graphs averaged over the class samples and corresponding to sleep stages are depicted in Figure \ref{Fig3} (c). As can be seen in this figure, these graphs have not necessarily had a tree-like topology, which means it is not efficient to assume that only the one-hop neighbor temporal epochs are connected. In fact, considerable variant types of connections are observed in this figure, such as the connection between the first (i.e., \(t-4\)) and the fifth (i.e., \(t+1\)) epoch corresponding to the REM stage, which is well-discriminative across other stages helping to probably more efficient classification. Note that these kinds of connections have been ignored in related studies \cite{jia2020graphsleepnet} by simply concatenation of the obtained feature vectors from the neighbor epochs.

\section{Conclusion}
This paper proposed an adaptive and automatic sleep staging network, namely ProductGraphSleepNet, which exploits Product Graph Learning (PGL) along with a Graph Convolutional Neural Network (GCN) to learn Spatio-temporal graphs obtained from the neighbor (sequential) sleep epochs. Due to the non-Euclidean nature of brain signals, using a graph-based structure preserves the geometry information. In particular, the proposed method learns the sequential connections between neighbor sleep epochs, which capture the dynamics of sleep stage transitions. The experimental results on the two public datasets, Montreal Archive of Sleep Studies (MASS) SS3 and  SleepEDF datasets, show that our method is competitive with state-of-the-art methods, as well as being medically interpretable in, contrast to the current automatic sleep staging methods.

\begin{figure*}[hbt!]
	\centering
	\includegraphics[width=14cm, trim=7cm 1cm 5cm 0.9cm]{Graphs.pdf}
	\caption{The binarized learned spatio-temporal graphs corresponding to different sleep stages from PSG recordings of MASS-SS3 dataset and averaged over the class samples. a) The significant conncetivity reduction of the non-Wake stages vs. Wake is shown in this figure. b) The notable increased connectivity between ECG and EEG channels in REM stage compared to the others is illustrated.}
	\label{Fig3}
\end{figure*}

\bibliographystyle{unsrt}

redpajama_set_name: RedPajamaArXiv


text:
<?php

namespace Ivory\GoogleMap\Helper\Renderer;

use Ivory\GoogleMap\Map;

/**
 * @author GeLo <geloen.eric@gmail.com>
 */
class MapBoundRenderer extends AbstractRenderer
{
    /**
     * @param Map $map
     *
     * @return string
     */
    public function render(Map $map)
    {
        return $this->getFormatter()->renderObjectCall(
            $map,
            'fitBounds',
            [$map->getBound()->getVariable()]
        );
    }
}

redpajama_set_name: RedPajamaGithub


LanaWhiteFlower ANGELSinBEDxxxxx jasmin. SaraCharm. 1SquirtXXshow. GoddessDevy.
saradominguez07TotalSexyForYou2StickyFingers1CelesteMilani .MatildaLoveBbStickyFingers1Eliza10IncheNiceRebeka .lynxmazesaradominguez071ZanderCuteGuyALEXOSORIO .MillenaGTestZilahicharmperfumeLustyHelga .TestZilahiAntonioMXxDANNALATINAxChalk .GregoryHandsomeDrPhilGood22ChocodirtyTSxdivalash .LeandroFtStebanHotDominik4USquirtyRachelHotDominik4U .shantalbetterBrigitteDommeIrinaBabetteNadiaKing .FIRMBODYHOT69divalashLustyHelgaChocodirtyTSx .BrigitteDommeStickyFingers1JohnTERMINATOR2ExoticLiberta .VanesaDanielleSaraxVegaLustyHelgaDebbyeDesire .
isabellaswonExoticLibertalouise21yourideaal .SamRicciiIrinaBabetteArasiKunBelleLarisa .HaneyBeesAntonioMXNEWmysterioustsantonylatino .VirginiaaMarikaDazzlingIASBELLAAinaWave .ScarlettBWildeIASBELLALeandroFtStebanChalk .BrigitteDommeVirginiaaHotDominik4UMatildaLoveBb .MarkDysondivalashNEWmysterioustsMillysa .PearlyWhiteEliza10IncheEmilyWallerNEWmysteriousts .isabellaswonAlysonCoxlouise21IrinaBabette .ariadnabigDanniTaylorEliza10IncheBrigitteDomme .SquirtyRachelUrTransMistrexXAlysonCoxandylove23 .


text:
Health selected
Cranberry juice 'can protect against urine infections'
By Michelle Roberts Health editor, BBC News online
https://www.bbc.co.uk/news/health-18768320
Image caption Compounds present in cranberries stop bacteria sticking to the wall of the bladder, studies show
Drinking cranberry juice to prevent urine infections does work say medical experts who have found scientific backing for this folk remedy.
The review in Archives of Internal Medicine looked at 10 head-to-head trials of cranberries versus no treatment in 1,500 volunteers.
But experts warn that the amount people must consume to gain this benefit - a couple of glasses daily for months or years - may be unacceptable to some.
And it doesn't work for everyone.
Cranberries contain a substance that can prevent certain bacteria from sticking on the walls of the bladder, studies show. These bacteria, called E. coli, cause over 80% of all urinary tract infections.
It can be costly and calorific and some people just don't like the taste of it
Dr Ruth Jepson, University of Stirling
The researchers who carried out the latest review found cranberry juice and other products containing cranberries that are sold in pharmacies can ward off some of these E.coli infections.
But they don't necessarily work every time or for all individuals.
They tend to work best in women experiencing recurrent urine and bladder infections.
And juice appeared to be slightly better than capsules or tablets containing cranberry, although more efficacy studies are needed to confirm this.
The National Taiwan University Hospital researchers who conducted the review said more work was also needed to discover the optimum dose.
Laboratory studies have shown the anti-adhesion activity of cranberry on bladder bacteria lasts for around eight hours after ingestion, which suggests a few doses a day might be ideal.
Far more common in women than men
Symptoms include burning on passing urine and the sensation of needing to pass urine frequently, as well as fever, shivers, and abdominal pains
They usually get better on their own
The mainstay of treatment are antibiotics
But since cranberry juice has a high sugar content, this might not be advisable for general health.
In one of the 10 studies the researchers looked at the dose had to be reduced because many of the women participating in the trial disliked drinking three glasses of the juice every day.
Dr Ruth Jepson of the University of Stirling has been carrying out a similar review into cranberry juice.
She said: "I'm not sure how many of us would want to be drinking this every day for an indeterminate amount of time. It can be costly and calorific and some people just don't like the taste of it."
Doctors advise people with urinary tract infections to drink plenty of fluid and the problem will usually get better on its own. They may recommend a course of antibiotics.
Cranberry can interact dangerously with warfarin medication, so patients on this blood-thinning drug are advised to avoid cranberry products.
Juice cocktail 'good for heart'
Archives of Internal Medicine
Cranberries for preventing urinary tract infections - Cochrane Summaries

redpajama_set_name: RedPajamaCommonCrawl


The handy-dandy content random this article is specifically for many reasons. One of the reasons is because some people the world need to know about the fantastic services the company golfer. One of the great things that they can offer you is the fact that they are absolutely the Best Plumbing Shawnee KS locals have ever experienced before. Human experience all this for yourself as you call 913.207.0779 to set up a fully inspection and consultation with one of their technicians today.
It is so simple to receive amazing services from the Best Plumbing Shawnee KS plumbers. The simple process is so easy that it will only take a few minutes out of your busy schedule. We don't want to make sure that you with all your time went towers on the computer Trinity arrondissement application. All you have to do is go to the website Quickreliefplumbing.com greatest musical top spots vacancy the very first page I have to scroll down to the bottom and will say exiled for a consultation. What should quick this artist's provide us with contact information and you are set.
I just met this information will be well your way to receive the Best Plumbing Shawnee KS has ever dreamed of. The fantastic because you don't have to worry about leaking pipes or anything else the plumbers deal with. Even if you don't even know what caused it with the can definitely help you. The colonoscopy to with and they will definitely tell you. Don't Phaedra fantastic services for fantastic price. The great discounts that they can offer you our senior discounts. Another discovery offer you is a military discount. Because of what you with 15% off any other cortisone provide to you.
That would make sure to provide you solutions all the plumbing problems you are experiencing. They always do it right the first time. There's no reason for you to worry about the quality products and parts they provide you because they definitely are quality. They're absolutely the best quality there's nothing else that you can compare them to. It's as if they know exactly but kind of quality you are looking for. And we know that you are looking for the best quality.
The nature that personal situation that they are put into the to the best possible job that they could ever imagine. Which is exactly what should pick up your phone and call them right away. Stick up your cell phone, cell phone, a phone, or any kind of cellular or cellular device and call the following number: 913.207.0779. If you want to call this number than another with you should contact them is through their website Quickreliefplumbing.com. We always want to provide you with a variety of options and contacting us to make sure that you are not limited in your ability to contact people or any other company. So just call them right away.
Are you sick of having crappy plumbers take care of your plumbing? There is still a terrible plumbers because there are so many other great options available for you out there in the world. Especially if you live in Kansas City are some fantastic plumbers. So if you are looking for the absolute Best Plumbing Shawnee KS . If you definitely need to choose Quick Relief Plumbing. This company is subject comes to plumbing of the can help you Is most of the week and I have to do is contact benefit the consultation through the phone number 913.207.0779.
Questions you can contact within they will be able to provide you with an appointment time for you to receive an absolutely for free inspection. Yes, you heard me right! That is a free inspection of any kind of services to are looking for. They want to make sure that you meet them and are comfortable with all of the services they offer before overcommitted to the services. This is been provided a free consultation to you can as we can leave you make sure that you are receiving the Best Plumbing Shawnee KS locals could ever hope for. You can experience all this in first opportunity set of appointment.
Once you set up consultation for the Best Plumbing Shawnee KS has to offer, you won't be disappointed because you can get play services for quality price. Community for yourself by sticking up testimonials on the website Quickreliefplumbing.com. The glitch that's a could see so many services to have available as both other people that helps previously. You definitely want to choose the best of the best by choosing the best bless you ultimately choose Quick Relief Plumbing. This understands want to help you every step of the way. There is no doubt that you should choose Quick Relief Plumbing.
They can provide you with some any guarantees included guarantee that their services are backed by the on budget services. They also want you to know that they are always on time. 100% of time there available to you and want to make sure that you don't waste any of your time waiting for our technicians. All technicians that we have or super honest and trustworthy knowledgeable when it comes to plumbing and other things. There is no need for you to worry about having technicians come into your home because they love helping you and all of your plumbing needs.
Just make sure that you contact them as soon as possible to get any kind of problem with your plumbing taken care of and efficient manner. 913.207.0779 of the numbers that you have to dial or to get in contact with their customer service representative or their office. This is absolutely amazing services that are available to you and you can also see the list of services on their website Quickreliefplumbing.com. Just know that we can help you with anything that you are looking forward to and just make sure that you contact them is as possible because they will meet any other quote available to you by 15%.


This online training course "Food Defense Awareness for the Intentional Adulteration Rule" was developed in partnership with the Food Safety Preventive Controls Alliance (FSPCA), Intentional Adulteration Subcommittee (IA Subcommittee).
This course is being offered free of charge and is available on the Food Safety Preventive Controls Alliance website and on the FDA's Food Safety Modernization Act (FSMA) website.
This course was designed specifically for individuals that are required to take a food defense awareness course to meet the requirements of the FSMA rule: Mitigation Strategies to Protect Food Against Intentional Adulteration (21 CFR Part 121) (IA Rule). Individuals who complete the online module will be able to generate a certificate of completion from the website.
More information about Food Defense and the IA rule.


text:
Home / Comics / Captain Kirk's Weekly Comic Review – 10/09/2019
Captain John K. Kirk 3 weeks ago	Comics
This week, it's the villains who come to our rescue as we consider the good … in being bad.
East of West #43
This book is always an intense experience to read. However, we have now entered what seems to be the endgame stage – of course, I've been thinking this for over a year now.
While Death's son has been found by the other riders of the Apocalypse, the real thrust of this story is the inevitable showdown between the House of Mao's Xiaolian Mao and the President of the Confederacy, Archibald Chamberlain on the field of battle.
While Mao fights for a higher purpose (the survival of her people), Chamberlain merely seeks to maintain his nation's position and exterminate and eradicate his enemy from the world's existence. I think the intensity in this book is in this battle when we see Chamberlain's true evil emerge.
Chamberlain is the epitome of evil in his wanton willingness to employ death and destruction in the thirst for his power. Anyone who isn't a friend is to be destroyed, and anyone who is a friend, not to be trusted. This is the state of things in the world in this world of Jonathan Hickman's.
Evil is in many forms in this comic, but the reason why it is so prevalent is because of the world-wide corruption involved in holding on to power. Other nations' leaders have been eliminated and the number of competitors for world domination has been diminished. However, the lesson that has been missed is that the struggle for power always destroys that which is controlled. That's why we have the Riders of the Apocalypse returning to bring their brand of destruction to the world.
Except for Death. Death sees that in order to preserve the world, the cycle has to be interrupted somehow. His wife and child are symbols of the need to break the chain of destruction that the world constantly seems to be in. But whether he succeeds or not is to be seen. After all, evil has been victorious to this point.
Hickman stretches my mind in the scope of how inhuman this view of the world's future is; it scares me that Nick Dragotta can actually envision it.
Oliver #4
Darick Robertson and Gary Whitta have brought a wonderful Dickensian-inspired villainy to this book. Corruption, the depreciation of the human soul and the notion of slavery are all themes in Dickens' Oliver Twist, and they are excellently illuminated by the contrast of the main character's innocence and intuitive understanding of these evils.
Oliver has learned to be good by seeing the harm that evil inflicts. It's a very successful exercise in contrast but it also showcases virtues of forgiveness and family, positive values made more meaningful as we continue to learn more and more about Oliver.
Robertson's art is always exemplary, but in this book, contextualized by such virtue, it's downright noble in its pursuit.
I have to confess though; I love the appearances of the Dickensian caricatures in their various incarnations; especially Bullseye. It was such a minor thing but I think it really captured the essence of this book and its intention.
Star Wars: Target Vader #4
When the best villain of the 20th and 21st centuries is the target of a bounty hunter cull, you get to see Darth Vader's mastery of evil at its finest. Part Four of Six from Robbie Thompson and Stefano Landini, titled "The Shot", these creators really capture the essence of Vader's instinctive ruthlessness in combat.
Of course, it's the Force – we all know that. But his emotionless mask emphasizes the fact that he is in control. There is no grimace of pain when he is wounded, no expression or sign of struggling; regardless of the odds, Vader has the advantage even when he is outnumbered.
I have always loved that feature of Vader's character and I think that's what Robbie Thompson has captured in this issue. Despite the bounty hunters' careful planning in luring Vader into a trap, it's clear that they don't stand a chance against the Sith Lord.
However, when there are scum and villainy involved, there's always a good, healthy sense of betrayal. In this battle, there's plenty of that and the focus on Vader distracts us away from the surprise at the end of this comic. It's a very cleverly arranged story.
Landini's artwork is staggeringly accurate in its likenesses, but I have to say that I also enjoyed seeing his interpretation on characters from the past like Beilert Valance, who I remember from Marvel's run on comics from the 1980's.
Darth Vader has made evil look … professional, and this is a wonderful read.
Doctor Doom #1
If Darth Vader is professionally evil, then Doctor Doom, the malevolent monarch of Latveria, makes evil look regal.
I've always enjoyed Doctor Doom's royal attitude. He is an absolute despot in a modern world, and that anachronism requires not only extra effort in maintaining the image but a great deal of power. He has an ample supply of both.
However, I am amazed that it took this long for someone to put a book about Doom. After all, in the past we have only seen snippets of Doom's life, his castle etc. This clearly will allow us to see a lot more insight and promises a greater emotional connection with the character.
Christopher Cantwell and Salvador Larroca have embraced this challenge whole-heartedly and I'm truly in awe at the depth they've managed to instill within Doom in this inaugural book. While we still see Doom the Despot (how he treats his servants, his interactions with members of the Press, or other super-powered individuals), we also get to see Doom the Insecure, the doubtful and even, Doom the Defeated.
Yeah, I'll spoil this much, but Doom gets beaten in the first issue of this comic book devoted to him. But, don't forget that even though he's a ruthless overlord, he's still a brilliant genius who managed to meld sorcery and science together. It's part of a bigger game and a character like Doom isn't in this for the short game. But there's a great theme of self-doubt and growing older in this book that we can all relate to. Even though Doom is a super-villain, he still has human failings and undeclared weaknesses.
I have always been a big fan of Salvador Larroca's art. I loved it in Star Wars and he doesn't fail to disappoint in this book either.
There's a tremendous amount of range in how we see this character in his own book and I have to say that I'm really interested to see how the story will turn out. Doom, the master strategist has found a new nemesis to overcome – himself.
That's why Doctor Doom #1 is the pick of the week. If Doctor Doom's worst enemy to overcome is himself in all of his hubris, ego and unwillingness to admit he is flawed, then what does that say about the rest of us?
Yeah, like I said, it's the villains who need to show us the value in being bad this week and though Doom's pursuit of power for his own evil and selfish means, he also has some good in the way that he does bear responsibility for his people. He values loyalty and good service and rewards those who offer him those virtues.
We always need to watch evil very carefully but this week, maybe we can learn something from it.
Pick of the Week: Doctor Doom #1
Captain Kirk's Weekly Comic Review - 10/23/2019
About Captain John K. Kirk
John Kirk is an English and History teacher and librarian in Toronto, Canada. In addition to the traditional curriculum, John tries to teach his students to make sense of geek culture. And with the name "J. Kirk," it's hard for him to not inject "Star Trek" into his lessons. Comics, RPGs and the usual fanboy gear make up his classroom resources.
@@capjkkirk


James Toseland will lose his MotoGP place to Ben Spies in 2010, Yamaha have today confirmed.
A Yamaha statement confirmed that Spies will make the switch up to MotoGP, taking Toseland's seat in the 2010 Championship.
Whilst the statement made no mention of Toseland's future, the British rider, who has struggled to meet expectations in his second season in MotoGP with the Yamaha Tech 3 team, has not been offered a contract to remain with Yamaha in MotoGP in 2010.
Spies will therefore partner Colin Edwards, who has also agreed a one-year contract extension, in what will be all-American Tech 3 garage. The move means that Britain is without a rider in the 2010 premier class.
"I want to thank Yamaha globally for their faith in me," said Spies. "They have treated me like royalty!
"Yamaha just recently allowed me to rethink my contract for 2010-2011; I did sign for WSB for 2010 but after Indy I began thinking of the competition and my age versus waiting until 2011.
"This was not an easy decision, the Yamaha World Superbike Team crew and especially Massimo (Maio) Meregalli have been fantastic and it's been great fun to put in a season together. I am hoping that I can give Maio, Laurens Klein Koerkamp and everyone else a championship to remember this year with me.
"I have enjoyed his series and this first year racing in Europe. Now I have two challenges, to complete this year giving my best and to move towards new goals in MotoGP. Herve Poncharal at Tech 3 is showing his support by allowing me to come on board with my Crew Chief Tom Houseworth and Gregory Wood, my mechanic, who have both been with me since AMA days. We will have learning curves to overcome next year but I am up for the new goal.
The arrangement is likely to see Toseland swap bikes with Spies, with the Yorkshireman taking the now vacant seat in the Sterilgarda Yamaha WSB team to compete in the championship that he won in 2007, although this has yet to be confirmed.
The decision will come as little surprise to British fans, with Toseland's MotoGP future under threat for much of the year after a slow start to his second season, coupled with Spies' emergence on the WSB scene.
The 28-year-old, who stands in 13th position in the championship, has finished in the top six on just three occasions this season and finished ahead of teammate Edwards on just one occasion, in Motegi back in round two, although Toseland also finished 10th when Edwards was taken out of the San Marino Grand Prix a month ago by Alex de Angelis.
Toseland's manager Roger Burnett was believed to have spoken to a number of teams, including Pramac Ducati and teams in the new Moto2 championship, but a return to WSB was always thought to be most likely.


text:
SUNRISE AT 4:29 AM
Ron Patton | May 8, 2010
While the stock market took a huge dip, and immigration debates generate a xenophobic attitude about earthly aliens. There are several other than earthly alien stories that have been in the news and while oil rig disasters and terrorism false flag events take center stage, there are other rings in this circus and in one of them is the "information creep" about an extra-terrestrial presence in space and on earth.
As we see the first attempts at "disclosure" there are people are looking for, and expecting, the wrong answers, when the correct, but far less glorious, fulfilling, and even mundane answers may well be right under our noses.
Most of the information about UFOS and aliens will most assuredly not be what the New Age alien cultists are expecting because many have put faith in their discoveries and made them spiritual and faith-based, sometimes bordering on, or fully, a delusional response to the phenomenon.
When our government or the world governments bring forth information on Aliens or UFO's it will be on their terms. Any information will be permitted and most certainly will not appease the majority of those that have put their time and money into theories of known unknowns. This would cause a schism and would most certainly become the first shot at developing a mix of science and religion for the ecumenical world order.
Believing there's something else "out there" (where ever that may be) that is largely unknown is one thing, but faith in specific attributes about that unknown thing, especially relating to the typical cataclysmic model, is quite another. People say that seeing is believing, but now we live in times where people are capable of creating sophisticated fakery, where stories are created in CIA board rooms and clear images of UFO's show up on the internet in order to show homegrown computer graphics packages.
One of the true skeptics I used to respect was Michael Shermer. I remember reading a book of his called the Science of Good and Evil and I was intrigued by how he seemed to be able to pull apart belief systems and put them back together. I have talked with him on my show at least twice. The second time I had him on I wanted him to give his skeptical take on aliens and disclosure. He gave his opinion that most UFO believers have not had any experiences themselves. The only things that people put their faith in are blurry pictures.
Some UFOS are even faked for effect and now on You tube we have seen sophisticated CGI programs create believable scenes where clear UFO's appear in cities and over exotic locales like Haiti. I have to admit that when I first saw these videos I was thinking that they were too clear to be faked and that the reaction and sounds seemed pretty believable. Until I looked carefully and realized I had been had.
[youtube up5jmbSjWkw]
Shermer lost points with me when during a TV special where he had children make fake pie tin UFO's, place them on strings and then proceeded to pass them off as real. He showed the pictures to people and claimed that he was writing a book on the subject and asked their opinions. Many seemed pretty ambivalent to the shots. Many were fascinated of course. Here was a guy in suit with a book ready peddle his fake UFO pictures.
[youtube o7Tu-28hyow]
He also took them to an analyst who on national TV was made to look foolish as he proceeded to demonstrate the realness of both pictures.
Later Shermer told the people his photos were faked and he seemed surprised by the response. The People still believed in UFO's. Shermer basically admonished people for believing in a bunch of blurry pictures.
I often tell people that this type of logic in the skeptic world baffles me. The argument that it is magical thinking that UFO believers still believe even though they haven't experienced a sighting is completely ludicrous. I believe in bank robberies even though I have never been in one. I have seen blurry photos of suspects placed on television. I still believe. I have seen faked robberies on television during crime dramas and I have seen police completely botch a case with mistaken identity and through all of this I still believe that bank robberies happen and that they exist.
[youtube EyygUjwc2ZM]
We are so quick to believe everything based on what we see, even though we can no longer trust our eyes. That is why the topic of UFO's is now one of those "I don't know yet" topics when shown visual evidence.
[youtube WqZsKdCYEPQ]
Some people go to great lengths to fake a visual experience and soon we may not be able to tell the difference.
[youtube fmxsZbssmDE]
There happened to be a UFO video that intrigued me and while I mentioned it briefly on my show back in December of 2009, I filed it in the "I don't know yet "category. Now I am wondering if maybe there could be something that indicates either a real UFO video with a Corroborating story – or just a hoax.
The twisted tale of UFO hijinks begins with a video that wound up on Russian television of a huge flying Triangle floating above Red Square. Two men were able to allegedly get up close to the craft at night in their car.
[youtube X_rR1IrVkik]
Someone from far away was able to get footage during the day of the black pyramid over the Kremlin.
[youtube fa2TN4ZSBm8]
Now before you cry out HOAX I need to acquaint you with a recent article I found, reported by The BBC.
The BBC is reporting that A Russian MP has asked President Dmitry Medvedev to investigate claims by a regional president that he has met aliens on board a spaceship. Kirsan Ilyumzhinov, the leader of the southern region of Kalmykia, made his claim in a television interview.
MP Andre Lebedev is not just asking whether Mr Ilyumzhinov is fit to govern. He is also concerned that, if he was abducted, he may have revealed details about his job and state secrets.
The MP has written a letter to Mr. Medvedev raising a list of his concerns.
In his letter he says that – assuming the whole thing was not just a bad joke – it was an historic event and should have been reported to the Kremlin.
He also asks if there are official guidelines for what government officials should do if contacted by aliens, especially if those officials have access to state secrets.
Mr Ilyumzhinov said in an interview on primetime television that he had been taken on board an alien spaceship which had come to planet Earth to take samples – and claims to have several witnesses.
He has been president of Kalmykia, a small Buddhist region of Russia which lies on the shores of the Caspian Sea, for 17 years.
Now don't you find it interesting that the Kremlin has not dismissed the story outright? They are merely expressing concern that Ilyumzhinov could have given state secrets to these alien beings. This was front page Russian news. A High ranking Government official discloses alien contact!
Now this abduction happened in Moscow in 1997. This is not a tongue and cheek story. The BBC is reporting with a straight face that The Russian MP Andre Lebedev is alleging that a Russian governor has been feeding highly classified state secrets to aliens. Ilyumzhinov appeared on Vladimir Posner's TV show in Russia. He is the equivalent of Larry King in the United States.
[youtube A5p0MT3wlMc]
This is right after Physicist Stephen Hawking had dropped a bomb on the new age alien loving crowds by stating that while aliens exist, we must not try to contact them or welcome them with open arms. Hawing said quote; "If aliens visit us, the outcome would be much as when Columbus landed in America, which didn't turn out well for the Native Americans."
It was after that statement from Hawking that the British Tabloid SUN reported that NASA was confident that life exists on Mars. This article according to NASA had everyone buzzing but of course NASA stated that the report was misleading and that there is no way they can make that claim.
Leave it to NASA to be tight lipped about obvious science and worldwide interest in Aliens. We dump some 17 Billion dollars of tax payer money into the Space program you'd figure we would be entitled to straight answers.
In 2009 when NASA Discovered Methane was being produced on Mars Scientists around the world concluded that methane could only be produced from life forms under the planet's soil. The first discovery of this was in 1976. Gilbert Levin was the scientist who found that the Viking Missions produced evidence of life On Mars. NASA immediately dismissed it. Europe has already declared that life exists on Mars—NASA seems to be clueless.
NASA has provided us with two rovers that are giving us breathtaking pictures and data. One can only speculate what news will come next. But of course those who control the information will give us only small bites to digest when we are hungry for more than crumbs. NASA is not just a 17 billion dollar science project.
NASA is a military installation claiming to be civilian. The billions of dollars going to "philanthropic space programs" are mostly funneling into the military industrial complex. Sure there are missions to send probes out in space. But the real reason for NASA is to research the idea of using space as the new battlefield. The dreams get the good press but the nightmares are performed in secret.
The statements by NASA indicating that reports of alien life are misleading is exactly why real disclosure will be difficult to verify.
You may not believe in Aliens, but it is evident that space agencies around the world do and that the United States Military, NASA and other black ops organizations have sensitive information that they have kept from the American people regarding strange anomalous activities in space. It is getting to the point where NASA is no longer capable of keeping everything "top secret." Some things cannot easily be explained by science.
And while UFO pictures can be fuzzy and many can be photo shopped and hazy there is one area of the spectrum that is pretty hard to fake. That is aural evidence of a UFO encounter.
In the year 2001 I filed a report for UFO Magazine called "The Secret Eye of the Triangle." In the story I was familiar with a UFO encounter with police officers in Millstadt Illinois. On January 5, 2000 at approximately 4:29 am Police officer Craig Stevens heard on his radio that the Highland Police Department had a report of a large object flying in the air. Several Police agencies were called in to investigate a huge Bright light that eventually came closer and was reportedly a three story Flying Triangle.
I had in my archives the recording of the incident on the police radio. The descriptions were dramatic and you could hear that the officers were shaken up by the incident.
Around the same time I received a recording of yet another incident where police officers had recorded on their police radios a UFO encounter. This encounter happened in St. Helens Oregon in 1981. According to the now defunct Pacific harbor press,
It was misty and very cold in the early morning hours of March 17, 1981, as Sergeant Russell Yokum of the Saint Helens, Oregon, Police Department Patrolled Highway 30 west of the Columbia River. Saint Helens is a small town about 20 miles northwest of Portland.
At 4:03 a.m., Yokum's attention was drawn to a bright light that was moving up the river, easterly, toward the Portland Airport 20 miles away. Aircraft pass over that area regularly on their way to the airport. But this light was extraordinarily bright, lighting up the river like daylight. Yokum was immediately convinced that the light was not from an aircraft. He radioed headquarters, and drove on to Saint Helens to look at the light from the county courthouse on the banks of the Columbia River, which afforded a clear view.
Other law enforcement officers, Ricky Cade and Tom McCartney of the Oregon State Police, and a few citizens, met Yokum at the county courthouse.
By this time, Yokum was in radio contact with Donald Askins, who had the CB handle "Lucky 13." Askins, who was located in a house across the river Ridgefield Washington, southeast of Saint Helens, had picked up the police radio traffic. He said that he had also seen the light, and was seeing it now. The light was stationary over the river turning the whole area into daylight.
There was initial confusion as the officers in Saint Helens described the light to Askins. They claimed it was bobbing up and down. Askins insisted that the light he saw was stationary. Later, it was proved that the light the officers saw during the first moments of the sighting from the court house was an industrial light on Sauvie Island across the river.
The Fog was creating the bobbing effect.
Finally, understanding their confusion, the officers turned and looked to the south. There they saw the light that Askins was watching!
It was low and standing out starkly over the river where there were few, and only then, faint lights.
Askins had heard the light emitting an eerie, extremely loud sound, and the officers set up a portable tape recorder 18 inches from their police radio to record their conversation and the sound, should it recur.
Askins volunteered to dangle his CB microphone out of the window of the rented house he was in so that he could transmit the sound to the officers. The police drove to a nearby high bluff that gave them an excellent view.
Yokum and Cade said the light was spherical, showing no structure. Cade said it was orange-red in color; Yokum, light orange. The round light had hovered 80 to 100 feet above the river, reflecting light off the water. The estimated size of the light was 30 feet in diameter.
With the reflection of low lying Fog and the water of the Columbia River it looked as if the Sun was rising at 4:29 in the morning.
The officers had commented that it was too early for the sun to be coming up. The Light was bright enough to cut through the fog and the sound it produced was something not heard on this earth from any machine. The officers described the noise as similar to a power plant's turbine engines whining.
The audio recording in this case and the Police officers story is compelling evidence that something very bizarre happened that night. Skeptics always say that extraordinary claims require extraordinary evidence. In my 15 years of investigating these stories I haven't heard anything like this.
It is truly extraordinary.
Tagged as aliens, Andre Lebedev, British Tabloid SUN, CGI programs, CIA, Columbia River, Craig Stevens, disclosure, Dmitry Medvedev, Kalmykia, Kremlin, Mars, Michael Shermer, Millstadt Illinois, NASA, Portland, Portland Airport, Saint Helens, Science of Good and Evil, Stephen Hawking, The Secret Eye of the Triangle, UFO believers, UFO hijinks, UFO Magazine, UFO video, UFOS, Vladimir Posner, xenophobic attitude.
9/7/18: QUELL W/ ELANA FREELAND
Ron Patton | September 7, 2018
5/18/18: ALIEN A.I. – ROLLING BACK THE DOMAIN OF THE GODS W/ ...
Ron Patton | May 18, 2018
IN HIS IMAGE: GAME OVER


"STAR WARS: THE FORCE AWAKENS | Official Movie Poster"
CL Nation, Disney has unleashed the official poster for STAR WARS: THE FORCE AWAKENS!
THE FORCE AWAKENS stars Harrison Ford, Carrie Fisher, Mark Hamill, Anthony Daniels, Peter Mayhew, Kenny Baker, John Boyega, Daisy Ridley, Oscar Isaac, Andy Serkis, Max von Sydow, Lupita Nyong'o, Crystal Clarke, Pip Anderson, Christina Chong and Miltos Yerolemou.
The STAR WARS saga continues with this seventh entry — the first under the Walt Disney Co. umbrella. The film will act as the start of a new trilogy set after the events of Return of the Jedi. J.J. Abrams directs from a script by Michael Arndt.


OE Synthetic Motor Oil provides excellent wear control and protects critical engine parts like pistons and cams. This added protection is particularly important in the extreme environments produced by today's smaller displacement engines that often run lower viscosity oil. Camshaft wear can result in poor valve timing, greatly reducing engine efficiency and leading to expensive repairs.
Click here to view the MSDS Data Sheet for AMSOIL OET.
Votes & Reviews: 32 vote(s) and 122 user reviews.
I like the OE products for my cars and trucks. My vehicles are "normal" and don't require any fancy, special-purpose oils. I can get Amsoil quality for my vehicles and not break the bank $!
the cost of other synthetics are in most cases more than Amsoil, even some synthectic blends are more expensive than Amsoil.........but the quality is NOT.
Not only does the oil come out much cleaner looking, but my gas mileage actually went up .5. Not much, but I thought it was pretty cool. Great product and I will continue to buy all AMSOIL products for the multiple different fluids my vehicles need.
I will use amsoil products the rest of my life!
I use this OE 10w30 engine oil in my can-am commander. The product takes the heat really well and stays clean. I will continue using it!
I have been using Amsoil for many years. I drive a 1997 Jeep Cherokee with close to 160k miles. The engine has had Amsoil for years. I use 10w30 and it runs fine. It uses a quart in 5k miles and that's when I change it. Great stuff.
I use Amsoil in everything I own. Cars, (3) Boat, Lawn Equipment and Harley Davidson, as well as a 2 cycle motocross motorcycle. I use the 10w-30 in my 3 cars. You might say I use almost every type of oil product Amsoil makes. I have never had a single mechanical failure, and have always had the best performance using Amsoil. I would not think of using anything else in my vehicles.
I use this oil in my c5 vette and gmc typhoon I drive these cars on weekends only less than a 1k miles a year .
So far I like synthetic oil. I'm new to using it. Love not having to worry about going little over 3,000 mile before changing. Will keep you posted as my experience with it matures.
SN rated PAO synthetic with low NOACK - bargain at this price from a helpful dealer focused on customer service.
Oil analysis showed 15% life left after double the recommended oil change interval.


Louis Riel T-shirt - Indig Inc.
Louis Riel t-shirt with the saying, " My People will Sleep for one hundred years, when they awaken it will be the Artists who give them back their Spirit. This shirt also includes his signature and a silhouette image of his face.
Louis Riel T-shirt With prophetic statement on uprising of Indigenous peoples . His signature is also on this shirt, it runs horizontally.