\section{Introduction} \label{sec:intro}
The black hole $X-$ray binaries (BXBs) are showing many types of spectral states in observations, and their states can be represented on the hardness-intensity diagram (HID) \citep{fbg04,rbm17,dzp21,drd21}. This diagram is also called as q- diagram, see first plot of middle row in Fig.\ref{fig1}. It is believed that a typical BXB can follows the cycle from quiescence state (QS) - low hard state (LHS) - high hard/soft intermediate states (HISs) - high soft state (HSS) - low intermediate states (LISs) - quiescence state (QS). Commonly, the jets are found in LHS to HISs \citep{mr98}, and when they launched from the disk, then a source enters HSS \citep[In case of full outburst][]{drd21}. Moreover, the quasi-periodic oscillations (QPOs) are also associated during LHS to HIS \citep{rm06}.

We know that the multicolor blackbody spectrum component of the spectral states is well explained by a non-advective cool Keplerian disk (KD), which was proposed in 1973 \citep{ss73}. We used a hot corona for the understanding of high energy power-law components in the study of active black holes (BHs). The origin of this corona is a mystery! Eventually for the modeling of the sources, there are some advective solutions  or lamppost model (unknown origin or may be jet base) was proposed by the authors \citep{ct95,emn97,frs89}, which can act like the hot corona. Those advective solutions can also generate the outflows/jets. In this regards, there are two hot advective solutions are very popular and named as the shock solution \citep{c89}, and advection-dominated accretion flow (ADAF) solution \citep{ny94}. Interestingly, many authors have evidenced in their studies that the ADAF and KD can be coupled due to some instabilities \citep{emn97,h96,kg19}.  Thus the KD can also work as a gas source for the ADAF.
This radial coupling model is also known as two-zone radial hybrid disk model \citep{emn97}, and the non-advective (Keplerian) to advective (sub-Keplerian) flow coupling region is called as transition region. Eventually, many studies have used this model to explain the spectral states and their transitions with adjusting the location of the transition region ($R_t$), and mass accretion rate \citep{emn97,fph06,mfp18}. In BXBs, it is believed that the accretion rate is increasing during QS - LHS - HISs - HSS path and decreasing during HSS - LISs- QS path, but both paths are different.
Now the problem is here and explained in the following section.
\section{HID with Two-component flows}\label{sec:2cf}
If the ADAF-KD radial coupling is true and the ADAF can act like corona and also generate outflows then the presentation of the observed spectral states can not be similar to q$-$ diagram. It would be look similar to $\ammaG-$ diagram (flipped $\Gamma$), and the source can not follow the left part of the HID. Since when the source reached on HSS and having KD, which can be extended to ISCO radius. Now, the source can move to LISs/QS, so the KD must disappear\footnote{The KD can be disappeared two-ways: 1. It can disappear from inside with increasing the location of transition radius, 2. It can disappear from outside or in other words, the KD is not truncated. Both ways can have different time scales of the KD disappearance, and the later case can take much time due to non-advective nature of the KD flow (viscous time-scale is large). We prefer former case and reason explain in main text, and addendum.}
via same advective coupling flow, and same way but with increasing location of $R_t$ due to lack of sufficient mass accretion rate at the outer boundary, which means that the inner KD disk is gradually converting to the ADAF, so retracing $R_t$ (Opposite to $R_t$ is decreasing during LHS - HISs - HSS). Thus, the spectral state should be retraced on the HID (like HSS - HISs - LHS), and finally, the KD is disappeared, and source reach to QS stage. However, this is not happening in the BXBs, but it is true that the KD should disappear from the HSS - LISs - QS.
Thus, there is a two possibilities here:
\begin{enumerate}
\item The ADAF-KD coupling is not existed, so when the inner part of KD flow is reached to the ISCO radius, and should be disappeared from the outside through the ISCO due to no stable orbits within the ISCO. Thus this scenario favors the two-component accretion flow (TCAF) model, which is also a hybrid disk model but the KD is sandwiched between the sub-Keplerian hot flow \citep{ct95}. This hot flow can also be have shocks in the inner region of the disk depending on the outer boundary conditions (OBCs), which are explained in detail \citep{ky21}. But the disappearance of the KD from outside is not suitable with the behavior of BXBs path from HSS$\rightarrow$ LISs$\rightarrow$ QS because the last emitted photon should be soft photon, which should come from the ISCO and last inner part of the KD. So technically, the QS should be located in left part of the HID, and the HID should be similar to $p-$ shape not q- shape. Thus the TCAF model is not suitable to explain the HSS$\rightarrow$ LISs$\rightarrow$ QS path. So, we prefer that the KD should disappear from the inside through the ADAF coupling, and it can produce the weak non-thermal photons, thus the QS can be located in the right side of the HID, and follows the q- shape. Moreover, the failed \citep[or hard-only, restricted-successful][]{drd21} outbursts can also supports for the ADAF-KD coupling, since in those cases, the source can reach to any low state (LIS or QS) after the outburst without following HSS, means the truncated KD can not reach to the ISCO. Now the truncated KD must disappear through the ADAF/ADAF-like coupling flow with approaching towards the QS or other low states.
\item The ADAF-KD coupling is existed, but inner part of the ADAF disk can not produce enough Comptonized photons maybe due to lack of seed soft photons, and weak intrinsic radiations (Bremsstrahlung and synchrotron), so failed to explain the LHS and HISs. The lack of soft seed photons maybe due to radial coupling geometry, which can not  provide the enough interception of the photons of the KD with the inner hot part of ADAF. Moreover, the outer part of the ADAF has higher gas density, and can work as the warm corona or reflector for the KD photons due to moderate/high optical depth with intermediate/low temperature gas, which depends on the location of $R_t$, and transition accretion rate ($\dot{m}_t$) at $R_t$.
This behavior is found natural in the theoretical study of ADAF when matter becomes advective from the non-advective cool KD flow \citep{ky21}.
\end{enumerate}
Overall, the key points are that individually, both (ADAF-KD coupling or TCAF) models are unable to explain the whole behavior of the BXBs, and it is also clear that the ADAF-KD model is not suitable to act like corona or produce the outflowing gas in the LHS and HISs.
However, this coupling is much suitable for the understanding of the HSS$\rightarrow$ LIS$\rightarrow$ QS path.
Thus, the ADAF-KD coupling disk needs some extra (or third) component of sub-Keplerian hot accretion flow, which can explain the high hard X-rays and generation of bipolar jets. Or in other words, the insertion of the ADAF component in the TCAF model. So, the hybrid disk structure can has total three component of the accretion flow, which are the KD, inner ADAF and third component.
This Three-Component of Accretion Model (abbreviated as TCAM) can has many advantages in explanation of the typical and peculiar properties of the BXBs, like, understanding of the soft-excess radiation, help in formation of the outflow/jet with the gas of the third component through ADAF barrier, heartbeat states \citep[repeatedly moving between HSS and HIS][]{fbg04} of GRS 1915+105 through intermittent inflow of the third component, and failed outburst \citep[not following all the HID stages, especially skip the HSS stage][]{drd21}, and so on.  We can also assume that the quick changes in the active galactic nuclei (known as `Changing-look' active galactic nuclei, AGN) can also be understood with change in the inflow rate of the third component.
The detail properties of the TCAM is explained in the following sections. Importantly, we would like to mention that the gas of KD can produce the ADAF like flows, which are mostly sub-sonic nature (becomes transonic very close to the BH),
and the third component can undergo shock transition when becomes super-sonic far away from the BH.
So both advective flows (ADAF and third component) are different in nature and the third component is mostly hotter and faster \citep{ky21}.
Hereafter, we will call the third component as a hot component flow.
\section{Properties of inner ADAF}\label{sec:adaf}
\cite{ky21} have shown that the nature of the advective solutions can be changed with changing temperature ($T$) of the accreting gas at an outer accretion boundary (AB), which changed the viscous time scale of the flow, since $\tau_{\mu\nu}\propto T$, where $\tau_{\mu\nu}$ is the viscous stress tensor. The viscous time scale changes the angular momentum (AM) transportation and viscous heating in the flow, which can change the dynamical time scales and radiative emissivities.
It has also been found that the ADAF or ADAF-like flow is least advective and highest AM flow in the all possible general advective (sub-Keplerian) flows.
We have noticed some interesting properties of the ADAF with changing size of it as following:
\begin{itemize}
\item The local specific energy (or Bernoulli parameter, say $B$ ) of the ADAF is decreasing with decrease in $R_t$, and when $R_t<50r_g$ then whole flow has $B<1$ or $0$ \citep{kg18,kg19,ky21}. So the ADAF may not has outflows due to negative local energy as mentioned in \cite{ny95}.
\item The AM distribution, and optical depth of the ADAF raises with deceasing $R_t$, even keep same inflow rate, for instance, when $R_t\lower.5ex\hbox{$\; \buildrel < \over \sim \;$} 100r_g$ then the ADAF flow becomes optically slim/thick (optical depth $\tau_T>1$, for $\dot{m}_t=0.01\dot{M}_{\rm Edd}$ see left panel of Fig.\ref{fig3a} in appendix \ref{app:caf}). If $R_t<10r_g$ then the AM distribution of flow becomes close to the Keplerian AM distribution, which may behave like optically thick slim disk, and may contribute in very high soft-state. Moreover, the temperature and inflow velocity decreases with decrease in $R_t$.
\end{itemize}
So the small/intermediate size of the ADAF can produce intermediate energy part of the spectrum (maybe soft-excess radiation), specially, the outer part of ADAFs with Compton thick/slim scattering, which is close to the KD.
So the HSS -LIS- QS and soft-excess (depends on accretion rate) can be suitably understood by the ADAF-KD model.
The key point  is that the ADAF is changing their nature with size and accretion rate, so one could be very careful during modeling of the BXBs with adjusting the $R_t$. Moreover,  qualitative power-law relation between $R_t$ and flow variables of the ADAF is presented in \cite{kg19}, which can be used in the modeling.
\section{Three-component flows}\label{sec:3cf}
As we discussed above that for understanding of LHS and HISs stages require the hot component along with the ADAF-KD model. This hot component can produce the high energy power law component and outflowing gas, and it can disappear with launching off the jet due to lack of accreting gas at the AB. The strength of the jet should be depended on the strength (mass flux) and nature (magnetized/non-magnetized) of accreting gas of the hot component. It can also be considered that the jet strength can also depends on the ADAF strength (rigidity), which can provide partial/strong barrier to the gas of the hot component, and the gas can turn upward (some details in appendix \ref{app:caf}). The strength of the ADAF is increased with decreasing size of it (or $R_t$), which raised the gas density and AM of the flow \citep{ky21}.
In the typical HID, we believed that the KD flow is evolved from LHS - HISs - HSS, and disappeared during HSS - LISs - QS, but origin of the KD flow is very poorly understood.
Now, there could be three possibilities of formation of it with the hot component flow, and temperature distribution of the accreting gas at the outer AB:
\begin{enumerate}
\item The part of sub-Keplerian gas flow can condensate to form the KD or KD-like flow around the equatorial plane, which has local thermal equilibrium (LTE). Unfortunately, this physical process is not much understood but recently, we have found some possibility to form the KD flow on the equatorial plane in the two-dimensional inflow-outflow study with two viscous stress tensor namely, $\tau_{r\phi}$ and $\tau_{\theta\phi}$ and this work is under preparation \citep{kgy21}. When the gas moves upward then the AM of the flow can effectively transfer towards the equatorial plane, so the equatorial gas can obtain the Keplerian distribution with some suitable combinations of both viscous tensors. Thus we believed that this may happen in the real  situation with the help of the viscous stress tensors or other kinds of the dissipative processes in the accretion flow.
\item Recently, we have investigated two types of the inflow gases at the outer AB, namely, the hot ($T_{AB}\sim T_{vir}$) and cold ($T_{AB}<<T_{vir}$) gases, and both produced different types of AM distributions in the solutions, where $T_{vir}$ is a Virial temperature \citep{ky21}. So, it indicates that the cold accreting gas at AB can produce the KD or KD-like disk with some dissipative processes, since the cold accreting gas always produced high AM flow (low efficiency of AM transfer) than the hot accreting gas. Other the hot accreting gas can generate sub-Keplerian hot accretion flow (3rd component), which can make corona-like structure and jet gas close to the BH.
It is natural that the accreting gas at outer AB can has non-uniform temperature distribution. Recently, formation of two-phase (cold gas surrounded by hot gas) inflow gas due to a thermal instability has been investigated very far away from the BH \citep{byz20}.
\item Third possibility is both (points 1 and 2) processes can help in formation of the KD flow\footnote{The KD formation rate may depend on $T_{AB}$ of the accreting gas. We believed that if $T_{AB}<<T_{vir}$ then formation rate can be higher since this gas gives least advective and higher AM flow \citep{ky21}.}.
\end{enumerate}
The key point is that the both flows (hot sub-Keplerian and cool KD) should evolve together during the LHS to HIS, and the KD flow should evolve close to the equatorial plane due to high AM. Interestingly, there is an interpretation of the two simultaneous flows on the basis of observations \citep{shs02}.
\section{Summary and Discussion}\label{sec:sum}
Based on the above new views with the latest theoretical understanding of the accretion flows \citep{kg18,kg19,ky21,kgy21}, there are the typical hybrid disk configurations can possible to explain the HID stages, which are presented in the Fig.\ref{fig1} with assuming all the above stated possibilities and physical processes in the flow. The dominance/strength of each component of these configurations of the disk is dependent on the quality and quantity of the accreting gas at the outer AB.
In starting with QS stage (Fig.\ref{fig1}A), it has very weak or no accretion flow. When the accretion flow is evolved enough
 then BXB enters to LHS stage with filling space around the BH. When LHS is starting to evolve with luminosity means both flows (sub-Keplerian and KD) are building up the disk. The key point is that the condensation/formation of the KD disk should be much faster than the evaporation in the inner part of it. The evaporated gas can make inner ADAF or ADAF-like disk, which also get stronger as the other both flows evolved.
Now, the possible roles of the ADAF disk can i) produce weak intrinsic radiations (Bremsstrahlung and Synchrotron emission), and soft-excess (Compton thick/slim scattering) when $R_t\lower.5ex\hbox{$\; \buildrel < \over \sim \;$} 100r_g$, ii) make barrier (centrifugal and pressure supported surface) for in-falling gas of the hot component flow, which can help in the jet generation or storage of the gas around the inner region of the disk  or when preceding flow is compressed enough then the flow can make strong shock in the disk. Although, the shocks can be generated without the additional ADAF barrier as shown in many theoretical and numerical studies \citep{kg18,ky21,lck16,gbc20,soa21}, and the post-shock region is much hotter than other flows, which can produce very high energy radiations, like, hard $X-$ray, and soft $\gamma-$ray.
This shock can be standing or quasi-steady or moving inward in accretion flow, and these kind of geometries is represented for the LHS and HISs.
This geometry has total three-accretion components in the disk Fig.\ref{fig1}(plots B,\& C). Here, the upper hot component can act as corona as well as base of the outflows, and the KD can produce seed photons for Compton scattering, ADAF, and thermal component of the spectrum.
When the upper hot flow is disappeared with launching off jet due to lack of the accretion gas at AB, then the transition happen from hard HIS to soft HIS stage. Now the KD with inner ADAF disk is remained and mostly the thermal photons can be seen in the observations during the HSS stage (Fig.1D). The KD is also gradually disappeared via ADAF due to absence of the accreting gas at AB (So, no further evolution of the KD), and source moves to the LISs then QS stage with increasing $R_t$.

 Interestingly, It has been found in the MHD simulation that the magnetic field (both toroidal and poloidal) is rapidly evolved in the post-shock region of the hot component than other part of the disk, which will be helpful to make outflows faster and collimated \citep{gbc20}. Moreover, if the ADAF can thread with the magnetic field lines then the field lines can be hold relatively against the gas of hot component, since the ADAF is always slow moving inward due to high AM.
 So, the possibility of interaction of field lines with the plasma is higher in the TCAM, over the co-moving of the plasma and lines in same component. Thus the chance of plasma of upper flow in the disk can move along the poloidal field lines is higher with support of the ADAF barrier.
 This scenario can also be suitable for the magnetic reconnection with the interception of the magnetic lines of fast and slow moving flows in the inner region (Fig.\ref{fig2}).
 Moreover, as the KD approaches towards the BH with increasing accretion rate, so the ADAF reduces size, and becomes stronger. At the same time, momentum flux of the upper hot component is also increased with moving inward as decreasing the $R_t$ (see right panel of Fig.\ref{fig3a}). So, the impact of interaction of both flows will be stronger, which can turn the upper flow more strongly upward, so it can make the stronger jet when moving from LHS to HISs.
 So the ADAF component could be the key component in the TCAM, which created new possibilities of physical processes in the inner part of the disk together with the upper hot component for the understanding of the jet generation, launching and evolution of jet strength during LHS to HISs.

The presented model scenario can be tested in future with numerical simulation study of the accretion flow with the accreting gas at the AB, which has temperature distribution (can be magnetized/non-magnetized), broadly two types of gases (hot $T_{AB}\sim T_{vir}$ and cold $T_{AB}<<T_{vir}$) with relevant radiative and viscous dissipations. We believed that a suitable numerical setup can generate hot sub-Keplerian and cool Keplerian disk flow with large AB locations due to their different viscous and radiative time scales.
We are also expecting some modeling studies of the BXBs in near future.

\begin{figure*}
\begin{center}
\includegraphics[width=0.3\linewidth]{fig1d-eps-converted-to.pdf}
\includegraphics[width=0.3\linewidth]{fig1c-eps-converted-to.pdf}
\end{center}
\begin{center}
\includegraphics[width=0.2\linewidth]{fig1q.jpg}
\includegraphics[width=0.3\linewidth]{fig1b-eps-converted-to.pdf}
\end{center}
\begin{center}
\includegraphics[width=0.3\linewidth]{fig1e-eps-converted-to.pdf}
\includegraphics[width=0.3\linewidth]{fig1a-eps-converted-to.pdf}
\end{center}
\caption{
An illustration of the HID (q$-$diagram) in middle of first column, and typical disk structures from A to E, which can represent the various spectral states of BXBs. The $H/r$ height profiles are obtained from theoretical solutions by numerically solving general relativistic fluid equations \citep{ky21}, except the KD flow, which is sketched with a dash-dotted line.
The vertical line (magenta color) is representing transition radius ($R_t$), and arrow is representing moving direction.
\label{fig1}}
\end{figure*}
\begin{figure}
\begin{center}
\includegraphics[width=\linewidth]{fig2.jpg}
\end{center}
 \caption{
An illustration of the disk-jet system for proposed TCAM scenario.
 }
\label{fig2}
\end{figure}

\begin{acknowledgments}
This work is funded by the National Natural Science Foundation of China (grant Nos. 11725312, 11421303). R. K. is supported by CAS Presidents International Fellowship Initiative (PIFI), Grant number 2021PM0017.
\end{acknowledgments}


\section{Introduction}
\label{sec:intro}
\input{1_intro}

\section{Related Work}
\label{sec:related}
\input{2_related}

\section{Methodology}
\label{sec:method}
\input{3_methodology}

\section{Experiments}
\label{sec:exp}
\input{4_exp}

\section{Conclusion}
\label{sec:conclusion}
\input{5_conclusion}

\begin{acks}
\input{6_ack}
\end{acks}

\bibliographystyle{ACM-Reference-Format}

\subsection{Preliminaries}

\paragraph{Graph Neural Networks.} Consider an attributed graph $\mathcal{G}=\{\mathcal{V}, \mathcal{E}\}$, where $\mathcal{V}$ is the set of nodes and $\mathcal{E}$ is the set of edges. GNNs use the same model framework as follows~\cite{hamilton2017inductive,xu2018powerful}:

\begin{equation}
\label{eq:gnn}
h_i^k=\phi(h_i^{k-1}, \mathcal{F}_{agg}(\{h_j, j\in\mathcal{N}(i)\}))
\end{equation}
\noindent where $\mathcal{F}_{agg}$ represents an aggregation function and $\phi$ represents an update function. The objective of GNNs is to update the embedding of each node by aggregating the information from its neighbor nodes and the connections between them.

\subsection{GIPA Architecture}

In this part, we present the architecture of GIPA\xspace, which  extracts information from node features as well as edge features in a more general way. The process of GIPA\xspace is shown in~\figurename~\ref{fig:gipa_process}. Consider a node $i$ with feature $h_i$ and its neighbor nodes $j\in\mathcal{N}(i)$ with feature $h_j$. $e_{i, j}$ represents the edge feature between node $i$ and $j$. The problem is how to generate an expected embedding for node $i$ from its own node feature $h_i$, its neighbors' node features $\{h_j\}$ and the related edge features $\{e_{i, j}\}$.

The GIPA\xspace process mainly consists of three parts: \texttt{attention}, \texttt{propagation} and \texttt{aggregation}. Firstly, GIPA\xspace computes the embedding $\tilde{h}_i$, $\tilde{h}_j$, and $\tilde{e}_{i, j}$ by means of linear projection from the features $h_i$, $h_j$ and $e_{i,j}$, respectively. Secondly, the \texttt{attention} process calculates the multi-head attention weights by using fully-connected layers on $\tilde{h}_i$, $\tilde{h}_j$, and $\tilde{e}_{i, j}$, respectively. Thirdly, the \texttt{propagation} process focuses on propagating information of each neighbor node $j$ by combining $j$'s node embedding $\tilde{h}_j$ with the associated edge embedding $\tilde{e}_{i, j}$. Finally, the \texttt{aggregation} process aggregates all messages from neighbors to update the embedding of $i$. The following subsections introduce the details of each process.

\begin{figure*}
    \centering
    \includegraphics[width=1.0\textwidth]{figures/gipa_process.png}
    \caption{Process of GIPA\xspace, which consists of \textit{attention}, \textit{propagation}, and \textit{aggregation} process.}
    \label{fig:gipa_process}
\end{figure*}

\subsubsection{Attention Process}

Different from the existing attention mechanisms like self-attention or scaled dot-product attention, we use MLP to realize a multi-head attention mechanism.
The \texttt{attention} process of GIPA\xspace can be formulated as follows:

\begin{equation}
\label{eq:att}
\tilde{a}_{i, j} = \mathcal{F}_{att}(\tilde{h}_i, \tilde{h}_j, \tilde{e}_{i, j}|W_{att}) = \text{MLP}([\tilde{h}_i || \tilde{h}_j || \tilde{e}_{i, j}]|W_{att})
\end{equation}

\noindent where the attention function $\mathcal{F}_{att}$ is realized by an \texttt{MLP} with learnable weights $W_{att}$ (without bias). Its input is the concatenation of the node embedding $\tilde{h}_i$ and $\tilde{h}_j$ as well as the edge embedding $\tilde{e}_{i, j}$. As the scale of parameters of \texttt{MLP} is adjustable, this attention mechanism could be more representative than previous ones simply based on dot-product. The final attention weight is calculated by an edge-wise \texttt{softmax} activation function:

\begin{equation}
\label{eq:softmax}
a_{i, j} = \text{softmax}(\tilde{a}_{i, j}) = \frac{\exp(\tilde{a}_{i, j})}{\sum_{k\in N(i)}\exp(\tilde{a}_{i, k})}
\end{equation}

\subsubsection{Propagation Process}

Unlike GAT~\cite{velivckovic2017graph} that only considers the node feature of neighbors, GIPA\xspace incorporates both node and edge features during the \textit{propagation} process:
\begin{equation}
\label{eq:prop}
p_{i, j} = \mathcal{F}_{prop}(\tilde{h}_j, \tilde{e}_{i,j}|W_{prop}) = \text{MLP}([\tilde{h}_j || \tilde{e}_{i, j}]|W_{prop}),
\end{equation}
\noindent where the propagation function $\mathcal{F}_{prop}$ is also realized by an \text{MLP} with learnable weights $W_{prop}$. Its input is the concatenation of a neighbor node embedding $\tilde{h}_j$ and the related edge embedding $\tilde{e}_{i, j}$. Thus, the $propagation$ is done edge-wise rather than node-wise.

Combining the results by \texttt{attention} and \texttt{propagation} by element-wise multiplication, GIPA\xspace gets the message $m_{i, j}$ of node $i$ from $j$:
\begin{equation}
\label{eq:msg}
m_{i, j} = a_{i, j} * p_{i, j}
\end{equation}

\subsubsection{Aggregation Process}

For each node $i$, GIPA\xspace repeats previous processes to get messages from its neighbors. The \texttt{aggregation} process first gathers all these messages by a reduce function, summation for example:

\begin{equation}
\label{eq:sum}
m_{i} = \sum_{j\in\mathcal{N}(i)}m_{i, j}
\end{equation}

\noindent Then, a residual connection between the linear projection $\hat{h}_i$ and the message of $m_i$ is added through concatenation:

\begin{equation}
\label{eq:proj}
\hat{h}_i=W_{proj}\tilde{h}_i
\end{equation}

\begin{equation}
\label{eq:agg}
o_{i} = \mathcal{F}_{agg}(m, \hat{h}_i | W_{agg}) = \text{MLP}([\sum_{j\in\mathcal{N}(i)}m_{i, j} || \hat{h}_i]|W_{agg})
\end{equation}

\noindent where an \texttt{MLP} with learnable weights $W_{agg}$ is applied to get the final output $o_{i}$.
Finally, we would like to emphasize that the process of GIPA\xspace can be easily extended to multi-layer variants by stacking the process multiple times.
\subsection{Dataset and Settings}

\textbf{Dataset.} In our experiments, the \textit{ogbn-proteins} dataset from OGB~\cite{hu2020open} is used. The dataset is an undirected and weighted graph, containing 132,534 nodes of 8 different species and 79,122,504 edges with 8-dimensional features. The task is a multi-label binary classification problem with 112 classes representing different protein functions.

\noindent \textbf{Evaluation metric.} The performance is measured by the average ROC-AUC scores. We follow the dataset splitting settings as recommended in OGB and test the performance of 10 different trained models with different random seeds.

\noindent \textbf{Hyperparameters.} The hyperparamters used in GIPA\xspace and its training process are concluded in \tablename~\ref{tab:hyper}.

\noindent \textbf{Environment.} GIPA\xspace is implemented in Deep Graph Library (DGL)~\cite{wang2019deep} with Pytorch~\cite{paszke2019pytorch} as the backend. The experiments are done in a platform with GeForce TITAN RTX GPU (24GB RAM) and AMD Ryzen Threadripper 3960X CPU (24 Cores).

\begin{table}[t]
\caption{Hyperparameters of GIPA\xspace model.}
\begin{tabular}{lc}
\hline
\textbf{Hyperparameter}      & \textbf{Value} \\ \hline
Node embedding length        & 80             \\
Edge embedding length        & 16             \\
Number of attention layers   & 2              \\
Number of attention heads    & 8              \\
Number of propagation layers & 2              \\
Number of hidden units       & 80             \\
Number of GIPA layers        & 6              \\
Edge drop rate               & 0.1            \\
Aggregation                  & SUM            \\
Activation                   & ReLU           \\
Dropout of node embedding    & 0.1            \\
Dropout of attention         & 0.1            \\
Dropout of propagation       & 0.25           \\
Dropout of aggregation       & 0.25           \\
Dropout of the last FC layer & 0.5            \\
Learning rate & 0.01            \\
Optimizer & AdamW~\cite{loshchilov2017decoupled}  \\ \hline
\end{tabular}
\label{tab:hyper}
\end{table}
\begin{table}[!h]
\caption{Test and validation performance (ROC-AUC) on \textit{ogbn-proteins} dataset.}
\begin{tabular}{lcc}
\hline
Method              & Test ROC-AUC  & Validation ROC-AUC \\ \hline
MLP                 & $0.7204\pm0.0048$ & $0.7706\pm0.0014$      \\
GCN                 & $0.7251\pm0.0035$ & $0.7921\pm0.0018$      \\
GraphSAGE           & $0.7768\pm0.0020$ & $0.8334\pm0.0013$      \\
DeeperGCN           & $0.8496\pm0.0028$ & $0.8971\pm0.0011$      \\
GAT                 & $0.8682\pm0.0018$ & $0.9194\pm0.0003$      \\
UniMP               & $0.8642\pm0.0008$ & $0.9175\pm0.0006$      \\
UniMP+CrossEdgeFeat & $0.8691\pm0.0018$ & $\mathbf{0.9258\pm0.0009}$      \\
GIPA (ours)         & $\mathbf{0.8700\pm0.0010}$ & $0.9187\pm0.0003$      \\ \hline
\label{tab:performance}
\end{tabular}
\end{table}
\subsection{Performance}
\tablename~\ref{tab:performance} shows the average ROC-AUC and the standard deviation for the test set and the validation set, respectively. The results of the baselines are retrieved from the ogbn-proteins leaderboard\footnotemark[1]. Our GIPA\xspace outperforms all previous methods in the leaderboard and reaches an average test ROC-AUC higher than 0.8700 for the first time.