\section{Conclusion}
In the state of the art on person re-identification, inference cost is widely neglected, and adding attention blocks at only few positions was not considered so far.
We have shown that adding a channel-wise attention block at a single position in a ResNet-50 is sufficient to perform better in person re-identification than a ResNet-101.
Therefore, the common practice of adding attention to each residual block should be questioned.

In addition, due to the progressive reduction of the design space during neural architecture search, we were able to derive a set of rules for where attention blocks should be integrated in a ResNet architecture.
By integrating the proposed attention block C-NL at three positions in a ResNet-50 derived by the NAS, we improved the mAP for the Market-1501 dataset, surpassing the performance of the ResNet-101 by a large margin, while slowing down the inference by only 5\%, which is marginal compared to the 45\% slowdown caused by doubling the depth.
Combining this architectural modification with a large-margin loss function further improved the re-identification ability.
Compared to HAC block \cite{li2018harmonious}, which slows down inference by 9\% (nearly twice as much as C-NL) and performs $1.6\sigma$ worse in this setup, the modifications of the C-NL attention block building on the channel-wise attention block setup of HAC proved effective.

Finally, we confirmed the performance gain for robotic data when transfer learning with pre-training on a re-id benchmark dataset is applied.
The improved mAP shows that few attention blocks in the architecture enable the extracted re-id features to better discriminate individuals in difficult cases where individuals look very similar.
This will help the robot to better keep track of its users and will consequently improve clinical applications where a mobile robot needs to follow or guide its user.

In future work it would be interesting to apply the proposed approach to neural architectures designed for mobile devices that have shown fast inference speed on Jetson hardware, such as ShuffleNet v2 \cite{ma2018shufflenet} or MobileNet v3 \cite{howard2019searching}.
We are curious if the same design rules apply and if a single channel-wise attention block is sufficient for these architectures as well.

\section{Experiments on a Robotic Dataset}
\label{sec:robotic_dataset}

To examine, whether our improvements by using attention blocks at few positions can also be achieved on robotic data, we conducted an experiment on an extended version of the ROREAS dataset, which was recorded with a mobile robot following a user in a clinical environment \cite{eisenbach2015user}.
We will refer to this extended version as ROREAS+ in the following.

The ROREAS+ dataset differs significantly from typical re-id benchmark datasets, like Market-1501 \cite{zheng2015scalable}, DukeMT\-MC-reID \cite{ristani2016performance}, or CUHK03-NP \cite{li2014deepreid}.
These re-id benchmark datasets have been obtained from footage taken from outdoor surveillance cameras mounted at several meters height.
In contrast, our target robotic platforms, as in \cite{eisenbach2015user}, operates inside buildings and the camera is mounted at eye level.
This results in completely different images, which differ in perspective, occlusions, lighting conditions, and contrast from the typical benchmark datasets.
Through our evaluation on the ROREAS+ robotics dataset, which follows the protocol of \cite{ye2021deep} as in all experiments in this paper, we should get a clear picture regarding how much attention helps to improve person re-identification in a real-world robotics application scenario.
Due to data protection laws, we are not allowed to show pictures, but only report benchmarking results.

\begin{table}[b!]
		\vspace*{-4mm}%
    \caption{Re-id performance on the robotic dataset RO\-RE\-AS+ using a ResNet-50 with and without attention.}
		\vspace*{-2mm}%
    \centering
    \setlength{\tabcolsep}{2.6pt}
    \begin{tabular}{ccccc}
        \hline
        & Dataset pre-taining & Dataset fine-tuning & Attention & mAP \\
        \hline
        (1) & Market-1501 \cite{zheng2015scalable} & --- &  --- & 0.6630 \\
        \rowcolor{lightgray!50!white!80} (2) & Market-1501 \cite{zheng2015scalable} & --- & C-NL & \textbf{0.6684} \\
        \hline
        (3) & Market-1501 \cite{zheng2015scalable} & ROREAS+ & --- & 0.7551 \\
        \rowcolor{lightgray!50!white!80} (4) & Market-1501 \cite{zheng2015scalable} & ROREAS+ & C-NL & \textbf{0.7658} \\
        \hline
    \end{tabular}
    \label{tab:roboticResults}
\end{table}

The ROREAS+ dataset contains 421 different persons.
In order to get a realistic setup, out of 421 recorded persons, we have chosen only 50 persons with the largest amount of recorded images as training data, the rest is taken as test data to get a representative test set with many different (but potentially similarly looking) persons included, where attention can make the difference.
Recording many views for 50 people with a mobile robot can already be a challenging and potentially time-consuming task in some scenarios, but we found this number to be a good trade-off between the amount of work needed to record the data and the amount of data available for a deep-learning approach.
Therefore, 34012 images of 50 identities are available for training, while the test set is split into 945 query images and 2520 gallery images of 371 identities.

Due to the limited amount of training data, we must use transfer learning to profit from pre-training on larger re-id benchmark datasets.
Especially the small amount of different identities can be an issue, otherwise.
For transfer learning, we first pre-train on Market-1501 \cite{zheng2015scalable} and then use a two-step fine-tuning approach \cite{chen2018deep}, where first the randomly initialized classification layer is fine-tuned on our target dataset ROREAS+, followed by the fine-tuning of all network weights.
For training, we follow the strong baseline setup \cite{luo2019bag}.
As loss function, we employ circle loss, as it showed to be beneficial in our previous experiment.
Following our best results, we added the best performing attention block C-NL at positions 6, 8, and 14.

In \autoref{tab:roboticResults}, we report the best results over three runs, since this would be the configuration that we would select for later application.
First, we could use a ResNet-50 for feature extraction that was trained on re-id benchmark data only, as in our previous experiments, without any fine-tuning on ROREAS+.
We can see, that using a ResNet-50 for feature extraction that was never trained on robotic data (rows 1, 2) performs worse, both with and without attention, compared to the corresponding results with fine-tuning on ROREAS+ (rows 3, 4).

In both cases -- with or without finetuning -- C-NL noticeably improves the ability to re-identify persons. The standard deviation over three runs is $\sigma = 0.006$. Therefore the results with attention are ca. $1.5 \sigma$ better than without attention.

\section{A Fast and Powerful Attention Block}
\label{sec:attentionBlocks}

After providing an overview of the state of the art in person re-identification (re-id) with architectures that incorporate attention blocks, we will now describe the attention blocks that we considered in our NAS.
Then, we derive a novel attention block that attempts to combine the advantages of these attention blocks in terms of inference speed and re-identification capabilities, as described in the following.

\subsection{Attention Blocks for Re-Identification}

\autoref{fig:attentionBlocks} shows the overall layout of channel-wise and non-local attention blocks that achieve the best re-id performance in benchmarks.
In \cite{li2018harmonious} (harmonious attention channel-wise, HAC), it has been shown that the performance (with channel-wise attention) improves most when attention blocks are inserted between residual blocks, as it has been shown first in \cite{wang2018non} for another type of attention applied to computer vision applications.
We visualize the modified position relative to the residuum in \autoref{fig:attentionBlocks}(b) (see \autoref{fig:attentionBlocks}(a) for the original position as proposed in the squeeze-and-excitation block \cite{hu2018squeeze}).
We also examined local attention multi-granularity network (LAG) \cite{gong2021lag}, but its attention block clearly underachieved in comparison to HAC.
Therefore, these results are not shown in the following experiments for reasons of clarity.

Also non-local attention blocks are placed between residual blocks (see \autoref{fig:attentionBlocks}(c)), which has been shown to be superior in \cite{wang2018non}.
We consider three non-local attention blocks, namely attention generalized mean pooling with weighted triplet loss (AGW) \cite{wang2018non} (applied for re-id in \cite{ye2021deep}), relation-aware global attention (RAGA) \cite{zhang2020relation}, and attentive but diverse (ABD) \cite{chen2019abd}.
All non-local attention blocks follow the same layout as shown in \autoref{fig:attentionBlocks}(c) with three branches, called query, key, and value in the transformer architecture \cite{vaswani2017attention}.
In all these three attention blocks, $f$ computes the matrix product of queries $q$ and keys $k$, with only the normalization of the result being different.
RAGA also comes with a further small difference, which translates into additional computational costs.

\begin{figure*}
    \begin{minipage}{0.81\linewidth}
        \includegraphics[width=\linewidth]{img/resnet50.pdf}
    \end{minipage}
    \begin{minipage}{0.175\linewidth}
        \textit{\tiny Input tensor size for the attention}\\[-2.5mm]
        \textit{\tiny block at each possible position}\\[-1.4mm]
        \textit{\tiny Residual blocks with an $1{\times}1$}\\[-2.5mm]
        \textit{\tiny convolution in the skip connection}\\[-1.2mm]
        \textit{\tiny Normal bottleneck}\\[-2.5mm]
        \textit{\tiny residual blocks}\\[-1.0mm]
        \textit{\tiny Global average pooling (GAP) and fully}\\[-2.5mm]
        \textit{\tiny connected (FC) classification layer}\\[-1.0mm]
        \textit{\tiny Positions where attention blocks can}\\[-2.5mm]
        \textit{\tiny be inserted marked as diamonds}\\[-3.8mm]
    \end{minipage}
    \caption{\label{fig:resnet}ResNet-50 with positions where attention blocks can be inserted marked as diamonds. Applying attention blocks directly on the input is somewhat questionable and has shown to not improve the performance in our experiments. All other 17 positions between the initial stem and the final GAP+FC should be considered. Residual blocks with an $1{\times}1$ convolution in the skip connection (gray) mark the beginning of each of the four stages of the ResNet-50.}
\end{figure*}

\subsection{Deriving a Novel Attention Block}

Both channel-wise and non-local attention blocks provide advantages for re-identification, which we attempt to combine in our novel attention block as described below.
In our experiments, we show that non-local attention blocks slow down inference speed too much to be worth considering for a mobile robotic application.
Therefore, our novel block should be designed like the HAC attention block \cite{li2018harmonious}.
Its low inference costs are a result of the early reduction of the spatial resolution by global average pooling.
The benefit of non-local attention blocks are, however, the three-branch layout to appropriately weight the features in the value branch to estimate global correlations.
Therefore, our block uses an early global average pooling to reduce computational costs and then follows the three-branch design of the non-local attention blocks to model the inter-channel correlation in a non-local way.
\autoref{fig:attentionBlocks}(d) shows the derived design of our channel-wise non-local attention block (C-NL).
The three-branch layout comes with minor computational costs in this setting as a result of the matrices of reduced size that need to be multiplied (see matrix dimensions shown in gray in \autoref{fig:attentionBlocks}(c, d)).
Therefore, this design leads to low computational costs and achieves a high re-id performance, as we show in the following experiments.

\section{How Much Attention Do We Need for Person Re-Identification?}

In what follows, we will first address the knowledge gap regarding the extent to which attention blocks slow down inference speed.
Then, we will analyze how many attention blocks we really need to significantly improve person re-identification (re-id), by performing extensive NAS over the design space of attention blocks, positions of integration, and hyperparameters.
We take inspiration by the Designing Network Design Spaces NAS approach \cite{radosavovic2020designing} that showed how the design space can gradually be reduced and, as a consequence, a set of design rules can be derived.
We primarily reduce the design space regarding positions of integration to derive a set of rules of where attention blocks should be added to get a large improvement with low computational cost.

\subsection{Experimental Setup}

As basis for our experiments, we use the strong baseline and training setup of \cite{luo2019bag} consisting of a modified ResNet-50 \cite{he2016deep}, which we also deploy in real-time on our mobile robots, in combination with common augmentation and training strategies.
For benchmarking, we decided in favor of one of the most popular datasets, namely Market-1501 \cite{zheng2015scalable}, and used the evaluation protocol of \cite{ye2021deep}.
We could also benchmark on DukeMTMC-reID \cite{ristani2016performance} or CUHK03-NP \cite{li2014deepreid}, but instead we decided in favor of a robotic dataset, for which we report results in Sec.~\ref{sec:robotic_dataset}, to show generalization ability.

Note, that better results can be achieved on Market-1501 with more elaborate baselines.
But these baselines come with architectural modifications, which result in higher computational costs.
This is contrary to our goal of fast inference on a robot.
Also by integrating several time frames, like in \cite{wang2019spatial}, or other costly techniques, better benchmark results could be achieved, but this is again contrary to our goal of deploying these networks in real-time on a mobile robot.

During NAS, all hyper-parameters are tuned, such that the reported results are trustworthy and not a fragment of the choice of hyper-parameters.

\subsection{Analysis of Inference Speed of Attention Blocks for Person Re-Identification}
\label{sec:inferenceSpeed}

First, we measure the inference speed of attention blocks that have been proposed or applied for re-id as described in \autoref{sec:attentionBlocks}\footnote{For implementation of the attention blocks, where available, we used publicly available code provided by the authors: AGW, evaluation code -- {\tiny\url{https://github.com/mangye16/ReID-Survey}}, RAGA -- {\tiny\url{https://github.com/microsoft/Relation-Aware-Global-Attention-Networks}}, ABD -- {\tiny\url{https://github.com/VITA-Group/ABD-Net}}, LAG -- {\tiny\url{https://github.com/SWJTU-3DVision/LAG-Net}}}.
We add the different attention blocks at single positions in the ResNet-50.
The diamonds in \autoref{fig:resnet} show the locations where attention blocks can be integrated.
We measure the inference speed on a Jetson AGX Xavier (Jetpack 4.6, TensorRT 8.0.1.6, Cuda 10.2, 16 bit floating point precision), which is a special device typically used for deep-learning computations on a mobile robot.
Since for each image we need to extract features for all persons in the camera, we chose a batch size of 16 for the measurement of inference speed.
With this batch size, we would be able to extract features for 16 persons in the field of view simultaneously, which we consider as a good worst-case estimate for crowded scenes, like in hospital hallways.
We report the number of batches that can be processed per second during inference.
The reported results are the average over 10000 processed batches of the Market-1501 benchmark dataset \cite{zheng2015scalable} with cropped and scaled RGB person images of size $256{\times}128$.
However, the inference speed does not depend on the dataset, but only on the input image size, which is identical in all our experiments in this paper.

\begin{figure}[t!]
    \centering
    \begin{minipage}{0.83\linewidth}
        \includegraphics[width=\linewidth]{img/speed_plot.pdf}
    \end{minipage}
    \begin{minipage}{0.155\linewidth}
        \includegraphics[width=\linewidth]{img/RINGO-robot-with-Jetson.png}\\[-1mm]
        \textit{\tiny Jetson AGX}\\[-2.5mm]\textit{\tiny Xavier for deep}\\[-2.5mm]\textit{\tiny  learning com-}\\[-2.5mm]\textit{\tiny putations on}\\[-2.5mm]\textit{\tiny our rehab com-}\\[-2.5mm]\textit{\tiny panion robot}\\[3mm]
    \end{minipage}
    \caption{\label{fig:speedPosition}Inference speed on a Jetson AGX Xavier for integrating single attention blocks at different positions measured by number of batches processed per second. The input tensor size for each attention block at the respective positions is listed at the top of the figure.}
\end{figure}

\begin{figure*}
    \centering
    \includegraphics[width=\linewidth]{img/bar_plot_with_error_bars.pdf}
    \caption{\label{fig:mapPosition}Re-id performance measured by the mean average precision (mAP) on the Market-1501 benchmark dataset \cite{zheng2015scalable} for different attention blocks integrated at a single position in the ResNet-50 (as displayed in \autoref{fig:resnet}).
    Downwards pointing triangles symbolize that the bar is below the displayed range of values.
    Results for positions where inference with attention blocks is slower than ResNet-101 but does not match its performance, or where its performance is even worse than ResNet-50, are shown in light colors to symbolize that these configurations should not be considered.}
\end{figure*}

\autoref{fig:speedPosition} shows the inference speed for the attention blocks as a function of the single position where the individual attention block is integrated.
Note that the inference speed depends only on the size of the input tensor for the attention block (listed at the top of the figure) and is thus identical for several positions.
As reference, we show the inference speed of the ResNet-50 without any attention block and of the ResNet-101 (without attention) that is twice as deep.
We can see that channel-wise attention blocks shown in blue and green (HAC \cite{li2018harmonious}, C-NL) are faster to compute than non-local attention blocks shown in orange to red (AGW \cite{ye2021deep}, RAGA \cite{zhang2020relation}, ABD \cite{chen2019abd}).
A ResNet-50 with a single non-local attention block at an early position (1--4) is significantly slower than a ResNet-101.
Thus, these configurations should not be considered for robotics applications.
Furthermore, at late positions (9--17) only AGW \cite{ye2021deep} can cope with the inference speed of channel-wise attention blocks to some degree.
Also at middle positions (5--8) channel-wise attention is faster to compute than non-local attention.
Comparing the channel-wise attention blocks, for most positions (1--14) the proposed C-NL block achieves a higher inference speed than HAC \cite{li2018harmonious}.
To conclude, considering only inference speed so far, channel-wise attention is preferable over non-local attention in robotic applications.

\subsection{How Many Attention Blocks Do We Really Need?}

Next, we analyze, how many attention blocks we need to significantly improve the re-id performance.
Therefore, inspired by \cite{radosavovic2020designing}, we performed NAS by progressively reducing the design space over attention blocks, positions of integration, and hyperparameters regarding positions of integration while considering computational costs as measured above.
\autoref{fig:mapPosition} shows the re-id performance when attention blocks are inserted at a single position measured by the mean average precision (mAP) on Market-1501 \cite{zheng2015scalable}.
Furthermore, the plot shows mean and standard deviation over three training runs.

As we can see, even a single attention block integrated at a suitable position can top the performance of the ResNet-101 that is twice as deep as the baseline.
Furthermore, we can see that channel-wise and non-local attention perform very similar in this regime of few attention blocks.
By considering the lower inference costs, we clearly should prefer channel-wise attention over non-local attention.

\subsection{Where Should We Add Attention Blocks?}

Now that we have seen that adding a single attention block is a powerful approach, we like to investigate whether combinations of computationally effective channel-wise attention blocks at different positions can further improve the performance.
From the extensive NAS with plenty of combinations of positions and types of attention, as part of the progressive reduction of the design space, we were able to derive the following set of rules to improve the re-identification performance most by using as few as possible attention blocks:

\vspace*{3pt}%
\begin{enumerate}
    \itemsep3pt
    \item Select positions where the single-position performance is good (see \autoref{fig:mapPosition}). Most often, positions at the end of each stage seem to perform slightly better than earlier positions in each stage.
    \item The distance between selected positions should be large. Positions in different ResNet stages should be preferred.
    \item A combination of different types of attention (channel-wise, non-local) performs slightly better than single-type combinations (but is not worth it for robotic applications due to the computational cost of non-local attention blocks).
\end{enumerate}

\begin{table}[t!]
    \caption{Re-id performance on the Market-1501 benchmark dataset~\cite{zheng2015scalable} and inference speed for attention blocks at single and multiple positions using cross entropy and circle loss~\cite{sun2020circle} respectively.}
		\vspace*{-2mm}%
    \centering
    \setlength{\tabcolsep}{1.0pt}
    \begin{tabular}{cccc}
        \hline
        & & Re-id performance & Inference speed \\
        Attention & Loss function & [mAP] & [batches/second] \\
        \hline
        HAC \cite{li2018harmonious} @ 8 & cross entropy & 0.8733 $\pm$ 0.0006 & 25.17 \\
        \rowcolor{lightgray!50!white!80} AGW \cite{ye2021deep} @ 7 & cross entropy & 0.8733 $\pm$ 0.0012 & 22.59 \\
        C-NL @ 8 & cross entropy & 0.8755 $\pm$ 0.0007 & 26.70 \\
        \hline
        \rowcolor{lightgray!50!white!80} HAC \cite{li2018harmonious} @ 8, 14 & cross entropy & 0.8773 $\pm$ 0.0006 & 25.06 \\
        AGW \cite{ye2021deep} @ 8, 14 & cross entropy & 0.8805 $\pm$ 0.0014 & 21.09 \\
        \rowcolor{lightgray!50!white!80} C-NL @ 8, 14 & cross entropy & 0.8788 $\pm$ 0.0007 & 25.91 \\
        \hline
        HAC \cite{li2018harmonious} @ 6, 8, 14 & cross entropy & 0.8780 $\pm$ 0.0010 & 24.93 \\
        \rowcolor{lightgray!50!white!80} AGW \cite{ye2021deep} @ 6, 8, 14 & cross entropy & 0.8815 $\pm$ 0.0008 & 19.88 \\
        C-NL @ 6, 8, 14 & cross entropy & 0.8806 $\pm$ 0.0007 & 25.89 \\
        \hline
        \rowcolor{lightgray!50!white!80} HAC \cite{li2018harmonious} @ 6, 8, 14 & circle loss \cite{sun2020circle} & 0.8897 $\pm$ 0.0012 & 24.93 \\
        AGW \cite{ye2021deep} @ 6, 8, 14 & circle loss \cite{sun2020circle} & 0.8916 $\pm$ 0.0006 & 19.88 \\
        \rowcolor{lightgray!50!white!80} C-NL @ 6, 8, 14 & circle loss \cite{sun2020circle} & \textbf{0.8916} $\pm$ 0.0005 & \textbf{25.89} \\
        \hline
    \end{tabular}
    \label{tab:attentionCombinations}
		\vspace*{-4mm}%
\end{table}

\vspace*{3pt}%
The best results of the NAS for each type of attention block was achieved for a pair of attention blocks at positions 8 and 14, and for the triplet of attention blocks at positions 6, 8, and 14.
Using the same type of attention block at more than three positions did not improve the results further.
\autoref{tab:attentionCombinations} lists the best results for attention blocks at multiple positions.
Furthermore, \autoref{fig:mapVsSpeed} shows the re-id performance versus inference speed in a graph.
We can see that for the channel-wise attention blocks HAC and C-NL, the inference speed is affected only slightly.
Inference with the proposed C-NL at positions 6, 8, and 14 is only 1.706$\,$ms slower (ca. 5\%) than the ResNet-50 baseline without any attention blocks (38.623$\,$ms vs. 36.917$\,$ms).
Therefore, while just 1.196 fewer batches can be processed per second, the re-identification performance far exceeds that of ResNet-101 (mAP 0.8806 $\pm$ 0.0007 vs. 0.8707 $\pm$ 0.0006), which is twice as deep and takes 53.454$\,$ms to process a batch.

\subsection{Can the Performance Be Improved Further?}
\label{sec:lossFunction}

In \cite{aganian2021revisiting}, an alternative way of improving the re-identification performance was examined.
By using a large-margin loss function, the recognition rate is increased without affecting the inference speed.
We also apply the best performing loss function, namely circle loss \cite{sun2020circle}.
In \cite{aganian2021revisiting}, circle loss achieved a mAP of 0.882 on the Market-1501 dataset using the strong baseline \cite{luo2019bag} setup as in our experiments.
We observed, that the combination of a large-margin loss function and few attention blocks at suitable positions complement each other very well and boosts the performance further (see \autoref{tab:attentionCombinations}).
The proposed C-NL attention block matches the performance of the much more costly AGW non-local attention block, while being faster during inference (highlighted in bold).

\section{Introduction}
In recent years, mobile robots that follow a user have become increasingly important, especially in the field of clinical rehabilitation.
One example is a robot coach for walking and orientation training of stroke patients~\cite{gross2017roreas}, where the robot had to accompany the patients during their walking exercises that they had to perform on their own to improve their mobility as well as their orientation skills.
This way, the robot addressed the patients' insecurity and anxiety of not being able to perform the exercises or not being able to find the way back to their apartment, which are possible reasons for neglecting self-training.
Another example is a robotic assistant that coached patients during their walking exercises, which were taught to them by physiotherapists after hip endoprosthetics surgery~\cite{scheidig2021robot,vorndran2018always}.
The robot was intended as a kind of physiotherapist replacement to provide immediate feedback to the patient regarding any deviations from the expected physiological gait pattern.

\begin{figure}[t!]
    \centering
    \includegraphics[width=\linewidth]{img/map_vs_speed.pdf}
    \caption{\label{fig:mapVsSpeed}Re-identification performance on the Market-1501 benchmark dataset \cite{zheng2015scalable} versus inference speed for different attention blocks and insertion positions in the ResNet-50 (see \autoref{fig:resnet}).
    You only need to add a single block of the proposed C-NL attention to ResNet-50 to outperform ResNet-101, and this only slightly affects the inference speed.}
\end{figure}

In both applications, it is mandatory for the robot to permanently be available to the users and to closely follow or guide them when desired.
To achieve this, the mobile robot has to track its users for a long time, even if they are occasionally out of sight for some time.
Therefore, person re-identification (re-id) plays a key role in such scenarios \cite{bellotto2007multisensor,alvarez2012feature,satake2013visual,eisenbach2015user,koide2016identification,an2017online,cosar2017volume,islam2019person,koide2019convolutional,algabri2020deep,algabri2022online,eirale2022human} either to provide robust features that can be used internally in a tracker for track continuation in case of temporal total occlusions \cite{muller2020multi} or to continue a track by re-identifying a person after a track is lost in a separate module \cite{wengefeld2016may}.

In recent years, ConvNet-based features have proven to be very reliable for non-biometric re-identification after the user has been unobserved for some time.
Therefore, these features can be used as visual cues in a probabilistic multi-person tracking framework \cite{muller2020multi}.
A person detector extracts cropped RGB images for all persons in the observed scene.
Next, single-shot re-identification features are extracted for these individuals.
The tracker then handles the temporal integration and fusion with other tracking inputs.

Fortunately, real-time deep-learning-based feature extraction on a mobile robot is no longer a problem due to the availability of specialized GPU devices such as the NVIDIA Jetson series.
However, inference speed still needs to be considered as these devices are typically shared for several neural-network-based classification or regression tasks.
This is in contrast to the current trend in person re-identification, where performance is improved at all costs, neglecting the significant slowdown during inference.
An example of this is the heavy use of attention blocks, especially non-local attention, which significantly slows down inference speed, as we will show in our experiments (see \autoref{sec:inferenceSpeed}).
We will demonstrate that we can improve the re-id performance with attention blocks without any significant slowdown in inference speed, if we find the right kind of attention blocks and a minimum set of appropriate positions in a ResNet-50 architecture.
Therefore, our contributions are as follows:

\vspace*{3pt}%
\begin{enumerate}
    \itemsep3pt
    \item We investigate the tradeoff between inference speed and re-id performance for different attention blocks, which is neglected in the current state of the art.
    \item We propose the new attention block C-NL that is faster than current attention blocks and performs better in a regime with few attention blocks.
    \item We perform extensive neural architecture search (NAS) to derive a set of rules for where and what type of attention should be integrated into a ResNet-50 to significantly improve re-id performance while only slightly affecting inference speed.
    \item We confirm that by integrating C-NL attention blocks into the ResNet-50 as specified in the derived set of rules, the re-id performance is also improved on a robotic dataset.
\end{enumerate}

\section{Related Work}

Nowadays, attention is widely used in computer vision.
To improve ConvNet-based single-shot person re-identification (re-id), mainly, three types of attention blocks are available:
channel-wise attention~\cite{hu2018squeeze}, %
spatial attention~\cite{wang2017residual}, %
and non-local attention~\cite{wang2018non}, %
with self-attention of the transformer architecture~\cite{vaswani2017attention} %
being a special form of the latter.

\textit{Channel-wise attention}
is based on the principles of the squeeze-and-excitation block \cite{hu2018squeeze} (\autoref{fig:attentionBlocks}(a)).
First, the spatial resolution is reduced to $1{\times}1$ by global average pooling.
Then, the inter-channel correlation is modeled using a bottleneck in which the number of channels is reduced by $r$.
Finally, the output of the attention block, which is typically used as a per-channel weight, is normalized by a sigmoid activation.
Channel-wise attention based on these principles is widely applied for re-id \cite{li2018harmonious, yang2019attention, wu2021attention, chen2019self, ji2020attention, gong2021lag}.
Fully attention~\cite{wang2018mancs} is a variant of channel-wise attention for re-id that deliberately omits the global average pooling.
In \cite{chen2019mixed} fully attention was extended to higher-order statistics in the context of re-id.
We will show that channel-wise attention is cheap to compute, and should therefore be considered for a mobile robotic application where inference speed plays a key role.

\textit{Spatial attention}
is based on the network proposed in \cite{wang2017residual}.
First, the input volume is reduced to one channel while keeping the spatial resolution.
Then, the spatial resolution is reduced and subsequently restored to model the spatial correlation.
Finally, a sigmoid activation normalizes the output of the attention block, which is typically used as spatial weighting.
Spatial attention is also widely applied for re-id \cite{li2018harmonious, yang2019attention, wu2021attention, chen2019self, ji2020attention, zhou2019discriminative, li2020scanet}.
A variant that deviates from the design principles described above is the use of a foreground mask as spatial attention for re-id \cite{song2018mask, zhou2019foreground}.
This requires an additional network for extracting a foreground mask, which comes at an additional inference cost.
Spatial attention often does not perform that well on re-id benchmarks unless it is used in combination with other attention blocks.
Therefore, many re-id approaches combine spatial attention with channel-wise attention.
Channel-wise and spatial attention are computed in two branches that are either multiplied element-wise \cite{li2018harmonious, yang2019attention, wu2021attention} or concatenated at the channel dimension \cite{chen2019self, ji2020attention}.

\textit{Non-local attention}
aims at modelling global inter-pixel correlation.
Most often, it follows the design principles of the attention-module in the transformer architecture~\cite{vaswani2017attention} (\autoref{fig:attentionBlocks}(c)).
In a query and a key branch similarities of features are computed at different spatial positions which are then used to weight the features calculated in the value branch.
For re-id, non-local attention has been applied in this pure form \cite{ye2021deep, zhang2020relation}, with second-order statistics \cite{xia2019second}, and as self-attention \cite{chen2019abd}.
Non-local co-attention is also used to match person images of the probe and gallery \cite{zhang2019learning}.
The latter approach is therefore not relevant for our application of re-id features as tracking input, since we aim for extracting features for single person images.
Non-local attention achieves the best results in re-id benchmarks, but comes with high inference costs, as we will \mbox{show in our experiments.}

Besides these three main forms of attention blocks that are applicable for single-shot re-id on RGB images, there are also \textit{other forms of attention} applied for re-id.
In \cite{lan2017deep, chen2019self, zhang2020person} reinforcement learning is used to adapt attention weights.
In contrast, our focus is on learning the attention weights by error backpropagation.
Selecting proper semantic and soft-biometric attributes for re-id by attention has been addressed in
\cite{zhang2020person, xu2018attention, tay2019aanet}.
To be able to utilize attributes would require an additional network for attribute extraction.
This is in contrast to our goal of affecting the inference speed only little.
Qian et al.~\cite{qian2019leader} use a multi-branch multi-scale architecture for person re-identification and train an attention block to select the best scale for an input.
Multi-scale approaches require significantly more operations, and therefore slow down inference, which is contrary to our goal.
Attention has also been applied to video-based re-id on streams of images~\cite{li2018diversity,si2018dual,karianakis2018reinforced,subramaniam2019co,fu2019sta,chen2019spatial,zhang2020multi,shim2020read,li2020relation,li2020temporal}.
These approaches primarily focus on temporal attention.
In addition, attention has been applied to re-id with other input modalities, such as infrared~\cite{ye2020dynamic} or depth images~\cite{haque2016recurrent}.
Both temporal attention and customized attention for other modalities are not the focus of the work presented in this paper.

\begin{figure*}
    \centering
    ~~\includegraphics[height=7.5cm]{img/attention_se.pdf}~~
    ~~\includegraphics[height=7.5cm]{img/attention_hac.pdf}~~
    ~~\includegraphics[height=7.5cm]{img/attention_non_local.pdf}~~
    ~~\includegraphics[height=7.5cm]{img/attention_cnl.pdf}~~\\[-2mm]
    {\footnotesize(a) SE \cite{hu2018squeeze}\hspace{1.8cm}
    (b) HAC \cite{li2018harmonious}\hspace{2.5cm}
    (c) NL \cite{wang2018non}\hspace{4.1cm}
    (d) C-NL (proposed)\hspace{1cm}~}\\
    \caption{\label{fig:attentionBlocks}Layout of attention blocks: Channel-wise attention is realized by the (a) squeeze-and-excitation block \cite{hu2018squeeze} (SE) and the (b) harmonious attention channel-wise block \cite{li2018harmonious} (HAC). The prototype design of non-local attention \cite{wang2018non} (NL) is shown in (c). Our proposed block (C-NL) that is derived from these channel-wise and non-local attention blocks is shown in (d).}
\end{figure*}

\subsubsection*{Effect of integrating attention blocks on inference speed}
The drawback of adding attention blocks, namely slowing down inference, is completely neglected in the current state of the art.
However, fast inference is a prerequisite for real-time mobile robot applications.
Therefore, in this paper we address this issue.
We show that it is possible to integrate attention blocks at only few positions in a ResNet-50 to achieve similar performance to much deeper networks or computationally demanding attention blocks, while only marginally affecting the inference speed, if the correct positions can be identified.
Therefore, we perform extensive NAS to derive a set of rules to guide the integration.

\subsubsection*{Backbone}
We decided in favor of ResNet-50 as backbone for our experiments, since it is the de-facto standard for evaluating on person re-identification benchmarks.
Therefore, our results are easily comparable to the state of the art.
However, since we explore design spaces during NAS, the gained knowledge about integration of attention blocks is also transferable to other similar architectures \cite{radosavovic2020designing}.


\section{Introduction}\label{sec:intro}

Up to June 2022, the Hard X-ray Modulate Telescope (dubbed as \textit{Insight}-HXMT), the first space X-ray telescope of China, has been operated successfully in orbit for five years since the launch on June 15, 2017 \citep{ZSN2020}. There are three main payloads onboard \textit{Insight}-HXMT, namely, Low Energy Telescope (LE; \citealt{ChenY2020}), Medium Energy Telescope (ME; \citealt{CaoXL2020}) and High Energy Telescope (HE; \citealt{LiuCZ2020}). All three payloads working in the collimated way are combined to cover a broad energy range from 1 keV to 250 keV. The layout of the three telescopes on the main structure is shown in Fig.~\ref{fig:HXMT}, and the main parameters are shown in Table~\ref{tab:para}.

There are three main observation modes assigned to \textit{Insight}-HXMT: pointing observation, Galactic plane scanning survey and gamma-ray burst observation.
Although the data analysis methods and processes are different, the background estimation is one of the crucial steps for fulfilling each task. For gamma-ray burst observation, the background is usually taken from pre- and post-burst \citep{Luo2020}. For scanning observation, the background is derived from the scanning light curves with sources subtracted \citep{Sai2020}. However, the background estimation of the pointing observation is much more complicated, since the traditional on-off mode of \textit{BeppoSAX}/PDS \citep{Frontera1997a,Frontera1997b} and \textit{RXTE}/HEXTE \citep{Rothschild1998} for observation of the background was not adopted in the design of \textit{Insight}-HXMT.
Therefore, in order to estimate the background accurately, a dedicated background model has been constructed during the operation of \textit{Insight}-HXMT \citep{Liao2020_le,Liao2020_he,Guo2020}.

Before the launch of \textit{Insight}-HXMT, thorough background simulation works have been done (e.g., \citealt{LiG2009,XieF2015}). With the Geant4 tools and the mass model of \textit{Insight}-HXMT, these works result in the expected in-orbit background of \textit{Insight}-HXMT, such as the background spectra induced by various particles. The first two-year background observation of \textit{Insight}-HXMT proved the consistency between the in-orbit observation and the on-ground simulation, which also suggests that our understanding of the in-orbit background of \textit{Insight}-HXMT is reliable \citep{ZhangJ2020}.

\textit{Insight}-HXMT is a low Earth orbit satellite. The orbit is approximately a circle with an attitude 550~km and an inclination 43$^\circ$. As shown in previous works (e.g., \citealt{LiG2009,XieF2015,Liao2020_le,Liao2020_he,Guo2020,ZhangJ2020}), the orbital space environment of \textit{Insight}-HXMT is complex and various particles can interact with the satellite platform and instruments to generate many background components (e.g., \citealt{Alcaraz2000a,Alcaraz2000b}). The cosmic-ray protons contribute most to the background of \textit{Insight}-HXMT, while the electrons, neutrons, cosmic X-ray background (CXB), as well as earth albedo of gamma-rays can also contribute to the background.
\textit{Insight}-HXMT can explore the environment of charged particles with its Particle Monitor (PM; \citealt{LuXF2020}), which is on board in the top panel of the satellite platform and sensitive to protons ($>20$~MeV) and electrons ($>1.5$~MeV). As shown in Fig.~\ref{fig:PM_map}, the geographical distribution of the PM count rate has varied little over the past five years for the whole orbit of \textit{Insight}-HXMT, including the low-latitude and high-latitude region, as well as the South Atlantic Anomaly (SAA). For the region with low count rate near the magnetic equator, the average count rate is stable at 1.7--1.8~$\rm cts~s^{-1}$. Therefore, the space environment has changed very little during the last five years. However, we have observed some long-term variations in backgrounds, which are largely related to some degradation effects for LE and ME detectors and  the activation effects of HE by bombardment of charged particles.
Therefore, the background of \textit{Insight}-HXMT after five years of operation needs to be reviewed, including the evolution of the observed characteristics and the validity of the background model.
In this paper, the blank sky observations in the high Galactic latitude region (Table~\ref{tab:obsid}) during the five years since the launch of \textit{Insight}-HXMT are used to investigate the in-orbit background of the three main payloads of \textit{Insight}-HXMT, including the observational characteristics and the systematic error analysis of the background model.

It is worth noting that all the three telescopes of \textit{Insight}-HXMT have various Field of Views (FoVs) with different sizes and orientations (Fig~\ref{fig:FOV} and Table~\ref{tab:para}). For the pointing observations of LE and ME, the small FoV detectors are encouraged to be used for scientific analysis. Accordingly, the observed characteristics and background models of LE and ME in this paper mainly focus on these detectors.
This paper is organized as follows. The backgrounds of LE, ME and HE are described in Sections 2--4, respectively. In Section 5, a summary and conclusion are given.

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f1.png}
		\caption{Main structure of \textit{Insight}-HXMT.}
		\label{fig:HXMT}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.9\textwidth]{f2a.pdf}
		\includegraphics[width=0.9\textwidth]{f2b.pdf}
		\caption{Geographical distributions of the PM count rate in the first {\bf (top)} and fifth {\bf (bottom)} years.}
		\label{fig:PM_map}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.3\textwidth]{f3a.pdf}
		\hspace{.1in}
		\includegraphics[width=0.3\textwidth]{f3b.pdf}
		\hspace{.1in}
        \includegraphics[width=0.3\textwidth]{f3c.pdf}
		\caption{FoVs of LE, ME and HE.}
		\label{fig:FOV}
\end{figure}

\begin{table}[ht]
\begin{center}
\begin{minipage}{260pt}
\caption{Main instrumental parameters of LE, ME and HE.}\label{tab:para}%
\begin{tabular}{@{}llll@{}}
\toprule
   & LE  & ME & HE\\
\midrule
Detector Type       &  SCD      & Si-PIN  &  Phoswich  \\
Energy range (keV)\footnotemark[1]  &  0.7--13  & 5--40   & 20--250  \\
Geometrical area (cm$^2$)  &  384  &  952  &  5096  \\
Small FoV (FWHM) & $1.6^\circ\times6^\circ$ & $1^\circ\times4^\circ$ & $1.1^\circ\times5.7^\circ$  \\
Large FoV (FWHM) & $4^\circ\times6^\circ$   & $4^\circ\times4^\circ$ & $5.7^\circ\times5.7^\circ$  \\
\botrule
\end{tabular}
\footnotetext[1]{Design value that will be adjusted over time.}
\end{minipage}
\end{center}
\end{table}

\begin{table}[ht]
\begin{center}
\begin{minipage}{280pt}
\caption{Observation of \textit{Insight}-HXMT background in five years}\label{tab:obsid}%
\begin{tabular}{@{}lll@{}}
\toprule
  ObsID  &  Duration  &  Target\footnotemark[1]  \\
\midrule
P0101293 (001-191)  &  2017-11-02 to 2019-06-26  &  Blank Sky  \\
P0202041 (001-161)  &  2019-07-10 to 2020-07-22  &  Blank Sky  \\
P0301293 (001-115)  &  2020-08-06 to 2021-08-30  &  Blank Sky  \\
P0401293 (001-115)  &  2021-09-14 to 2022-08-29  &  Blank Sky  \\
\midrule
P0101297 (201-217)  &  2017-09-13 to 2018-09-14  &  PSR B0540-69  \\
P0101322 (001-001)  &  2017-07-19 to 2017-07-23  &  PSR B0540-69  \\
P0114550 (001-003)  &  2017-09-20 to 2017-09-27  &  GW~170817  \\
P0101326 (001-018)  &  2017-07-08 to 2019-02-19  &  Cas~A  \\
P0202041 (200-208)  &  2019-07-13 to 2020-07-29  &  Cas~A  \\
P0302291 (001-020)  &  2020-08-23 to 2021-08-21  &  Cas~A  \\
P0101326 (001-015)  &  2021-09-17 to 2022-08-19  &  Cas~A  \\
\botrule
\end{tabular}
\footnotetext[1]{The blank sky observations can be used as the background observations of LE, ME and HE. The observations of PSR~B0540-69, GW~170817 and Cas~A are used only for HE background analysis as the sources are too weak to be detected by HE, and thus can be regarded as `Blank sky'.}
\end{minipage}
\end{center}
\end{table}

\section{Background of the Low-Energy Telescope}
\label{sec:le_bkg}

LE is made of Swept Charge Device (SCD) detectors with a geometrical area of 384~${\rm cm^2}$ and a band pass of 1--13~keV.
LE has three boxes with the FoV orientation differing by 60$^\circ$. Each box has 20 small FoV detectors (some are broken one after another), 6 large FoV detectors, 2 detectors with their collimators blocked by aluminum covers.
The blocked small FoV detector is designed to measure the particle background, and the blocked large FoV detector is planted with a $^{55}$Fe radioisotope to monitor the energy response.
During the five years operation of \textit{Insight}-HXMT, some of the LE detectors were broken and shut down. The details of the LE bad detectors can be obtained from the `Bad Detector FITS file' that is included in the \textit{Insight}-HXMT Data Analysis software.

Fig.~\ref{fig:le_lc} presents the LE light curve of a blank sky, which shows the typical outline of the LE background embedded with a series of special features.
The whole time range can be divided into the abnormal and normal stages of the instrument.
For the abnormal stage, LE is usually troubled by a large amount of low-energy charged particles and visible light due to the relatively large FoV, which enter from the collimator and are difficult to accurately estimate. In severe cases, LE detectors will be saturated with in-orbit storage overflows.
The instrument's normal stage can be divided into three types. First is the time interval of earth occlusion, where the light curves of the detectors with different FoVs coincide with each other and no CXB photons are recorded. Second is the flare time interval and the flares can be detected in both the small and large FoV detectors. Moreover, the flare flux is basically proportional to the FoV size. Finally, the time interval with neither earth occlusion nor flare is considered as the good time interval (GTI). The usual scientific analysis only uses the GTI data. In order to estimate the background accurately, the background analysis procedure deals with both the regular GTI judgement, and its count rate comparison between the small and large FoV detectors \citep{Liao2020_le}.
The observational characteristics of the LE background and the effectiveness of the background model in the past five years are shown in what follows.

\subsection{Observational characteristic and long-term evolution of the LE background}
\label{sec:le_obs_char}
Fig.~\ref{fig:le_map} shows the comparison of the geometrical distributions of the LE background before and after 2021-06-30. It can be seen that the distribution with longitude ($lon$) and latitude ($lat$) has not changed, but the intensity has increased significantly.
The spectra of the same geographical region ($55^\circ<lon<210^\circ$, $-15^\circ<lat<15^\circ$) are specifically investigated for every year. Fig.~\ref{fig:le_spec_small} shows the 5-year background spectra of the small FoV detectors. It can be seen that the spectra have little change at the low energy end, while showing a gradually increasing trend at the high energy end. The continuous broadening of the emission lines is the result of the decline of the LE energy resolution. As demonstrated by the on-ground simulations \citep{ZhangJ2020} and previous in-orbit observations, the LE background can be simplified into a diffuse X-ray background dominant in low energy band and a particle background dominant in high energy band. Thus, the difference between the two geographical distributions shown in Fig.~\ref{fig:le_map} is mainly due to the change in the high energy band. Fig.~\ref{fig:le_spec_blind} shows the 5-year background spectra of the blocked FoV detectors, and the results are consistent with these of the small FoV detectors.
As described in \cite{ZhangJ2020}, the background of \textit{Insight}-HXMT can be produced by various incidence. The background that can be recorded immediately after the incidence is called the prompt background, and the background recorded for a long time (hour to month) after the incidence is called delayed background. It is worth noting that both the backgrounds caused by the CXB and cosmic-ray protons are prompt background.

The behaviour of the LE background light curve has changed very little over the five years. The most obvious features are that the count rate is stable in the low energy band and is modulated significantly by the geomagnetic field in the high energy band, which is also shown in Fig.~\ref{fig:le_lc}.

\begin{figure}[ht]%
\centering
\includegraphics[width=0.6\textwidth]{f4.pdf}
\caption{Light curve of the LE small (green), big (blue) and blocked (red) FoV detectors (ObsID: P030129303601).}\label{fig:le_lc}
\end{figure}

\begin{figure}[ht]%
\centering
\includegraphics[width=0.6\textwidth]{f5.pdf}
\caption{Geographical distributions of the background of LE small FoV detectors before (top) and after (bottom) 2020-06-30.}\label{fig:le_map}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f6.pdf}
		\caption{Spectra of the LE small FoV detectors for every year since \textit{Insight}-HXMT operate in orbit.}
		\label{fig:le_spec_small}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f7.pdf}
		\caption{Same as Fig.~\ref{fig:le_spec_small} but for the LE blocked FoV detectors.}\label{fig:le_spec_blind}
\end{figure}

\subsection{Validity of the LE background model}
\label{le_bkg_model}

\cite{Liao2020_le} has found that the background spectral shape of LE blocked FoV detectors does not change with geographical location, and can be used to characterize the particle background spectral shape of small FoV detectors. The LE background model just takes advantage of this feature to give a simple and reliable background estimation.

There is an evolution of the LE background over the five years, although it is not very significant (Fig.~\ref{fig:le_map}).
It can be seen from Fig.~\ref{fig:le_spec_small}--\ref{fig:le_spec_blind} that the small and blocked FoV detectors have a similar evolution trend, i.e., the lower limit of the spectral energy range becomes higher and the count rate becomes larger as the in-orbit time increases.
For the LE detector, a large signal can be recorded as several split-events in several pixels at the same time. However, only the events above a certain threshold will be recorded and involved in the subsequent split-event reconstruction. For example, a large signal with energy E can be recorded as two signals with E0 and E1 ($\rm E0+E1=E$). If E1 is less than the threshold, this large signal will be treated as a single-event with E0. With increasing irradiation damage of the LE detector, the distribution of noise signal becomes wider. In order to eliminate the influence of noise signal on the working energy band, the threshold is also adjusted higher. This will raise the lower energy limit of the LE detector, as shown in the low energy band of the spectra in Fig.~\ref{fig:le_spec_small}--\ref{fig:le_spec_blind}. Moreover, the small signals that can be recorded and participate in the split-event reconstruction before threshold adjustment will not exceed the threshold after that, i.e., the double-events that can be reconstructed before will not be reconstructed after threshold adjustment. As the threshold becomes higher, a larger proportion of the double-events will not be reconstructed but will be treated as single-events with a lower energy. As shown in Fig.~\ref{fig:le_spec_small}--\ref{fig:le_spec_blind} for the evolution of the background spectra, the background spectrum shifts to the left year by year. Therefore, the increasing trend of LE background is the result of LE detector irradiation damage and split-event reconstruction in data processing.
Moreover, the spectra of LE blocked FoV detectors in high energy band can be mixed with the super-threshold signal peaks, so the energy band of the blocked FoV detectors in background model has been adjusted accordingly.

The validity of the background model is investigated, as it is critical for scientific analysis. With the same method in \cite{Liao2020_le}, we perform the background estimation for every blank sky observation. Fig.~\ref{fig:le_bkg_est} shows the background spectrum estimation (ObsID: P030129310101) as an example.
For each year, the parameters of background model are updated to maintain the accuracy of the background estimate and then the systematic error of the background model is investigated. Fig.~\ref{fig:le_bkg_est_eb} shows the deviations of the LE background estimation in eight energy bands in the fourth year as an example.
The systematic errors of different energy bands between 2 to 10~keV in each year are shown in Fig~\ref{fig:le_err_sys}, and the results show that the systematic error does not change significantly compared to the first two years since the launch of \textit{Insight}-HXMT, i.e., the background model is stable and can give accurate background estimate. However, as the data around 1.5~keV is often affected by the electronic noise, the detection threshold is adjusted upward, thus only the systematic errors above 2~keV are given in this paper.

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f8.pdf}
		\caption{An example of the LE background estimation (ObsID: P030129310101). Top: spectrum of a blank sky observation (black) and the estimated background spectrum (red). Bottom: residuals in terms of errors ($\sigma$).}
		\label{fig:le_bkg_est}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.9\textwidth]{f9.pdf}
		\caption{Deviations of the LE background estimation for eight energy bands in 4-th year.}\label{fig:le_bkg_est_eb}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f10.pdf}
		\caption{Systematic errors of the LE background model for every year after the launch of \textit{Insight}-HXMT.}\label{fig:le_err_sys}
\end{figure}

\section{Background of the Medium-Energy Telescope}\label{me_bkg}
As shown in Table~\ref{tab:para}, ME is a collimating telescope sensitive in 5–40~keV with a total geometrical area of 952~$\rm cm^2$. It consists of 54 sensors in three boxes, and each sensor handles 32 SI-PIN pixels. For each box, there are 15 sensors with small FoV collimators and one sensor with the blocked FoV collimator that is used for background estimates.
The ME background characteristics have some similarities to those of LE in high energy band, especially the light curve feature and the geographical distribution.
However, the portion of each background component is very different, and the particle background is dominant in the whole energy band \citep{Guo2020,ZhangJ2020}.
Fig.~\ref{fig:me_map} shows a comparison of the ME background geographical distribution in the first and fifth years, respectively. It can be seen that the ME background in the fifth year is slightly higher than that in the first year.
In the region near SAA ($330^\circ<lon<360^\circ$, $0^\circ<lat<30^\circ$), the background is significantly higher than the most regions with similar latitudes.
This indicates that as the satellite passes through the high particle flux region (e.g., SAA), the ME background will rise firstly and then decay with time, i.e., the ME background has a delayed component.
Although the ME delayed background is relatively insignificant, it results in long time scale evolution of the ME background.
In order to improve the accuracy of the background estimation, the parameters of the background model should be given for every year.

\subsection{Observational characteristic and long-term evolution of the ME background}
\label{me_obs_char}

Fig.~\ref{fig:me_spec_geo} shows the spectra with different geographical latitude ranges. It can be seen that the intensity varies greatly, but the spectral shape keeps almost the same. Thus the evolution of the ME background, especially the intensity of the silver line, shall be carefully addressed in order to ensure the accuracy of the background model.

The ME background light curve exhibits significant orbital modulation (Fig.~\ref{fig:me_lc}).
A clear anti-correlation between ME background and geomagnetic cut-off rigidity has been shown in \cite{Guo2020}. There is an obvious peak in the light curve that is caused by particle events and is usually present in the high latitude region; accordingly, the corresponding time is excluded from GTI.

Fig.~\ref{fig:me_small_spec} shows the evolution of the background spectra of the small FoV detectors in the geographical region ($115^\circ<lon<125^\circ$, $-5^\circ<lat<5^\circ$).
It can be seen that the ME background level increases with the increasing in-orbit operation as a cumulative effect of the weak delayed component. Since the delayed background increases with decreasing energy roughly \citep{ZhangJ2020}, the evolution of ME background in low energy band is more significant than that in high energy band.
It is worth noting that the low-energy noise distributions have become wider as a result of the irradiation damage for some pixels, thus the spectrum below 11 keV for the fifth year has increased significantly.
The centers of the silver lines have also shifted over time, indicating a moderate change in the Energy-Channel relationship.
The spectral evolution of the blocked FoV detector is shown in Fig.~\ref{fig:me_blind_spec}. The positions of the silver lines do not shift significantly, which indicate that the blocked FoV detectors suffer less from radiation damage than the small FoV detectors.

\subsection{Validity of the ME background model}\label{me_bkg_model}
\cite{Guo2020} has constructed the background model and the corresponding database. Since the ME background spectral shape has a non-negligible change with the geographical location compared with LE, the average background of each detector at each geographical location must be considered in the background model.
In each background estimate, we first use the database to obtain the primary prediction spectra of the small FoV and blocked FoV detector in each geographical location the satellite has passed by, and then use the observation of the blocked FoV detector to make a further correction. The ME database produces the background spectra with time-averaged normalization for each individual geographical site, and the contemporary particle intensity can be determined by the blocked FoV detector and then used to correct the background model.

In each year, the backgrounds of all blank sky observations are estimated by the background model with the model parameters in the corresponding year. Fig.~\ref{fig:me_bkg_est} is an example of the ME background estimation to an blank sky observation (ObsID: P030129306901). A statistical analysis with the method in \cite{Guo2020} is performed to obtain the systematic error of the background estimation of each energy band. Fig.~\ref{fig:me_bkg_est_eb} shows the deviations of the ME background estimation for six energy bands in the fourth year.
Fig.~\ref{fig:me_err_sys} shows the systematic errors for six energy bands over the five years. The results show that the systematic error has no significant increasing trend over the five years. It can be seen that the systematic errors are relatively large in 10--15~keV with a mean value of $\sim2\%$, and the systematic errors in 10--40~keV are $\sim1.6\%$, indicating that the ME background model is still reliable.
It is worth mentioning that the detection below 10~keV is affected significantly by the electronic noise, so the current reliable energy range of ME begins from 10~keV.

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f11.pdf}
		\caption{Geographical distributions of the background of ME small FoV detectors in the first and fifth years since \textit{Insight}-HXMT operation in orbit.}\label{fig:me_map}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f12.pdf}
		\caption{Spectra of the ME small FoV detectors in different geographical latitude ranges.}\label{fig:me_spec_geo}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f13.pdf}
		\caption{Light curves of the background observation by the ME small FoV detectors in six energy bands (ObsID: P040129309401, T0=2022-06-18T07:15:53.5).}\label{fig:me_lc}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[scale=0.4]{f14.pdf}
		\caption{Background spectra of the ME small FoV detectors in every year since \textit{Insight}-HXMT operation in orbit.}\label{fig:me_small_spec}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[scale=0.4]{f15.pdf}
		\caption{Same as Fig.~\ref{fig:me_small_spec} but for the blocked FoV detectors.}\label{fig:me_blind_spec}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f16.pdf}
		\caption{An example of the ME background estimation (ObsID: P030129306901). Top: spectrum of a blank sky observation (black points) and the estimated background spectrum (red line). Bottom: residuals in terms of errors ($\sigma$).}
		\label{fig:me_bkg_est}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.9\textwidth]{f17.pdf}
		\caption{Deviations of the ME background estimation for six energy bands in the fourth year.}\label{fig:me_bkg_est_eb}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[scale=0.4]{f18.pdf}
		\caption{Systematic errors of the ME background model in every year.}\label{fig:me_err_sys}
\end{figure}

\section{Background of the High-Energy Telescope}
\label{he_bkg}
HE has 18 NaI(Tl)/CsI(Na) phoswich detectors that are surrounded by 18 anti-coincidence detectors (ACDs) for active background shielding. Among the 18 detectors, 15 of which have a small FoV, two have a large FoV and one has a blocked FoV for background estimate.
The on-ground simulations have shown that the NaI and CsI crystals can be activated by the charged particles around the orbit of \textit{Insight}-HXMT, and the radioactive decay of the activated crystals is responsible for most of the HE background. As the satellite operates in orbit continuously, the crystals in HE detectors continue to be activated. After a significant rising in the first year, the rising trend of the HE background has gradually slowed down \citep{LiG2009,XieF2015}.

\subsection{Observational characteristic and long-term evolution of the HE background}\label{he_obs_char}

The graphical distributions of the HE background for the first and fifth years are shown in Fig.~\ref{fig:he_map}. It can be seen that the distribution has little difference, but the background count rate in the fifth year is significantly higher than that in the first year. Unlike the LE and ME backgrounds that are dominated by the prompt components, HE background is dominated by the time-delayed components due to the activation of the crystals by charged particles.
Consequently, the background is very different in the ascending and descending orbit phases even for the same geographical location.

Fig.~\ref{fig:he_spec} shows the spectra at geographical locations in ascending and descending orbit phases for every year.
The spectra in $(lon,lat)=(345^\circ,15^\circ)$ are shown in Panel (e) and (f).
In the ascending orbital phase, when the satellite passes through the SAA, the crystals of the detectors seriously activated do not have enough time to decay, and thus the background has a relatively high level. However, the background in descending orbit phase is lower. As it has been a long time since the last time the satellite passed through intense charged particle region and hence the background is dominated by the long-decay components.

The HE background spectra at different geographical locations show long-term evolution.
Compared with the results shown in other panels, the evolution shown in Panel (e) is less significant. This is because the satellite has just passed through the SAA, thus a large proportion of the background is contributed by the short-time scale component. Moreover, the intensity of SAA has not changed significantly over the five years (Fig.~\ref{fig:PM_map}).

As described in \cite{LiG2009} and \cite{XieF2015}, the spectra of HE background consist of various emission lines, which are induced by the interactions of the detectors with high-energy particles. It can be seen from Fig.~\ref{fig:he_spec} that the spectral shape is stable during the five years.

Fig.~\ref{fig:he_lc} shows the light curves of HE observation of a blank sky region in six energy bands. For every energy band, the background rises to a high level when the satellite has just crossed SAA, then decays gradually and shows significant geomagnetic modulation.
There are also differences between the light curves in different energy bands, since the background is composed of numerous components characterized with different portions, spectral shapes and typical variable time scales.

\subsection{Validity of the HE background model}\label{he_bkg_model}
Based on the HE background characteristics, \cite{Liao2020_he} has made the HE background model. The principle is similar with that of ME but more complex. In order to obtain the background at any geographical location and any time, the empirical function with time as an independent variable is constructed to describe the long-term evolution of the HE background. Thus the preliminary estimation can be obtained with the orbital parameters and observation time, and some further correction will be performed with data of the blocked FoV detector.

Therefore, HE background estimation is heavily dependent on the mathematical description of the background long-term evolution, i.e., the accuracy of the empirical function is critical to the background estimation. Fig.~\ref{fig:he_long_term_lc} shows the long-term evolution of background count rates at six different geographical locations in 45--70~keV. For each energy channel, the long-term evolution for each energy channel is described by a broken line with two slopes. The fitting curve is merged from the broken lines in this energy band with different break times, thus it shows a smooth transition without an obvious break.
It is worth noting that the broken line with two slopes is a function we chose to depict the long-term evolution of background count rate over time in an energy channel.
Although other functions might be also acceptable, the selected broken-line shape could describe the observation data very well.
As predicted by the on-ground simulation \citep{LiG2009,XieF2015}, the activated isotopes lead to a rapid decay of the background rate after each SAA passage and a long-term accumulation as the in-orbit operation days increase. This accumulation rises rapidly in the initial epoch after launch and becomes slower after hundreds of days as the long lifetime isotopes are not dominant. This predicted long-term evolution is consistent with the observations shown in Fig.~\ref{fig:he_long_term_lc}.

With the background model, the background estimate for all blank sky observations is preformed. Fig.~\ref{fig:he_bkg_est} is the background estimation to a blank sky observation (ObsID: P040129307701) shown as an example.
For each year, the deviations in eight energy bands between 25~keV and 250~keV (Fig~\ref{fig:he_bkg_est_eb} for the fifth year as an example) are obtained to
calculate the systematic errors.
Following the method in \cite{Liao2020_he}, the systematic errors can be obtained for each year (Fig.~\ref{fig:he_err_sys}). The results show that they are all less than 3\%, which is not much different from the results of the previous two years. So the HE background model is still effective.

\begin{figure}[ht]%
	\centering
		\subfloat[\label{fig:he_map_a}]{\includegraphics[scale=0.5]{f19a.pdf}}
		\hspace{.26in}
		\subfloat[\label{fig:he_map_b}]{\includegraphics[scale=0.5]{f19b.pdf}}
		\caption{Panel (a): Geographical distributions (top: ascending orbital phase, bottom: descending orbital phase)
		of the HE background intensity (DetID$=0$ \& $> 30~{\rm keV}$) in the first year.
		Panel(b): same as the left panel but for the fifth year.}\label{fig:he_map}
\end{figure}

\begin{figure*}[ht]
	\centering
		\subfloat[\label{fig:he_spec_ascend_0}]{\includegraphics[scale=0.3]{f20a.pdf}}
		\subfloat[\label{fig:he_spec_descend_0}]{\includegraphics[scale=0.3]{f20b.pdf}}
	\\
		\subfloat[\label{fig:he_spec_ascend_1}]{\includegraphics[scale=0.3]{f20c.pdf}}
		\subfloat[\label{fig:he_spec_descend_1}]{\includegraphics[scale=0.3]{f20d.pdf}}
	\\
		\subfloat[\label{fig:he_spec_ascend_2}]{\includegraphics[scale=0.3]{f20e.pdf}}
		\subfloat[\label{fig:he_spec_descend_2}]{\includegraphics[scale=0.3]{f20f.pdf}}
	\caption{Spectra of the HE background (${\rm DetID}=0$ \& $> 30~{\rm keV}$) in every year with different orbital phases and geographical locations.
		 	Left: Panel (a), (c) and (e) are these for $(lon,lat)$ = $(140^\circ,0^\circ)$, $(180^\circ,0^\circ)$ and $(345^\circ,15^\circ)$ in ascending orbital phase.
			Right: same as the left panels but for the descending orbital phase.}
	\label{fig:he_spec}
\end{figure*}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f21.pdf}
		\caption{Light curves of the HE (DetID = 0) background observation in six energy bands (${\rm ObsID: P040129309001}$, T0 = 2022-05-22T18:15:38.5).
		The gap is due to the protective shutdown of HE instrument when the satellite pass through SAA.}\label{fig:he_lc}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f22.pdf}
		\caption{Long-term background evolution of the HE detector (DetID = 0) in 45--70~keV at six geographical locations.}\label{fig:he_long_term_lc}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f23.pdf}
		\caption{An example of the HE background spectrum estimation (ObsID: P040129307701). Top: spectrum of a blank sky observation (black) and the estimated background spectrum (red). Bottom: residuals in terms of errors ($\sigma$).}
		\label{fig:he_bkg_est}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.9\textwidth]{f24.pdf}
		\caption{Deviations of the HE background estimation for eight energy bands in 5-th year.}\label{fig:he_bkg_est_eb}
\end{figure}

\begin{figure}[ht]%
	\centering
		\includegraphics[width=0.6\textwidth]{f25.pdf}
		\caption{Systematic errors of the HE background model in 201707--202206.}
		\label{fig:he_err_sys}
\end{figure}

\section{Summary and Conclusion}\label{summary_conclusion}

\textit{Insight}-HXMT has been operating successfully in orbit for five years.
The backgrounds of the three telescopes exhibit different evolution trends compared with when \textit{Insight}-HXMT has just operated in orbit.
All of the background behaviors are consistent with the expectations in design.

The evolution of the LE background is mainly caused by the subsequent operations to solve the problems brought by the LE irradiation damage, e.g., the lower limit of the detection threshold is adjusted higher to avoid the noise signal whose distribution becomes wider with increasing LE irradiation damage. Moreover, the continuous broadening of the emission lines in the background spectrum is the result of the decline of the LE energy resolution.
With the increasing  operation in orbit, the ME background level increases as a cumulative effect of the weak delayed component, and the variation of ME background in low energy band is more significant than that in high energy band. In addition, the ME background in low energy band  can also be affected by the low-energy noise of some pixels.
The crystals of HE detectors continue to be activated that is why the background intensity increases obviously with time. During the first five years, the increasing trend gradually slows down and shows a behavior similar to saturation. The background evolution at different energies is not consistent, which means that the shape of the background spectrum also evolves with time for a certain geographical location.

Although the time evolution characteristics of the LE and ME backgrounds are not significant, in order to maintain the accuracy of the background estimation, the background model parameters are updated in each year. For HE background model, the evolution has been taken into account from the beginning of the model construction.
The statistical analysis shows that the systematic errors of the three telescopes change little during the first five years of \textit{Insight}-HXMT operation, thus the background models are still effective and reliable.

As described in \cite{Liao2020_le}, the LE background model constructed with the blank sky observations can effectively estimate for both the particle background and the diffuse background caused by the CXB. Therefore, it can be used to the pointing observation with the target in high Galactic latitude ($\lvert{b}\rvert\gt10^\circ$).
In order to estimate the diffuse background accurately in low Galactic latitude ($\lvert{b}\rvert\le 10^\circ$), the diffuse X-ray background in the Galactic Plane \citep{Jin2022} should be used in the LE background estimation.

It is worth noting that the current background models of the three telescopes rely heavily on the blocked FoV detectors. So the blocked FoV detectors are critical, especially for HE as it has only one blocked FoV detector. This is a potential hazard for background estimates because of insufficient robustness. Therefore, an alternative to background estimation that does not rely on blocked FoV detectors must be planned in advance, such as using ACD and PM as prompt particle monitors for background estimation of LE and ME. For HE, a parametric background model that does not rely on blocked FoV detectors has been built. By considering the physical factors that generate the HE background, a mathematical model considering these physical processes has been successfully constructed \citep{You2021}.

\bmhead{Acknowledgments}
This work made use of the data from \textit{Insight}-HXMT mission, a project funded by China National Space Administration (CNSA) and the Chinese Academy of Sciences (CAS).
The authors thank supports from the National Key R\&D Program of China (2021YFA0718500), the National Natural Science Foundation of China under Grants Nos. U1838202, U1838201. This work was partially supported by International Partnership Program of Chinese Academy of Sciences (Grant No.113111KYSB20190020).

\section*{Declarations}
On behalf of all authors, the corresponding author states that there is no conflict of interest.