\section{Introduction}

Single image super-resolution (SISR) is a low-level vision problem that recovers a high resolution (HR) image from a low resolution (LR) observation, which is an ill-posed problem because multiple HR images can be degraded to the same LR image. To address this issue, researchers have proposed many approaches, which can be divided into three subclasses including interpolation-based methods \cite{zhou-et-al:scheme}, reconstruction-based methods \cite{tai-et-al:scheme}, and learning-based methods \cite{timofte2014a+, dong, 2016Deeply, 2017Image, 2017Deep, 2017Enhanced, 2020Wavelet, 2020Multi}.

Benefit from the powerful learning ability, various deep convolutional neural network based methods have been introduced and achieved a significant performance in image SR community \cite{2020Deep}. Firstly, Dong et al. \cite{dong} proposed a three-layer super-resolution convolutional neural network (SRCNN) to learn the nonlinear mapping function between LR and HR. Later, inspired by ResNet \cite{he2016deep} and DenseNet \cite{huang2017densely}, many complex deep neural networks \cite{zhang2018residual, dai2019second-order} have been presented to boost reconstruction performance. However, doing so will cause an increase in model parameters and computational cost, which greatly limit their practical applications in some computing devices, such as mobile and embedded applications. For solving these problems, some network architectures have been proposed by integrating recursive learning and lightweight models. For example, Deeply-Recursive Convolutional Network (DRCN) \cite{2016Deeply} and Deep Recursive Residual Network (DRRN) \cite{2017Image} used a recursive network to reduce the number of network parameters by parameter sharing strategies. Although DRCN and DRRN show favorable performances with a few parameters, they have a deeper model (DRRN ups to 52 convolutional layers) and require heavy computational costs. To further save computational overhead, some works proposed to directly extract feature in LR domain, and then upscale the features by deconvolution \cite{dong2016accelerating} or sub-pixel convolution \cite{shi2016real} at the end of the network. Afterward, Enhanced Deep Super-Resolution network (EDSR) \cite{2017Enhanced} adopted this post-processing strategy and empolyed a simplified ResNet architecture, which has about 165 convolutional layers. Zhang et al. \cite{zhang2018image} utilized residual-in-residual structure by adding channel attention block to construct a very deep model (up to 400 layers) for training a SR model. From these efforts, we find that the deeper network is helpful to extract more feature information to reconstruct HR images, but most of them suffered from large network parameters and computational cost.

Recently, some other researchers began to focus on design lightweight and efficient neural networks for image SR. Ahn et al. introduced a cascading residual network (CARN) \cite{2018Fast} to achieve efficient SR by using several cascading connections with group convolution \cite{2017Interleaved}. The information distillation network (IDN) \cite{hui} employed information distillation mechanism to divide the intermediate features into two parts, one part was retained and another part was further processed by succeeding convolution operations. To better balance performance and inference applications, Hui et al. \cite{2019Lightweight} further proposed a lightweight information multi-distillation network (IMDN) that extracted hierarchical features at a granular level, and split the preceding extracted features along channel dimension. At each step, its partial feature information was retained and the other features were processed in a subsequent step. It won the first place in the AIM 2019 constrained image super-resolution challenge \cite{AIM9022144}. In addition to the above-mentioned strategies, some other efficient methods (e.g. MobileNet \cite{2017MobileNets}, self-calibrated convolution \cite{2020Improving} and network architecture search \cite{2019Fast}) have also been proposed to design a lightweight deep neural network. Although these methods obtained comparable results, there is still room to obtain a better trade-off between model lightweight and SR performance.
\begin{figure*}
	\centering
	\includegraphics[width=\linewidth]{images/PDAN.pdf}\\
	\caption{The architecture of our pyramidal dense attention network (PDAN). }\label{fig:PDAN}
\end{figure*}

In recent years, attention mechanisms have been widely used in many computer vision tasks and achieved good results \cite{zhang2018image, hu2019channel}. Hu et al. \cite{hu2018squeeze} proposed the squeeze-and-excitation block to learn the channel-wise information and improve the representation ability of the model. Inspired by \cite{hu2018squeeze}, Zhang et al. \cite{zhang2018image} introduced a residual channel attention network (RCAN) by integrating the squeeze-and-excitation block into the residual architecture for image SR. In \cite{2018CBAM}, the channel attention and spatial attention are combined to adaptively capture rich contextual information. Although these attention-based networks have provided performance improvements, they have some shortcomings. For example, SENet \cite{hu2018squeeze} only concentrate on channel-wise attention and ignore the importance of spatial information. Its variant, \cite{2018CBAM} uses the channel and spatial attention mechanisms, but they are only computed independently, which is also not beneficial to capture rich discriminative feature representations.

In this paper, we propose a novel lightweight pyramidal dense attention network (PDAN) for SISR (illustrated in Fig. \ref{fig:PDAN}). Unlike most previous dense networks with a fixed channel growth rate, we introduce a lightweight pyramidal dense block with a various channel growth rate inspired by \cite{2020MC}. The proposed feature extractor adopts the pyramidal dense learning strategy that the output channel dimensionality grows up while the convolutional layers are deepening in each pyramidal dense block, which can effectively aggregate contextual information (shown in Fig. \ref{fig:PDAB}). While increasing the output feature dimensionality is beneficial to enhance the feature learning ability of the networks, the parameter explosion needs to be noted. Therefore, we use group convolution with the dimension cardinality (i.e. the number of groups in each convolutional layer) which increases linearly to relieve the parameter explosion. Besides, to further improve the discriminative representation ability of the network, we propose a novel joint attention to capture cross-dimension interaction between the channel dimension and the spatial dimensions \cite{Misra_2021_WACV} for high-frequency information extraction. The detail of joint attention is shown in Fig. \ref{fig:PDAB} (b), where an input tensor (the size is $C \times H \times W $) is transferred to four branches, two of which are responsible for modeling channel attention ($C \times 1 \times 1$) and spatial attention ($H, W$) respectively, and the other two branches are used to build cross-dimension interaction between the channel dimension $C$ and the spatial dimension $H$ or $W$.

The main contributions of this paper are summarized as follows:

(1)	We propose a lightweight pyramidal dense attention network (PDAN) for SISR. The proposed lightweight model can effectively improve SR performance via the pyramidal dense feature learning and joint attention mechanism with a few parameters.

(2)	We introduce a pyramidal dense block to fully exploit multi-level features as well as boost the flow of information with a few parameters. Moreover, the proposed joint attention can efficiently capture cross-dimension interaction between the channel dimension and the spatial dimensions to provide richer and more discriminative contextual information for feature rescaling. In comparison with state-of-the-art lightweight SR methods, our method achieves competitive results in terms of model size and performance.

The remainder of this paper is organized as follows. In Section II, we introduce the proposed PDAN for image SR in detail. Extensive experimental results and analysis are provided in Section III. Finally, the conclusion is given in Section IV.

\section{Proposed Method} \label{sec:Methodology}

In this section, we will show the proposed lightweight pyramidal dense attention networks for SISR in details.

\subsection{Network Architecture}

As illustrated in Fig. \ref{fig:PDAN}, the proposed PDAN consists of two modules: 1) feature extraction module, 2) reconstruction module. In this paper, we denote $I^{LR}$ and $I^{SR}$ as the input LR image and output HR image of our PDAN. The LR image is first fed to one ${\rm{3}} \times {\rm{3}}$ convolution layer for initial feature extraction. Next, the learned initial features are fed into several stacked lightweight pyramidal dense attention blocks to produce powerful feature maps. Lastly, we apply the reconstruction module that consists of a ${\rm{3}} \times {\rm{3}}$  convolution layer and a sub-pixel convolution layer \cite{shi2016real} to reconstruct the desired SR image. The SR procedure of the proposed PDAN can be written as following:
\begin{equation}
{I^{SR}} = {F_{PDAN}}\left( {{I^{LR}, \theta}} \right) = {F_{UP}}\left({F_{e}}\left({{I^{LR}}} \right) \right),
\end{equation}
where ${F_{PDAN}}$  represents the reconstruction function of our PDAN, $\theta$  denotes the learnable parameters of the network, ${F_{e}}$  is the feature extraction step and  $F_{up}$ denotes the up-sampling and reconstruction step.

Our PDAN is optimized with the  $L_1$ loss \cite{zhang2018image} by measuring the difference between a reconstructed SR image $I^{SR}$  and its HR ground-truth $I^{HR}$ . Given a set of training image pairs $\left\{ {I_i^{LR},I_i^{HR}} \right\}_{i = 1}^N$ , where $N$ denotes the number of training images. The loss function of our method can be expressed as
\begin{equation}
L(\theta) = \frac{1}{N}\sum_{i=1}^N \| F_{PDAN}(I^{LR}_i, \theta) - I^{HR}_i\|_1.
\end{equation}
Next, we will give more details about the proposed lightweight pyramidal dense attention block.

\begin{figure*}
	\centering
	\includegraphics[height= 6in, width=\linewidth]{images/PDAB.pdf}\\
	\caption{(a) The illustration of the proposed pyramidal dense attention block. (b) The details of the joint attention module.}\label{fig:PDAB}
\end{figure*}

\subsection{Pyramidal Dense Attention Block (PDAB)}

As illustrated in Fig. \ref{fig:PDAB} (a), our pyramidal dense attention block (PDAB) is mainly constructed by two stages: pyramidal dense feature learning, and feature refinement based on joint attention mechanism. Moreover, local residual learning is also adopted to improve the gradient information flow. Let  $H_{d-1}$ and  ${H_d}$ denote the input and output of the $d$-th PDAB, respectively. The process flow with PDAB can be formulated as following:
\begin{equation}
H_d = H_{d-1} + F_{PDAB}(H_{d-1}),
\end{equation}
where  $F_{PDAB}$ is the mapping function of the proposed PDAB.

\subsubsection{Pyramidal Dense Feature Learning}

Dense skip connections have been widely exploited in deep learning methods, which is beneficial to boost the flow of feature information and enhance the representation ability of the network. However, on the other hand, the dense connections will need large resources and memory for extracting and storing features. Additionally, due to the inconsistency between the number of input and output channels, the dense connections with a fixed channel growth rate can cause the loss of information and degrade the learning ability of the network. In this paper, we introduce a pyramidal dense connection mechanism with a varying growth rate, where the output feature dimensionality of convolutional layers gradually increases when the network layers are deepening to relieve this problem. However, when the convolution layers increase, this method will result in the parameter explosion. Specifically, the input channel dimensionality of two different dense connection layers can be written as follows.

\noindent The conventional dense connections with a fixed growth rate are
\begin{equation}
C_i = c_0 + \sum_{j=1}^{i-1} c_j = c_0 + (i-1) \times g_0 ,
\end{equation}
\begin{equation}
c_j = g_0.
\end{equation}

\noindent The pyramidal dense connections with a varying growth rate are
\begin{equation}
\widetilde C_i = c_0 + \sum_{j=1}^{i-1} \widetilde c_j = c_0 + (i-1) \times g_0 + g \times \frac{(i-1)(i-2)}{2} ,
\end{equation}
\begin{equation}
\widetilde c_j = g_0 + g \times (j-1),
\end{equation}
where $c_0$ is the number of initial input channels, $g$ is a increasing factor of growth rate $g_0$. $C_i$ and $\widetilde C_i$ are the number of input channels in the $i$-th layer, $c_j$ and $\widetilde c_j$ denote the output feature dimensionality of $j$-th convolutional layer.

From the above analysis, the group convolution is adopted to avoid the problem of parameter explosion. Note that we choose group convolution instead of conventional convolution since the former can reduce the number of parameters efficiently. Specifically, the corresponding parameters of them are given as
\begin{equation}
P_i = C^{out}_i \times C^{in}_i \times K \times K,
\end{equation}
\begin{equation}
P^G_i = G_i \times (\frac{C^{out}_i}{G_i} \times \frac{C^{in}_i}{G_i} \times K \times K) = \frac{P_i}{G_i},
\end{equation}
where $C^{in}_i$ and $C^{out}_i$ are the number of input and output channels in the $i$-th layer, $K$ indicates the corresponding kernel size, $G_i$ is the group size and the bias is omitted for simplicity. Obviously, the parameters of group convolution ($P^G$) are less than the parameters of conventional convolution ($P$). However, when $G_i$ is fixed, this will also cause the parameter to increase sharply since the output feature channels $C^{out}$ grows up linearly with the dense layers. Therefore, we introduce the adaptive group convolution to further alleviate the parameter explosion. In the experiment, the group size is set to grow linearly with dense convolutional layers,
\begin{equation}
G_i = i+1.
\end{equation}

Moreover, the bottleneck architecture ($1 \times 1$ convolutional filters) is also designed to refine the input cascaded features to be consistent with the output channels of the corresponding layer, which can keep them divisible by the group size and boost the information propagation among different group features in each layer.

\subsubsection{Joint Attention Mechanism (JA)}

The aim of the attention-based SR methods is to selectively focus on more important high-frequency details of the input feature for HR image recovery. Here, we introduce a novel joint attention that can capture cross-dimension dependencies \cite{Misra_2021_WACV} for improving the discriminative feature representations. First, we will revisit some attention mechanisms including channel attention and spatial attention \cite{hu2018squeeze,2018CBAM}.

\textbf{Convolutional Block Attention Module}: Unlike SENet \cite{hu2018squeeze}, convolutional block attention module (CBAM) \cite{2018CBAM} compute channel attention by using global average pooling and global max pooling. Specifically, let $X\in \mathbb{R}^{C\times H \times W}$ denote the input feature where $C$ is the number of input channels, $H$ and $W$ are height and width of the input features, respectively. The channel-wise global statistics are defined as
\begin{equation}
GAP(X) = \frac{1}{H\times W}\sum_{i=1}^H\sum_{j=1}^W X_{i,j},
\end{equation}
\begin{equation}
GMP(X) = \max\limits_{H,W}\ (X_{i,j}).
\end{equation}
Next, the multi-layer perception is shared to compute channel attention,
\begin{equation}
A_e = \sigma(W_2\delta(W_1GAP(X))+W_2\delta(W_1GMP(X))),
\end{equation}
where $\sigma$ and $\delta$ are the sigmoid function and the Rectified Linear Unit \cite{2011Deep}, $W_1\in \mathbb{R}^{\frac{C}{r}\times C \times 1 \times 1}$, $W_2\in \mathbb{R}^{C \times \frac{C}{r} \times 1 \times 1}$, and $r$ is the reduction radio. Moreover, the above average pooling and max pooling operations along the channel dimension are also used to produce the spatial attention weights in CBAM. The formulation can be written by
\begin{equation}
A_s = \sigma(w^{k \times k}([X^{avg}_s, X^{max}_s])),
\end{equation}
where $X^{avg}_s$, $X^{max}_s\in \mathbb{R}^{1 \times H \times W}$ , $W^{k \times k}$ denotes a convolution layer with the filter size of $k \times k$  and $\sigma$ is the sigmoid function.

\textbf{Joint Attention module}: Although CBAM \cite{2018CBAM} introduces channel attention and spatial attention to improve the feature representations, they ignore the importance of cross-dimension interaction. For this problem, we propose the joint attention to effectively model channel attention and spatial attention with few parameters.

As shown in Fig. \ref{fig:PDAB} (b), joint attention consists of four branches, two branches are responsible for modeling the channel attention ($C \times 1 \times 1$) and the spatial attention ($H, W$). The rest two branches aim to capture the interaction between the channel dimension and the spatial dimensions ($(H, C)$ or $(C,W)$). Specifically, given a input feature map $X \in \mathbb{R}^{C \times H \times W}$. In the channel attention branch, like SENet \cite{hu2018squeeze}, the channel attention weights are used to refine the feature $\hat X_{C}$. In the spatial attention branch, we first aggregate channel information to produce two 2D maps ($1 \times H \times W$ ) by the above average-pooling and max-pooling operations across the channel axis. Then, a $3 \times 3$ dilated convolution \cite{2016Multi} with dilation value 3 followed by a batch normalization \cite{2015Batch} and a sigmoid activation function are used for producing the attention weights ($1 \times H \times W$). Finally, the spatial attention weights are multiplied with the input tensor $X$ to generate the recalibrated feature $\hat X_{H \times W}$. For the remaining two branches, the input tensor $X$ is first permuted and then passed to the above similar operations to generate the corresponding attention weights ($1 \times H \times C$ or $1 \times C \times W$  ), respectively. Subsequently, the generated attention weights are applied on the permuted feature and then rotated ${90^ \circ }$ clockwise along the width axis or the height axis to retain the original input shape ($C \times H \times W$). Finally, the refined features of four branches ($\hat X_{C}$, $\hat X_{H \times W}$, $\hat X_{H \times C}$, $\hat X_{C \times W}$) are aggregated by the averaging operation to enhance the discriminative representation ability of the network, which can be written as
\begin{equation}
\hat X = \frac {\hat X_{C} + \hat X_{H \times W} + \hat X_{H \times C} + \hat X_{C \times W}} {4},
\end{equation}
where $\hat X$ is the output recalibrated feature. In addition to simple averaging, other simple and efficient aggregation methods can also be explored to further improve the performance.

\section{Experimental Results and Analysis} \label{sec:Experiments}

\subsection{Datasets and Metrics}

In our experiments, we use DIV2K dataset \cite{timofte2017ntire} as our training set, which contains 800 high-quality training images. We employ five standard benchmark datasets for testing our model, including Set5 \cite{bevilacqua2012low}, Set14 \cite{2010On}, Bsd100 \cite{2002A}, Urban100 \cite{2015Single} and Manga109 \cite{2017Sketch}. We conduct three different experiments with bicubic (BI),
blur-downscale (BD) and downscale-noise (DN) degradation models. The peak signal-to-noise ratio (PSNR) and the structural similarity index measurement (SSIM) \cite{wang2004image} are used for measuring the quality of the SR images on the luminance (Y) channel of the transformed YCbCr space.

\begin{table*}
	\newcommand{\tabincell}[2]{\begin{tabular}{@{}#1@{}}#2\end{tabular}}
	\centering
	\caption{Ablation study of different combinations in the method. We report the parameters, FLOPs and PSNR on Manga109 ($\times {\rm{4}}$).}\label{ablation}
	\begin{tabular}{l|C{1.2cm}|C{1.2cm}|C{1.2cm}|C{1.2cm}}
		\toprule
		& ${P_{\rm{1}}}$ & ${P_{\rm{2}}}$ & ${P_{\rm{3}}}$ & ${P_{\rm{4}}}$ \\
		\midrule
		Channel attention \cite{hu2018squeeze} & & \checkmark & & \\
		\hline
		Channel and Spatial attention \cite{2018CBAM} & & & \checkmark & \\
		\hline
		Joint attention & & & & \checkmark \\
		\hline
		Params (K) & 1471 & 1586 & 1588 & 1587 \\
		\hline
		FLOPs (G) & 31.78 & 31.85 & 31.87 & 31.87 \\
		\hline
		PSNR (dB) & 30.47 & 30.59 & 30.46 & 30.64 \\
		\bottomrule
	\end{tabular}
\end{table*}

\subsection{Implementation Details}

To construct the training pairs, we downsample the original HR images to generate LR Images by using bicubic interpolation. Data augmentation is performed on the above training dataset by randomly rotating ${90^ \circ }$, ${\rm{18}}{{\rm{0}}^ \circ}$, ${\rm{27}}{{\rm{0}}^ \circ }$ and flipping horizontally. In each training batch, 16 LR RGB patches with the size of $48 \times 48$ are randomly sampled as inputs. We adopt Adam optimizer \cite{2014Adam} to train the model by setting ${\beta _{\rm{1}}}{\rm{ = 0}}{\rm{.9}}$, ${\beta _{\rm{2}}}{\rm{ = 0}}{\rm{.999}}$, and $\varepsilon {\rm{ = 1}}{{\rm{0}}^{ - 8}}$. The learning rate is initialized as ${10^{ - 4}}$ and reduced to half every 200 epochs. We implement the proposed method by using the PyTorch framework \cite{paszke2017automatic} with a NVIDIA RTX 2080Ti GPU.

In our proposed model, the convolutional layers are set to 64 filters with $3 \times 3$ kernel size except for $1 \times 1$ convolutional layers for feature fusion. Additionally, in each pyramidal dense block, the initial input features are first reduced to $c_0$ and then passed to the subsequent pyramidal dense connection layers. For simplicity, the channel number of pyramidal dense layers can be expressed linearly as $c_j = (j+1) \times c_0$.  The operations of each pyramidal dense layer consists of a $1 \times 1$ convolution filter, activation function ReLU, $3 \times 3$ adaptive group convolution and ReLU. Specifically, $c_0$ is set to 16 and the number of pyramidal dense layers is 4 in a pyramidal dense block. Following the previous papers, we use the recently popular sampling method ESPCNN \cite{shi2016real} to perform the upsampling operation. The final convolution layer has 3 filters with kernel size of $3 \times 3$ that are used to reconstruct the HR image.

\begin{figure*}
	\centering
	\includegraphics[width=\linewidth]{images/urban.pdf}\\
	\caption{Visual comparison for $\times 4$ SR with BI model on Urban100 dataset. The best results are highlighted.
	}\label{fig:urban}
\end{figure*}

\subsection{Ablation Study}

In this subsection, we analyze the effects of the key configurations in our proposed method. For fair comparisons, our method and its variants are set with the same baseline structure. As shown in Table \ref{ablation}, we can see that the baseline method $P_1$ (without any attention) performs not very well and PSNR only reaches 30.47 dB on Manga109 ($\times 4$). It is noticed that when we compare the joint attention ($P_4$) with other attention mechanisms ($P_2$ and $P_3$), we can see that our joint attention module can achieve better performance (i.e., 30.64 v.s. 30.59 dB v.s. 30.46 dB).

In addition, we also compare other metrics, including parameters and FLOPs to demonstrate the effectiveness of our method. FLOPs is computed on the HR image with $512 \times 512$ pixels. From Table \ref{ablation}, it is observed that our joint attention model can improve the PSNR with less additional parameters and FLOPs. The results clearly indicate that joint attention is more effective than other attention mechanisms \cite{hu2018squeeze,2018CBAM}.

\begin{table*}
	\newcommand{\tabincell}[2]{\begin{tabular}{@{}#1@{}}#2\end{tabular}}
	\centering
	\caption{Quantitative results of different image SR methods with BI degradation module.The parameters and PSNR/SSIM comparison results for scaling factor $ \times {\rm{2}}$, $ \times {\rm{3}}$ and $ \times {\rm{4}}$. Red/blue text: best/second-best among all methods.}\label{comp_with_others_BI}
	\begin{tabular}{L{1.8cm}|C{0.8cm}|C{0.8cm}|C{0.8cm}C{0.8cm}|C{0.8cm}C{0.8cm}|C{0.8cm}C{0.8cm}|C{0.8cm}C{0.8cm}|C{0.8cm}C{0.8cm}}
		\toprule
		\multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} &
		\multicolumn{2}{c}{Set5} & \multicolumn{2}{c}{Set14} & \multicolumn{2}{c}{Bsd100} & \multicolumn{2}{c}{Urban100} & \multicolumn{2}{c}{Manga109}\\
		Methods & Scale & Params & PSNR & SSIM & PSNR & SSIM & PSNR & SSIM & PSNR & SSIM & PSNR & SSIM \\
		\midrule
		Bicubic & 2 & $-$ & 33.66 & 0.9299 & 30.24 & 0.8688 & 29.56 & 0.8431 & 26.88 & 0.8403 & 30.80 & 0.9339 \\
		SRCNN \cite{dong} & 2 & 57K & 36.66 & 0.9542 & 32.45 & 0.9067 & 31.36 & 0.8879 & 29.50 & 0.8946 & 35.60 & 0.9663 \\
		FSRCNN \cite{dong2016accelerating} & 2 & 13K & 37.00 & 0.9558 & 32.63 & 0.9088 & 31.53 & 0.8920 & 29.88 & 0.9020 & 36.67 & 0.9710 \\
		VDSR \cite{kim2016accurate} & 2 & 666K & 37.53 & 0.9587 & 33.03 & 0.9124 & 31.90 & 0.8960 & 30.76 & 0.9140 & 37.22 & 0.9750 \\
		DRCN \cite{2016Deeply} & 2 & 1774K & 37.63 & 0.9588 & 33.04 & 0.9118 & 31.85 & 0.8942 & 30.75 & 0.9133 & 37.55 & 0.9732 \\
		LapSRN \cite{lai2017deep} & 2 & 251K & 37.52 & 0.9591 & 32.99 & 0.9124 & 31.80 & 0.8952 & 30.41 & 0.9103 & 37.27 & 0.9740 \\
		DRRN \cite{2017Image} & 2 & 298K & 37.74 & 0.9591 & 33.23 & 0.9136 & 32.05 & 0.8973 & 31.23 & 0.9188 & 37.88 & 0.9749 \\
		SRMDNF \cite{zhang2018learning} & 2 & 1511K & 37.79 & 0.9601 & 33.32 & 0.9159 & 32.05 & 0.8985 & 31.33 & 0.9204 & 38.07 & 0.9761 \\
		SRResNet \cite{ledig2017photo} & 2 & 1370K & \textcolor{blue}{38.05} & \textcolor{blue}{0.9607} & \textcolor{blue}{33.64} & \textcolor{blue}{0.9178} & \textcolor{red}{32.22} & \textcolor{red}{0.9002} & \textcolor{blue}{32.23} & \textcolor{blue}{0.9295} & 38.05 & 0.9607 \\
		CARN \cite{2018Fast} & 2 & 1592K & 37.76 & 0.9590 & 33.52 & 0.9166 & 32.09 & 0.8978 & 31.92 & 0.9256 & \textcolor{blue}{38.36} & \textcolor{blue}{0.9765} \\
		PDAN (ours) & 2 & 1439K & \textcolor{red}{38.05} & \textcolor{red}{0.9607} & \textcolor{red}{33.65} & \textcolor{red}{0.9182} & \textcolor{blue}{32.20} & \textcolor{blue}{0.8998} & \textcolor{red}{32.36} & \textcolor{red}{0.9300} & \textcolor{red}{38.71} & \textcolor{red}{0.9771} \\
		\midrule
		Bicubic & 3 & $-$ & 30.39 & 0.8682 & 27.55 & 0.7742 & 27.21 & 0.7385 & 24.46 & 0.7349 & 26.95 & 0.8556 \\
		SRCNN \cite{dong} & 3 & 57K & 32.75 & 0.9090 & 29.30 & 0.8215 & 28.41 & 0.7863 & 26.24 & 0.7989 & 30.48 & 0.9117 \\
		FSRCNN \cite{dong2016accelerating} & 3 & 13K & 33.18 & 0.9140 & 29.37 & 0.8240 & 28.53 & 0.7910 & 26.43 & 0.8080 & 31.10 & 0.9210 \\
		VDSR \cite{kim2016accurate} & 3 & 666K & 33.66 & 0.9213 & 29.77 & 0.8314 & 28.82 & 0.7976 & 27.14 & 0.8279 & 32.01 & 0.9340 \\
		DRCN \cite{2016Deeply} & 3 & 1774K & 33.82 & 0.9226 & 29.76 & 0.8311 & 28.80 & 0.7963 & 27.15 & 0.8276 & 32.24 & 0.9343 \\
		LapSRN \cite{lai2017deep} & 3 & 502K & 33.81 & 0.9220 & 29.79 & 0.8325 & 28.82 & 0.7980 & 27.07 & 0.8275 & 32.21 & 0.9350 \\
		DRRN \cite{2017Image} & 3 & 298K & 34.03 & 0.9244 & 29.96 & 0.8349 & 28.95 & 0.8004 & 27.53 & 0.8378 & 32.71 & 0.9379 \\
		SRMDNF \cite{zhang2018learning} & 3 & 1528K & 34.12 & 0.9254 & 30.04 & 0.8382 & 28.97 & 0.8025 & 27.57 & 0.8398 & 33.00 & 0.9403 \\
		SRResNet \cite{ledig2017photo} & 3 & 1554K & \textcolor{blue}{34.41} & \textcolor{blue}{0.9274} & \textcolor{blue}{30.36} & \textcolor{blue}{0.8427} & \textcolor{blue}{29.11} & \textcolor{blue}{0.8055} & \textcolor{blue}{28.20} & \textcolor{blue}{0.8535} & \textcolor{blue}{33.54} & \textcolor{blue}{0.9448} \\
		CARN \cite{2018Fast} & 3 & 1592K & 34.29 & 0.9255 & 30.29 & 0.8407 & 29.06 & 0.8034 & 28.06 & 0.8493 & 33.50 & 0.9440 \\
		PDAN (ours) & 3 & 1624K & \textcolor{red}{34.44} & \textcolor{red}{0.9276} & \textcolor{red}{30.39} & \textcolor{red}{0.8437} & \textcolor{red}{29.11} & \textcolor{red}{0.8063} & \textcolor{red}{28.34} & \textcolor{red}{0.8563} & \textcolor{red}{33.63} & \textcolor{red}{0.9448} \\
		\midrule
		Bicubic & 4 & $-$ & 28.42 & 0.8104 & 26.00 & 0.7027 & 25.96 & 0.6675 & 23.14 & 0.6577 & 24.89 & 0.7866 \\
		SRCNN \cite{dong} & 4 & 57K & 30.48 & 0.8628 & 27.50 & 0.7513 & 26.90 & 0.7101 & 24.52 & 0.7221 & 27.58 & 0.8555 \\
		FSRCNN \cite{dong2016accelerating} & 4 & 12K & 30.72 & 0.8660 & 27.61 & 0.7550 & 26.98 & 0.7150 & 24.62 & 0.7280 & 27.90 & 0.8610 \\
		VDSR \cite{kim2016accurate} & 4 & 665K & 31.35 & 0.8830 & 28.02 & 0.7680 & 27.29 & 0.7251 & 25.18 & 0.7540 & 28.83 & 0.8870 \\
		DRCN \cite{2016Deeply} & 4 & 1774K & 31.53 & 0.8854 & 28.02 & 0.7670 & 27.23 & 0.7233 & 25.14 & 0.7510 & 28.98 & 0.8816 \\
		LapSRN \cite{lai2017deep} & 4 & 813K & 31.54 & 0.8850 & 28.19 & 0.7720 & 27.32 & 0.7270 & 25.21 & 0.7560 & 29.09 & 0.8900 \\
		DRRN \cite{2017Image} & 4 & 297K & 31.68 & 0.8888 & 28.21 & 0.7720 & 27.38 & 0.7284 & 25.44 & 0.7638 & 29.45 & 0.8946 \\
		SRMDNF \cite{zhang2018learning} & 4 & 1552K & 31.96 & 0.8925 & 28.35 & 0.7787 & 27.49 & 0.7337 & 25.68 & 0.7731 & 30.09 & 0.9024 \\
		SRResNet \cite{ledig2017photo} & 4 & 1518K & \textcolor{blue}{32.17} & \textcolor{blue}{0.8951} & \textcolor{blue}{28.61} & \textcolor{blue}{0.7823} & \textcolor{blue}{27.59} & \textcolor{blue}{0.7365} & \textcolor{blue}{26.12} & \textcolor{blue}{0.7871} & \textcolor{blue}{30.48} & \textcolor{blue}{0.9087} \\
		CARN \cite{2018Fast} & 4 & 1592K & 32.13 & 0.8937 & 28.60 & 0.7806 & 27.58 & 0.7349 & 26.07 & 0.7837 & 30.47 & 0.9084 \\
		PDAN (ours) & 4 & 1587K & \textcolor{red}{32.28} & \textcolor{red}{0.8957} & \textcolor{red}{28.66} & \textcolor{red}{0.7831} & \textcolor{red}{27.62} & \textcolor{red}{0.7378} & \textcolor{red}{26.27} & \textcolor{red}{0.7922} & \textcolor{red}{30.64} & \textcolor{red}{0.9098} \\
		\bottomrule
	\end{tabular}
\end{table*}

\subsection{Results with Bicubic Degradation (BI)}

We compare our proposed method with some state-of-the art SR methods, Bicubic, SRCNN \cite{dong}, FSRCNN \cite{dong2016accelerating}, VDSR \cite{kim2016accurate}, DRCN \cite{2016Deeply}, LapSRN \cite{lai2017deep}, DRRN \cite{2017Image}, SRMDNF \cite{zhang2018learning}, SRResNet \cite{ledig2017photo} and CARN \cite{2018Fast}. Table \ref{comp_with_others_BI} shows quantitative results for three different scaleing factors ($\times 2$, $\times 3$, $\times 4$). In addition, the number of parameters of these methods is also provided to show the model complexity. The results show that our PDAN achieves the comparable results on all the datasets and scaling factors in terms of both PSNR and SSIM. Although CARN \cite{2018Fast} has similar parameters as us, our PDAN can obtains superior performance. For example, when the scaling factor is $\times 4$, the average PSNR gain of our PDAN over the CARN \cite{2018Fast} are 0.20 dB, 0.17 dB on Urban100 and Manga109 datasets, respectively.

In Fig. \ref{fig:urban}, we present visual comparison with upscaling factor $\times 4$ on Urban100 dataset. We can see that that our method yields best visual results among all existing compared methods. For ``$image\_024$'', the other compared methods generate the blurred detailed textures. Our method can recover more accurate lines and details. For ``$image\_073$'', most methods suffer from the blurry effects while our method produces the sharper edges and finer details.

\begin{figure}
	\centering
	\includegraphics[width=\linewidth]{images/sandian.pdf}\\
	\caption{PSNR v.s. Parameters. The PSNR values are evaluated on Urban100 with scaling factor $\times 4$}\label{fig:sandian}
\end{figure}

\begin{table*}
	\newcommand{\tabincell}[2]{\begin{tabular}{@{}#1@{}}#2\end{tabular}}
	\centering
	\caption{Quantitative results with BD and DN degradation modules for scaling factor $ \times 3$. Red/blue text: best/second-best among all methods.}\label{comp_with_others_BD}
	\begin{tabular}{L{1.8cm}|C{0.8cm}|C{0.8cm}C{0.8cm}|C{0.8cm}C{0.8cm}|C{0.8cm}C{0.8cm}|C{0.8cm}C{0.8cm}|C{0.8cm}C{0.8cm}}
		\toprule
		\multicolumn{1}{c}{} & \multicolumn{1}{c}{} &
		\multicolumn{2}{c}{Set5} & \multicolumn{2}{c}{Set14} & \multicolumn{2}{c}{Bsd100} & \multicolumn{2}{c}{Urban100} & \multicolumn{2}{c}{Manga109}\\
		Methods & Model & PSNR & SSIM & PSNR & SSIM & PSNR & SSIM & PSNR & SSIM & PSNR & SSIM \\
		\midrule
		\midrule
		\multirow{2}{*}{Bicubic} & BD & 28.78 & 0.8308 & 26.38 & 0.7271 & 26.33 & 0.6918 & 23.52 & 0.6862 & 25.46 & 0.8149 \\
		& DN & 24.01 & 0.5369 & 22.87 & 0.4724 & 22.92 & 0.4449 & 21.63 & 0.4687 & 23.01 & 0.5381 \\
		\midrule
		\multirow{2}{*}{SRCNN \cite{dong}} & BD & 32.05 & 0.8944 & 28.80 & 0.8074 & 28.13 & 0.7736 & 25.70 & 0.7770 & 29.47 & 0.8924 \\
		& DN & 25.01 & 0.6950 & 23.78 & 0.5898 & 23.76 & 0.5538 & 21.90 & 0.5737 & 23.75 & 0.7148 \\
		\midrule
		\multirow{2}{*}{FSRCNN \cite{dong2016accelerating}} & BD & 26.23 & 0.8124 & 24.44 & 0.7106 & 24.86 & 0.6832 & 22.04 & 0.6745 & 23.04 & 0.7927 \\
		& DN & 24.18 & 0.6932 & 23.02 & 0.5856 & 23.41 & 0.5556 & 21.15 & 0.5682 & 22.39 & 0.7111 \\
		\midrule
		\multirow{2}{*}{VDSR \cite{kim2016accurate}} & BD & 33.25 & 0.9150 & 29.46 & 0.8244 & 28.57 & 0.7893 & 26.61 & 0.8136 & 31.06 & 0.9234 \\
		& DN & 25.20 & 0.7183 & 24.00 & 0.6112 & 24.00 & 0.5749 & 22.22 & 0.6096 & 24.20 & 0.7525 \\
		\midrule
		\multirow{2}{*}{IRCNN\_G \cite{zhang2017learning}} & BD & 33.38 & 0.9182 & 29.63 & 0.8281 & 28.65 & 0.7922 & 26.77 & 0.8154 & 31.15 & 0.9245 \\
		& DN & 25.70 & 0.7379 & 24.45 & 0.6305 & 24.28 & 0.5900 & 22.90 & 0.6429 & 24.88 & 0.7765 \\
		\midrule
		\multirow{2}{*}{IRCNN\_C \cite{zhang2017learning}} & BD & 33.17 & 0.9157 & 29.55 & 0.8271 & 28.49 & 0.7886 & 26.47 & 0.8081 & 31.13 & 0.9236 \\
		& DN & 27.48 & 0.7925 & 25.92 & 0.6932 & 25.55 & 0.6481 & 23.93 & 0.6950 & 26.07 & 0.8253 \\
		\midrule
		\multirow{2}{*}{SRMDNF \cite{zhang2018learning}} & BD & 34.09 & 0.9242 & 30.11 & 0.8364 & 28.98 & 0.8009 & 27.50 & 0.8370 & 32.97 & 0.9391 \\
		& DN & 27.74 & 0.8026 & 26.13 & 0.6974 & 25.64 & 0.6495 & 24.28 & 0.7092 & 26.72 & 0.8424 \\
		\midrule
		\multirow{2}{*}{PDAN (ours)} & BD & \textcolor{blue}{34.47} & \textcolor{blue}{0.9270} & \textcolor{blue}{30.43} & \textcolor{blue}{0.8425} & \textcolor{blue}{29.16} & \textcolor{blue}{0.8058} & \textcolor{blue}{28.28} & \textcolor{blue}{0.8533} & \textcolor{blue}{33.85} & \textcolor{blue}{0.9448} \\
		& DN & \textcolor{red}{28.51} & \textcolor{red}{0.8156} & \textcolor{blue}{26.57} & \textcolor{red}{0.7101} & \textcolor{red}{25.93} & \textcolor{red}{0.6600} & \textcolor{blue}{24.84} & \textcolor{blue}{0.7340} & \textcolor{blue}{27.88} & \textcolor{blue}{0.8574} \\
		\midrule
		\midrule
		\multirow{2}{*}{RDN \cite{zhang2018residual}} & BD & \textcolor{red}{34.58} & \textcolor{red}{0.9280} & \textcolor{red}{30.53} & \textcolor{red}{0.8447} & \textcolor{red}{29.23} & \textcolor{red}{0.8079} & \textcolor{red}{28.46} & \textcolor{red}{0.8582} & \textcolor{red}{33.97} & \textcolor{red}{0.9465} \\
		& DN & \textcolor{blue}{28.47} & \textcolor{blue}{0.8151} & \textcolor{red}{26.60} & \textcolor{blue}{0.7101} & \textcolor{blue}{25.93} & \textcolor{blue}{0.6573} & \textcolor{red}{24.92} & \textcolor{red}{0.7364} & \textcolor{red}{28.00} & \textcolor{red}{0.8591} \\
		\bottomrule
	\end{tabular}
\end{table*}

\subsection{Results with BD and DN Degradations}

Following \cite{zhang2018residual, zhang2017learning}, we also carry out experiments with blur-downscale (BD) and downscale-noise (DN) degradation models. The proposed method PDAN is compared with some state-of-the-art SR methods: SRCNN \cite{dong}, FSRCNN \cite{dong2016accelerating}, VDSR \cite{kim2016accurate}, IRCNN\_G \cite{zhang2017learning}, IRCNN\_C  \cite{zhang2017learning}, SRMDNF \cite{zhang2018learning}, and RDN \cite{zhang2018residual}. As shown in Table \ref{comp_with_others_BD}, our PDAN achieve the best performance on almost all quantitative results compared with other SR methods with scaling factor $ \times {\rm{3}}$. In particular, in comparison with RDN \cite{zhang2018residual}, our proposed PDAN still obtains comparable results. It is worth noting that PDAN has fewer parameters than RDN (1.6 M v.s. 22.3 M, 14$ \times $ less).

\subsection{Model size}

The number of parameters is a key factor for constructing a lightweight image SR model. From Table \ref{comp_with_others_BI}, we can observe that our method achieve better results than other stater-of-the-art lightweight methods. As shown in Fig. \ref{fig:sandian}, we illustrate the comparisons about parameters and PSNR on Urban100 with upscaling factor $\times 4$. In comparison with other methods, our PDAN obtains superior performance with comparable model size. It demonstrates that our method achieves a better trade-off between the performance and model size.

\section{Conclusion} \label{sec:Conclusion}

In this paper, we propose a lightweight pyramidal dense attention network (PDAN) for image SR. Specifically, in the proposed pyramidal dense attention block, the pyramidal dense connections can efficiently exploit feature information in different layers. Meanwhile, the adaptive group convolution whose group size increases linearly with layers is presented to relieve the parameter explosion. In addition, we introduce a new joint attention mechanism to recalibrate the feature responses more accurately by considering cross-dimension interaction in an efficient way. Extensive experiments on image SR demonstrate the superior performance of our PDAN in terms of quantitative and qualitative results.

\appendices

\ifCLASSOPTIONcaptionsoff
\newpage
\fi

\bibliographystyle{IEEEtran}


\section{Preliminaries}\label{S1}

Denote by $S(\R^d)$ the Schwartz space of test functions $\p\in C^\infty(\R^d)$ with the finite norms
 $$
  N_n(\p)=\sup_{\R^d}\max_{\|k\|\le n}\,(1+|x|)^n |D^k\p(x)|,\quad n=0,1,2,\dots,
 $$
where
$$
|x|=(x_1^2+\dots+x_d^2)^{1/2},\   k=(k_1,\dots,k_d)\in(\N\cup\{0\})^d,\ \|k\|=k_1+\dots+k_d,\  D^k=\partial^{k_1}_{x_1}\dots\partial^{k_d}_{x_d}.
 $$
 These norms generate the topology on $S(\R^d)$.  Elements of the space $S^*(\R^d)$ of continuous linear functionals on $S(\R^d)$ are called {\it temperate distributions}.
The Fourier transform of a temperate distribution $f$ is defined by the equality
\begin{equation}\label{a0}
\hat f(\p)=f(\hat\p)\quad\mbox{for all}\quad\p\in S(\R^d),
\end{equation}
where
$$
   \hat\p(y)=\int_{\R^d}\p(x)\exp\{-2\pi i\la x,y\ra\}dx
 $$
is the Fourier transform of the function $\p$. Also,
$$
   \check\p(y)=\int_{\R^d}\p(x)\exp\{2\pi i\la x,y\ra\}dx
 $$
 means the inverse Fourier transform. Note that the Fourier transform is an isomorphism of $S(\R^d)$ on $S(\R^d)$ and, respectively, $S^*(\R^d)$ on $S^*(\R^d)$.

  We will say that a set $A\subset\R^d$ is {\it locally finite} if the intersection of $A$ with any ball is finite,  $A$ is {\it relatively dense} if there is $R<\infty$
  such that $A$ intersects with each ball of radius $R$, and $A$ is {\it uniformly discrete}, if $A$ is locally finite and has a strictly positive separating constant
 $$
 \eta(A):=\inf\{|x-x'|:\,x,\,x'\in A,\,x\neq x'\}.
  $$
  A locally finite set $A$ is of {\it bounded density} if
  $$
        \sup_{x\in\R^d}\# A\cap B(x,1)<\infty.
  $$
  As usual, $\# E$ means a number of elements of the finite set $E$, and $B(x,r)$ means the ball with center in $x$ and radius $r$.

  An element $f\in S^*(\R^d)$ is called {\it a crystalline measure} if $f$ and $\hat f$ are  complex-valued measures on $\R^d$ with locally finite supports.
  The support of $\hat f$ for a  distribution $f\in S^*(\R^d)$ is called {\it spectrum} of $f$.

  Denote by $|\mu|(A)$ the variation of a complex-valued measure $\mu$ on $A$. If both measures $|\mu|$ and $|\hat\mu|$ have  locally finite supports and belong to $S^*(\R^d)$,
we say that $\mu$ is a {\it Fourier quasicrystal}. A measure $\mu=\sum_{\l\in\L}a_\l\d_\l$ with $a_\l\in\C$ and countable $\L$ is called {\it purely point}.

  Furthermore, it is well known that every distribution with locally finite support $\L$ has the form
 \begin{equation}\label{r0}
  f=\sum_{\l\in\L}\sum_k p_k(\l)D^k\d_\l,\quad k\in(\N\cup\{0\})^d,
  \end{equation}
 where the internal sum is finite for every $\l\in\L$, and $\d_\l$ is the unit mass at the point $\l$.
\begin{Pro}\label{P1}
i) If a distribution $f\in S^*(\R^d)$ has locally finite support, then
  \begin{equation}\label{r1}
f=\sum_{\l\in\L}\sum_{\|k\|\le K}p_k(\l)D^k\d_\l,\quad k\in(\N\cup\{0\})^d,
\end{equation}
where $K<\infty$ does not depend on $\l$ (\cite{F1}, Proposition 1),

ii) if a distribution $f\in S^*(\R^d)$ has uniformly discrete support, then $K$ does not depend on $\l$  and for some $M<\infty$ and all $k$
\begin{equation}\label{r2}
p_k(\l)=O(|\l|^M)\quad\mbox{as}\quad \l\to\infty,
\end{equation}
(\cite{P}, Proposition 1),

iii) if a distribution $f\in S^*(\R^d)$  of form \eqref{r1}  with locally finite support $\L$ and locally finite spectrum $\G$  has the property
\begin{equation}\label{r3}
\sum_{|\l|\le r}\sum_{\|k\|=0}^K|p_k(\l)|=O(r^{d+M})\phantom{XXXXXX}(r\to\infty)
\end{equation}
with $M\ge0$, then
\begin{equation}\label{m}
\hat f=\sum_{\g\in\G}\sum_{\|m\|\le M} q_m(\g)D^m\d_\g,\quad m\in(\N\cup\{0\})^d,
\end{equation}
If, in addition, $M$ is an integer, then for $\|m\|=M$
\begin{equation}\label{r}
q_m(\g)=O(|\g|^K)\quad\mbox{as}\quad \g\to\infty.
\end{equation}
 with the same $K$ as in \eqref{r1}. If $M$ is a positive real number, and $\G$ is uniformly discrete, then \eqref{r} is valid for all $m$ (\cite{F1}, Proposition 3),

In particular, if $f$ is a measure with uniformly bounded masses, support of bounded density, and locally bounded spectrum, then $K=M=0$ and $\hat f$ is a measure with uniformly bounded masses as well.
\end{Pro}
{\bf Remark 1}. Analysis of the proof of the Proposition shows that if  we replace $O(r^{d+M})$ by $o(r^{d+M})$ in \eqref{r3},
then we may replace the inequality $\|m\|\le M$  by $\|m\|<M$ in \eqref{m}.
\medskip

{\it A lattice} is a discrete (locally finite) subgroup of $\R^d$. If $A$ is a lattice or a coset of some lattice in $\R^d$, then its rank $r(A)$ is the dimension of the smallest
translated subspace of $\R^d$ that contains $A$.  Every lattice $L$ of rank $k$ has the form $T\Z^k$, where $T:\,\Z^k\to\R^d$ is a linear operator of rank $k$. For $k=d$ we get a  full-rank lattice.
In this case the lattice
$$
L^*=\{y\in\R^d: <\l,y>\in\Z\quad \forall \l\in L\}
$$
 is called the conjugate  lattice.  It follows from Poisson's formula
 $$
 \sum_{n\in\Z^d}f(n)=\sum_{n\in\Z^d}\hat f(n),\qquad f\in S(\R^d),
 $$
   that for a full-rank lattice $L=T\Z^d$ we have
 \begin{equation}\label{a1}
      \widehat{\sum_{\l\in L} \d_\l}=|\det T|^{-1}\sum_{\l\in L^*} \d_\l.
 \end{equation}

     \section{Introduction, one-dimensional case}\label{S2}

We begin with the following result of N.Lev and A.Olevskii \cite{LO1}:
\begin{Th}\label{T1}
Let $\mu$ be a crystalline measure on $\R$ with uniformly discrete support and spectrum. Then $\supp\mu$ is a subset of a finite union of translates of a single lattice $L\subset\R$
(i.e.,of arithmetic progressions with the same difference). Moreover, $\mu$ has the form
$$
   \mu=\sum_{j=1}^N \sum_{\l\in L+\tau_j}P_j(\l)\d_\l
$$
for certain reals $\tau_j$ and trigonometric polynomials $P_j(\l)$ ($1\le j\le N$).
\end{Th}
Theorem \ref{T1} remains valid under a weaker assumption that $\supp\mu$ is a relatively dense set of bounded density (not assumed to be uniformly discrete) \cite{LO2}. However there exist
 examples of crystalline measures on $\R$, whose supports  are not contained in any finite union of translates of a lattice (see for example \cite{LO3}, \cite{M2}, \cite{KS}).

In article \cite{LR} Theorem \ref{T1} was extended to temperate distributions on the line:
\begin{Th}\label{T2}
Let $f$ be a temperate distribution on $\R$ such that $\L=\supp f$ and
$\G=\supp\hat f$ are uniformly discrete sets. Then $f$ can be represented in the form
$$
f=\sum_{\tau,\omega, l, p}c(\tau,\omega, l, p)\sum_{\l\in L}\l^le^{2\pi i\l\omega}\d_{\l+\tau}^{(p)},
$$
where $L$ is a lattice on $\R$, $(\tau,\omega, l, p)$ goes through a finite set of quadruples such that $\tau,\omega$ are
real numbers, $l, p$ are nonnegative integers, and $c(\tau,\omega, l, p)$ are complex numbers.

The result is still correct if the support $\L$ has bounded density and the coefficients $p_k(\l)$ in \eqref{r0} satisfy \eqref{r2}, while the spectrum $\G$ is
uniformly discrete.
\end{Th}

\section{Introduction, multivariate case}\label{S3}

Various analogs of Theorem \ref{T2} for temperate distributions on $\R^d$ were obtained in \cite{P} under conditions of locally finite set of differences $\L-\L$ and $\G-\G$ and in \cite{F1}
under conditions of locally finite  $\L-\L$, not too fast approaching points from $\G$, and uniformly separated from zero and infinity coefficients $p_k(\l)$.
\smallskip

In \cite{LO1}, \cite{LO2}  N.Lev and A.Olevskii  proved the following result:
\begin{Th}\label{T3}
Let $\mu$ be a {\bf positive}  measure on $\R^d$ such that $\L=\supp\mu$ and
$\G=\supp\hat\mu$ are uniformly discrete sets. Then $\L$ is contained in a finite union of translates of a lattice of rank $d$.  The same is valid if $\L$ is locally finite and of bounded density not assumed to be uniformly discrete.
\end{Th}
It was proved in \cite{F2} that there is a signed measure on $\R^2$ such that its support and  spectrum
are both uniformly discrete and simultaneously are unions of two incommensurable full-rank lattices. Therefore neither support, nor spectrum can be finite unions of translations of a {\bf single} lattice.

But there are results of a different type
 for  measures in $\R^d$ with discrete support, in which a {\bf finite number of  lattices} are already involved (\cite{M3},  \cite{KL}, \cite{K1}, \cite{Co}).
 In fact, the following result was proved:
\begin{Th}\label{T4}
 Let  $\mu$ be a measure on $\R^d$ with uniformly discrete support $\L$. If complex masses $\mu(\{\l\})$ at  points $\l\in\L$  take  values only from a finite set $F\subset\C$,
 and the measure $\hat\mu$ be purely point and satisfies the condition
 \begin{equation}\label{mu}
|\hat\mu|(B(0,r))=O(r^d) \quad (r\to\infty),
\end{equation}
 then $\L$  is a finite union of translations of several, possibly incommensurable, full-rank lattices.
\end{Th}
In the paper \cite{LR}, the following problem was posed:
 \smallskip

{\bf Problem 2.} Let $\mu$ be a measure on $\R^d$ with discrete support $\L$ and its Fourier transform in the sense of distributions $\hat\mu$ is a measure with discrete support as well.
Does it follow that $\L$ can be covered by a finite union of translates of several lattices?
\smallskip

Remark that the periodic structure of a crystalline measure in the multidimensional case  follows under extra assumptions on  masses (\cite{F4},\cite{F5}):
\begin{Th}\label{T5}
 Let $\mu$ be a measure on $\R^d$ with uniformly discrete support $\L$ such that $\inf_{\l\in\L}|\mu(\{\l\})|>0$ and the measure $\hat\mu$  is purely point and satisfy \eqref{mu}.
 Then  $\L$  is a finite union of translates of several disjoint full-rank lattices.

  Moreover,  there exist an integer $J$, lattices $L_1,\dots,L_J$ in $\R^d$ of rank $d$ (some of them may coincide), points $\l_1,\dots,\l_J\in\L$,
  a bounded set $\{\a_s^j:\,s\in\N,\,1\le j\le J\}\subset\R^d$, and functions
  $$
  F_j(y)=\sum_s b_s^j e^{2\pi i\la y,\a_s^j\ra}\quad\mbox{with}\quad \sum_s|b_s^j|<\infty,\quad j=1,\dots,J,
  $$
   such that
 $$
 \mu=\sum_{j=1}^J F_j(\l)\sum_{\l\in L_j+\l_j}\d_\l.
 $$
 \end{Th}
 The proofs of theorems \ref{T4} and \ref{T5} are based on Cohen's Idempotent Theorem (see, e.g., \cite{Ru};
 the first using of the Idempotent Theorem to prove structure of tilings was in \cite {LM}):
\begin{Th}\label{T6}
Let $G$ be a locally compact Abelian group and $\hat G$ its dual group. If $\nu$ is a finite Borel measure on $G$ and is such that
its Fourier transform $\hat\nu(\l)$ takes only  values $0$ and $1$, then the set $\{\l:\,\hat\nu(\l)=1\}$ belongs to the coset ring of $\hat G$.
\end{Th}
Recall that the coset ring of an Abelian topological group $G$ is the smallest collection  of subsets of $G$ that is closed under complement, finite unions, and finite intersections,
  which contains all cosets of all open subgroups of $G$. In particular, if $G=\R^d_{discr}$, i.e., the space $\R^d$ with respect to the discrete topology, then the coset ring contains all cosets of all subgroups of $G$.
 \begin{Th}\label{T7}[M.Kolountzakis \cite{K1}, Theorem 3]
  All  elements of the coset ring for $\R^d_{discr}$, which are discrete in the Euclidean topology of $\R^d$, are finite unions of sets of the type $A\setminus(\cup_{j=1}^J B_j)$,
where  $A,\ B_j$ are discrete cosets in the Euclidean topology.
\end{Th}

\section{Results}\label{S4}

Here we get the following extension of Theorems \ref{T3} and \ref{T5} to temperate distributions:
\begin{Th}\label{T8}
Suppose that a temperate distribution $f$ has   locally finite support $\L$ of  bounded density, a uniformly discrete spectrum $\G$, and all coefficients $p_k(\l)$ in \eqref{r0} are
uniformly bounded and non-negative. Then $\L$ is contained in a finite union of translates of several full-rank lattices.

The non-negativity of  coefficients $p_k(\l)$  can be replaced by the following: for each $k\in(\N\cup\{0\})^d$ there are $\xi_k\in\R$ such that $\Arg\,p_k(\l)=\xi_k$
for all $p_k(\l)\neq0,\,\l\in\L$.
\end{Th}
\begin{Th}\label{T9}
Suppose that a temperate distribution $f$ of form \eqref{r0} has both uniformly discrete support $\L$ and spectrum $\G$.
If there are constants $c,\,C$ such that for  coefficients in \eqref{r0}  the inequalities  hold
\begin{equation}\label{c3}
 0<c\le\sum_k |p_k(\l)|\le C<\infty,\qquad \forall\ \l\in\L,
 \end{equation}
then $\L$ is  a  finite union of translations of several full-rank lattices.
More accurately, there exist integers $J,\,K$, lattices $L_1,\dots,L_J$ in $\R^d$ of rank $d$ (some of them may coincide), points $\l_1,\dots,\l_J\in\L$, and
a bounded set of exponents $\b _s=\b_s(j,k)\in\R^d$  such that
$$
f=\sum_{j=1}^J\sum_{\|k\|\le K}\sum_{\l\in L_j+\l_j} F_{k,j}(\l)D^k\d_\l,
$$
where
$$
F_{k,j}(x)=\sum\limits_s b_s(k,j) e^{2\pi i\la x,\b_s\ra}\quad\mbox{with}\quad\sum\limits_s|b_s(k,j)|<\infty.
$$
\end{Th}
{\bf Remark 2}.  Proposition \ref{P1} i) implies that $f$ has form \eqref{r1} with $K<\infty$, therefore we may replace the sum $\sum_k|p_k(\l)|$ by $\sup_k|p_k(\l)|$ in \eqref{c3}.
Also, we can replace the right-hand bound in \eqref{c3} by
$$
p_k(\l)=o(|\l|)\quad (\l\to\infty)\qquad\forall k,
$$
because in this case we get again $p_k(\l)=O(1)$ (see Remark 3 in Section \ref{S6}).

Nevertheless,  some upper and lower bounds are required.

Let $A=\{n\in\Z^2:\,n_1\neq0\}$. It follows from \eqref{a0} and \eqref{a1} that the Fourier transform of the measure with unbounded coefficients
 $$
   \sum_{n\in A} n_1\d_n=\sum_{x\in\Z^2}x_1\d_x
  $$
  is the distribution
$$
   \frac{i}{2\pi}\sum_{\l\in\Z^2}\frac{\partial}{\partial\l_1}\d_\l.
$$
Also, the Fourier transform of the measure whose masses do not separate from zero
$$
   \sum_{n\in A} \sin\left(\frac{2\pi n_1}{\sqrt{5}}\right)\d_n=\frac{1}{2i}\sum_{x\in\Z^2}\left(e^{2\pi ix_1/\sqrt{5}}-e^{-2\pi ix_1/\sqrt{5}}\right)\d_x
 $$
 is equal to
 $$
   \frac{1}{2i}\sum_{y\in\Z^2}\d_{y+\a}-\frac{1}{2i}\sum_{y\in\Z^2}\d_{y-\a},\qquad \a=(1/\sqrt{5},0),
$$
hence in both cases the Fourier transforms have uniformly discrete supports. But $A$  is not  a finite union of cosets of several full-rank lattices, because in the opposite case the set $\Z\setminus\{0\}$,
which is the projection of $A$ on $\R_{x_1}$, would be  a finite union of arithmetical progressions, that is impossible.
\smallskip

But for some distributions  with unbounded coefficients their supports are {\bf subsets} of a finite union of translations of full-rank lattices:
\begin{Th}\label{T10}
Let $f\in S^*(\R^d)$ of form \eqref{r0} have both uniformly discrete support $\L$ and spectrum $\G$.
If there are integers $h(k)\in\N\cup\{0\}$  and constants $c,\,C$ such that for coefficients in \eqref{r0}  the inequalities  hold
\begin{equation}\label{c0}
 0<c\le\max_k|p_k(\l)|(1+|\l|)^{-h(k)}\le C<\infty,\qquad \forall\ \l\in\L,
 \end{equation}
then $\L$ is a subset of a  finite union of translations of several full-rank lattices that can be incommensurable.
\end{Th}

The above theorems  are based on the following  result for  temperate distributions with uniformly discrete spectrum, which is also of independent interest:
\begin{Th}\label{T11}
 Suppose $f\in S^*(\R^d)$  of  form \eqref{r0} has  locally finite support $\L$ of bounded density and uniformly discrete spectrum $\G$. Set for  $k\in(\N\cup\{0\})^d$
 \begin{equation}\label{c2}
\mu_k=\sum_{\l\in\L}p_k(\l)\d_\l.
\end{equation}

a) If $p_k(\l)$ are uniformly bounded in $k\in(\N\cup\{0\})^d$ and $\l\in\L$, then  $\hat\mu_k\in S^*(\R^d)$ are measures of the form
$$
 \hat\mu_k=\sum_{\g\in\G_k}q_k(\g)\d_\g
$$
 with uniformly bounded masses $q_k(\g)$  and uniformly discrete supports $\G_k$ such that
\begin{equation}\label{et}
\eta(\G_k)\ge\eta(\G)\qquad \forall k.
\end{equation}

b) if both $\L$ and $\G$ are uniformly discrete, then $\hat\mu_k\in S^*(\R^d)$ are distributions of the form
\begin{equation}\label{m2}
\hat\mu_k=\sum_{\|m\|\le M}\sum_{\g\in\G_k}q_{k,m}(\g)D^m\d_\g
\end{equation}
with uniformly bounded coefficients $q_{k,m}(\g)$ and uniformly discrete supports $\G_k$, which satisfy \eqref{et}
\end{Th}
Since $\mu_k$ is the inverse Fourier transform of $\hat\mu_k$, we see that $\mu_k\in S^*(\R^d)$. Also, by Proposition \ref{P1} i), $\mu_k\equiv0$ for $\|k\|$ large enough. Hence,
\begin{equation}\label{f2}
   f=\sum_k D^k\mu_k,\phantom{XXXXX} \hat f(y)=\sum_k (2\pi i)^{\|k\|}y^k\hat\mu_k,
 \end{equation}
where, as usually, $y^k=y_1^{k_1}\cdot\dots\cdot y_d^{k_d}$.

\section{Almost periodic functions and distributions}\label{S5}

We recall here  definitions and properties of almost periodic functions and distributions that will be used in what follows. A more complete exposition of these issues is available in \cite{C},
\cite{A}, \cite{M1}, \cite{M2}, \cite{R}.

\begin{De}\label{D1} A continuous function $g$ on $\R^d$ is  almost periodic if for any  $\e>0$ the set of its $\e$-almost periods
 $$
  \{\tau\in\R^d:\,\sup_{t\in\R^d}|g(t+\tau)-g(t)|<\e\}
 $$
is a relatively dense set in $\R^d$.
\end{De}
An equivalent definition follows:
\begin{De}\label{D2} A continuous function $g$ on $\R^d$ is  almost periodic if for any sequence $\{t_n\}\subset\R^d$ there is a subsequence  $\{t'_n\}$ such that
 the sequence of functions $g(t+t'_n)$ converge uniformly in $t\in\R^d$.
\end{De}
Using an appropriate definition, one can prove various properties of almost periodic functions.
\begin{itemize}
\item almost periodic functions are  bounded and uniformly continuous on $\R^d$,
\item the class of almost periodic functions is closed with respect to taking absolute values and linear combinations of a finite family of functions,
 \item a limit of a uniformly convergent sequence  of almost periodic functions is also almost periodic,
 \item any finite family of almost periodic functions has a relatively dense set of common $\e$-almost periods,
\item  for any almost periodic function $g(x)$ on $\R^d$ the function $h(t)=g(tx)$ is almost periodic in $t\in\R$ for any fixed $x\in\R^d$; in particular,
$g(t_1,\dots,t_d)$ is almost periodic in each variable $t_j\in\R,\ j=1,\dots,d$, if the other variables are held fixed.
 \end{itemize}
Also, we will use the following definition
\begin{De} A distribution $g$ is  almost periodic if the function $(g(y),\p(t-y))$
is almost periodic in $t\in\R^d$ for each $C^\infty$-function $\p$ on $\R^d$ with compact support.
A measure $\mu$ is almost periodic if it is almost periodic as a distribution from $S^*(\R^d)$.
 \end{De}
Clearly, every linear combination of almost periodic distributions is almost periodic, and each almost periodic distribution  has a relatively dense support.

 Note that the usual definition of almost periodicity for measures (instead of $\p\in C^\infty$, we consider  continuous $\p$ with compact support)
 differs from the one given above. However these definitions coincide for nonnegative measures or measures with uniformly discrete support (see  \cite{A}, \cite{M1}, \cite{F2}).
\begin{Pro}\label{P2}
Let $\mu=\sum_{\l\in\L} p(\l)\d_\l,\  p(\l)\in\C$, be an almost periodic measure with uniformly discrete support $\L\subset\R^d$, then

i) the masses $p(\l)$ are uniformly bounded,

ii)  the measure $|\mu|=\sum_{\l\in\L}|p(\l)|\d_\l$ is almost periodic,

iii) if $\inf_{\l\in\L}|p(\l)|\ge\a>0$, then the measure $\d_\L:=\sum_{\l\in\L}\d_\l$ is almost periodic as well,

iv) for any $\e>0$ and $\l\in\R^d$ there is a relatively dense set $T$ such that $|\mu(\l')-\mu(\l)|<\e$ for all $\l'\in T$,

v) if $\mu_n,\,n=1,\dots,N$ be almost periodic measures  with uniformly discrete supports, then for any vector $\l\in\R^d$ and numbers $\e>0,\ \rho>0$ there is a relatively dense set $T$ such that
for every $\l'\in T$ and some $\l^{(n)}\in B(\l',\rho)$
\begin{equation}\label{m1}
|\mu_n(\l^{(n)})-\mu_n(\l)|<\e,\quad n=1,\dots,N.
\end{equation}
 \end{Pro}

  The condition on $\L$ to be uniformly discrete is essential (see\cite{FK}).
\smallskip

 {\bf Proof of Proposition \ref{P2}}. i) Let $\p$ be $C^\infty$ function with support in $B(0,\eta(\L)/2)$ such that $\p(0)=1$. The function $(\mu(y),\p(t-y))$ is almost periodic,
 hence it is uniformly bounded, and the numbers $p(\l)=(\mu(y),\p(\l-y))$ are uniformly bounded as well.

 ii) Every $C^\infty$-function  with compact support is a finite linear combination of shifts of nonnegative $C^\infty$-functions with supports in balls of radius $\eta/2$
 \footnote{Using Theorem on Partition of Unity, it suffices to prove this for the case of a real-valued  $\p\in C^\infty$ with support in the ball $B(x,\eta(\L)/4)$.
 Then we have $\p=\psi-(\psi-\p)$, where $\psi$ is an arbitrary $C^\infty$ nonnegative function with support in $B(x,\eta(\L)/2)$ such that $\p\le \psi$.}.
  Consequently we can check the almost periodicity  using only  such functions.

 Let $\p$ be one of them and $\supp\p\subset B(x,\eta/2)$. For each $t,\tau\in\R^d$ we have
 $$
       |(|\mu|(y),\p(t-y))-(|\mu|(y),\p(t+\tau-y))|=\left|\sum_{\l\in\L}\p(t-\l)|p(\l)|-\sum_{\l'\in\L}\p(t+\tau-\l')|p(\l')|\right|.
 $$
 Evidently, here each sum consists of at most one nonzero term. Therefore, for some $\l, \l'\in\L$ that depends on $t,\tau$ we get
 $$
       |(|\mu|(y),\p(t-y))-(|\mu|(y),\p(t+\tau-y))|=|\p(t-\l)|p(\l)|-\p(t+\tau-\l')|p(\l')||
 $$
  $$
   \le  |\p(t-\l)p(\l)-\p(t+\tau-\l')p(\l')|=|(\mu(y),\p(t-y))-(\mu(y),\p(t+\tau-y))|.
  $$
 Hence every $\e$-period $\tau$ of the function $(\mu(y),\p(t-y))$ is an $\e$-period of the function $(|\mu|(y),\p(t-y))$.

 iii) Pick $\e>0$ and $\b<\min\{\a,\eta(\L)/2\}$ such that $|\p(x)-\p(x')|<\e$ for $|x-x'|<\b$. Let $\psi$ be $C^\infty$-function such that
$\supp\psi(x)\subset B(0,\b)$ and $\psi(0)=1$.  Let  $\tau$ be some $\b$-almost period of the function $(\mu(y),\psi(t-y))$, and let $t\in B(\l,\b)$ for $\l\in\L$,
Since $\b<\eta(\L)/2$, we get for at most one $\l'\in\L$ that depends on $t+\tau$
   $$
      \b>\left|\sum_{\l\in\L}\psi(t-\l)p(\l)-\sum_{\l'\in\L}\psi(t+\tau-\l')p(\l')\right|= |p(\l)\psi(t-\l)-p(\l')\psi(t+\tau-\l')|.
   $$
For $t=\l$ we get
    $$
    |p(\l)-p(\l')\psi(\l+\tau-\l')|<\b.
    $$
 The inequality $\b<\a\le|p(\l)|$ implies that $\psi(\l+\tau-\l')\neq0$. Hence, $|\l+\tau-\l'|<\b$. Therefore, $|\p(t-\l)-\p(t+\tau-\l')|<\e$ and
  $$
       \left|\sum_{\l\in\L}\p(t-\l)-\sum_{\l'\in\L}\p(t+\tau-\l')\right|=|\p(t-\l)-\p(t+\tau-\l')|<\e.
  $$
This inequality is valid for all $\b$-almost periods $\tau$ from a relatively dense set. Thus the function $(\d_\L(y),\p(t-y))$ is almost periodic.

 iv) It is enough to consider the case $\e\le|\mu(\l)|$. Let $\rho<\eta(\supp\mu)/2$, and let $\p$ be $C^\infty$ function on $\R^d$ such that
 $$
   0\le\p(t)\le1,\quad \p(0)=1,\quad \p(t)=\p(-t),\quad \supp\p\subset B(0,\rho).
 $$
We have $\mu(\l)=(\p(\l-x),\mu(x))$.  If $\tau$ be an $\e/3$-almost period of the almost periodic function $(\p(t-x),\mu(x))$, we get
$$
|(\p(\l+\tau-x),\mu(x))-\mu(\l)|<\e/3.
$$
Therefore, there is a single point $\l'\in B(\l+\tau,\rho)$ such that
$$
|\mu(\l')\p(\l+\tau-\l')-\mu(\l)|<\e/3.
$$
We get $\l'\in\supp\mu$ and $|\mu(\l')|>2\e/3$. Since $-\tau$ is an $\e/3$-almost period as well, we see that for some $\l''\in B(\l'-\tau,\rho)$
$$
|\mu(\l'')\p(\l'-\tau-\l'')-\mu(\l')|<\e/3.
$$
We get $\l''\in\supp\mu$ and  $|\l''-\l|<2\rho$, hence, $\l''=\l$. Then we have for $t=\l+\tau-\l'$
$$
 |\mu(\l)-\mu(\l')|\le|\mu(\l)-\p(t)\mu(\l')|+|\mu(\l')-\p(t)\mu(\l')|\le|\mu(\l)-\p(t)\mu(\l')|+(1-\p^2(t))|\mu(\l')|
$$
$$
\le|\mu(\l)-\p(t)\mu(\l')|+|\mu(\l')-\p(-t)\mu(\l)|+|\p(t)\mu(\l)-\p^2(t)\mu(\l')|<\e.
$$
Recall that almost periods form a relatively dense set.

 v) The proof is very close to the previous one. We take $\rho<\min_n \eta(\mu_n)/2$,  the same function $\p$, and $\l'=\l+\tau$,
 where $\tau$ is a common $\e/3$-almost period of the functions $(\p(\l-x),\mu_n(x))$. Then, as $\l^{(n)}$ we take points satisfying  the conditions
$$
|\mu_n(\l^{(n)})\p(\l'-\l^{(n)})-\mu_n(\l)|<\e/3.
$$
Further, arguing as above, we obtain \eqref{m1}. \bs
\medskip

Typical examples of almost periodic functions on $\R^d$ are  sums of the form
\begin{equation}\label{W}
   f(t)=\sum_n a_n e^{2\pi i\la t,s_n\ra},\quad a_n\in\C,\quad s_n\in\R^d,\quad \sum_n|a_n|<\infty.
\end{equation}
It is not hard to prove that $\hat f=\sum_n a_n\d_{s_n}$.

Denote by $W$ the class of functions admitting  representation \eqref{W}. It is easy to check that a product of two functions from $W$ belongs to $W$ as well.
Also, in Section \ref{S7} we will use the following local version of the classical Wiener-Levi Theorem:
\begin{Pro}\label{P3}[\cite{F4}]
 Let  $K\subset\C$ be an arbitrary compact set,  $h(z)$ be a holomorphic function on a neighborhood of $K$, and $f\in W$. Then there is a function $g\in W$ such that if $f(x)\in K$ then $h(f(x))=g(x)$.
 \end{Pro}
Next, the Bohr compactification $\hR$ of $\R^d$ is a compact group with the dual $\R^d_{discr}$, and  $\R^d$ is a dense subset of $\hR$ with respect to the topology on $\hR$. Moreover,
restrictions to $\R^d$ of continuous functions on $\hR$ are almost periodic functions on $\R^d$ (see, e.g.,\cite{Ru}).

\section{Representation of distributions with discrete support and spectrum}\label{S6}

First we prove the following lemma:
\begin{Le}\label{L1}
Let $g_0(y), g_1(y),\dots, g_N(y)$ be temperate distributions on $\R^d$ such that for every $C^\infty$-function $\p(y)$ with compact support the functions
$$
(g_n(y),\p(t-y)),\quad t=(t_1,t_2,\dots,t_d),\quad n=0,\dots,N,
$$
are almost periodic in $t_1\in\R$ for every fixed $(t_2,\dots,t_d)\in\R^{d-1}$. If the distribution
$$
F=\sum_{n=0}^N y_1^n g_n(y),\quad y=(y_1,\dots,y_d),
$$
 has  a uniformly discrete support $T$, then the set $\cup_n\supp g_n$ is uniformly discrete and $\eta(\cup_n\supp g_n)\ge\eta(T)$.
\end{Le}

{\bf Proof}. Note that for every $C^\infty$-function $\p(y)$ with compact support we have
\begin{equation}\label{s1}
   (F(y),\p(t-y))=\sum_{n=0}^\N(g_n(y),y_1^n\p(t-y))=\sum_{m=0}^\N t_1^m \sum_{n=m}^N \P_{n,m}(t),
\end{equation}
where
$$
\P_{n,m}(t)=\binom{n}{m}(g_n(y),(y_1-t_1)^{n-m}\p(t-y)),\quad n\ge m.
$$
We will prove that for every $n,m\le N$ and every $y\in\supp g_n,\,\tilde y\in\supp g_m$ we get $|y-\tilde y|\ge\eta(T)$.

Consider the case $n=m=N$. Assume the converse. Then there are points $y',y''\in\supp g_N$ such that $0<|y'-y''|<\eta(T)$.
Pick
$$
\a<(1/2)\min\{|y'-y''|,\,(\eta(T)-|y'-y''|)\}
$$
 and $C^\infty$-function $\p$ with $\supp\p\subset B(0,\a)$ such that
 $$
\P_{N,N}(y')=(g_N(y),\p(y'-y))\neq0,\qquad \P_{N,N}(y'')=(g_N(y),\p(y''-y))\neq0.
 $$
 Take $\e>0$ such that $|\P_{N,N}(y')|>2\e$, $|\P_{N,N}(y'')|>2\e$. If $\tau$ is a common $\e$-almost period of the functions
 $$
    \P_{N,N}(y'+se_1),\quad \P_{N,N}(y''+se_1),\quad\mbox{where}\quad e_1=(1,0,0\dots 0),\quad s\in\R,
 $$
 then $|\P_{N,N}(y'+\tau e_1)|>\e$, $|\P_{N,N}(y''+\tau e_1)|>\e$. We have
$$
(y'_1+\tau)^{-N}(F(y),\p(y'+\tau e_1-y))=\P_{N,N}(y'+\tau e_1)+\sum_{m=0}^{N-1} (y'_1+\tau)^{m-N} \sum_{n=m}^{N-1}\P_{n,m}(y'+\tau e_1).
$$
Note that all  functions $\P_{n,m}(y'+se_1),\,0\le n,m\le N,$ are almost periodic in $s$, hence they are uniformly bounded on $\R$. Therefore if $\tau$ is large enough then
$$
(y'_1+\tau)^{-N}(F(y),\p(y'+\tau e_1-y))\neq0.
$$
Hence there exists a point $t'\in B(y'+\tau e_1,\a)\cap T$. For the same reason, there exists a point $t''\in B(y''+\tau e_1,\a)\cap T$ with the same $\tau$. We have
\begin{equation}\label{s2}
 |t'-t''|\le|t'-y'-\tau e_1|+|t''-y''-\tau e_1|+|y'-y''|<2\a+|y'-y''|<\eta(T).
\end{equation}
But this inequality is impossible.

Now suppose that our assumption is proved for all $n,\,m$ such that $n+m>M$ with $0\le M<2N$; then we prove it for $p, q\ge0$ such that $p+q=M$.

Assume the converse. There are points $y'\in\supp g_p,\ y''\in\supp g_q$ such that $0<|y'-y''|<\eta(T)$.  Pick
$$
\a<(1/2)\min\{|y'-y''|,\,(\eta(T)-|y'-y''|)\}
$$
and $C^\infty$-function $\p$ with $\supp\p\subset B(0,\a)$ such that
 $$
\P_{p,p}(y')=(g_p(y),\p(y'-y))\neq0,\qquad \P_{q,q}(y'')=(g_q(y),\p(y''-y))\neq0.
 $$
Take $\e>0$ such that $|\P_{p,p}(y')|>2\e$ and $|\P_{q,q}(y'')|>2\e$. Obviously, for a common $\e$-almost period $\tau$ of the functions
$\P_{p,p}(y'+se_1)$ and $\P_{q,q}(y''+se_1)$
  we get
  $$
  |\P_{p,p}(y'+\tau e_1)|>\e,\qquad |\P_{q,q}(y''+\tau e_1)|>\e.
  $$
   If there exists $n>p$ such that at least one of the numbers $\P_{n,j}(y'+\tau e_1),\,j=0,\dots,n$ does not vanish,
 then there is a point $t'\in B(y'+\tau e_1,\a)\cap\supp g_n$. Since $\P_{q,q}(y''+\tau e_1)\neq0$, we see that there is a point $t''\in B(y''+\tau e_1,\a)\cap\supp g_q$. By \eqref{s2}, $|t'-t''|<\eta(T)$, hence
 this inequality contradicts to our assumption $\eta(\supp g_n\cup\supp g_q)\ge\eta(T)$ for $n+q>M$. In the same way, we obtain a contradiction in the case
 when  at least one of the numbers $\P_{n,j}(y''+\tau e_1)$ for $n>q,\,j=0,\dots,n$,  does not vanish.

Therefore, for $t=y'+\tau e_1$ equality \eqref{s1} implies
$$
(y'_1+\tau)^{-p}(F(y),\p(y'+\tau e_1-y))=\P_{p,p}(y'+\tau e_1)+\sum_{m=0}^{p-1} (y'_1+\tau)^{m-p} \sum_{n=m}^{p-1}\P_{n,m}(y'+\tau e_1),
$$
and for $t=y''+\tau e_1$  implies
$$
(y''_1+\tau)^{-q}(F(y),\p(y''+\tau e_1-y))=\P_{q,q}(y''+\tau e_1)+\sum_{m=0}^{q-1} (y''_1+\tau )^{m-q} \sum_{n=m}^{q-1}\P_{n,m}(y''+\tau e_1).
$$
 If $\tau$ is large enough, then we get
$$
(y'_1+\tau)^{-p}(F(y),\p(y'+\tau e_1-y))\neq0,\qquad (y''_1+\tau)^{-q}(F(y),\p(y''+\tau e_1-y))\neq0,
$$
hence there exists $t'\in B(y'+\tau e_1,\a)\cap T$ and $t''\in B(y''+\tau e_1,\a)\cap T$ with the same $\tau$. By \eqref{s2}, we obtain a contradiction. Thus our assumption is valid for all $n,\,m\le N$. \bs

\medskip

{\bf Proof of Theorem \ref{T11}}. It follows from Proposition \ref{P1} i) that there is $K\in\N\cup\{0\}$ such that $\mu_k\equiv0$ for $\|k\|>K$.

In the case a), for any $\p\in S(\R^d)$ and $k$ we have
\begin{equation}\label{c}
(\hat\mu_k(y),\p(t-y))=(\mu_k(x),\check\p(x)e^{-2\pi i\la x,t\ra})=\sum_{\l\in\L}p_k(\l)\check\p(\l)e^{-2\pi i\la\l,t\ra}.
\end{equation}
Since $\check\p\in S(\R^d)$, we get $\check\p(x)=o(|x|^{-d-1})$. Hence,
$$
\sum_{\l\in\L}|p_k(\l)||\check\p(\l)|\le C_1+C_2\sum_{\l\in\L,|\l|>1}|\l|^{-d-1}=C_1+C_2\int_1^\infty s^{-d-1}dn(s),
$$
where $n(s)=\#(\L\cap B(0,s))$. The set $\L$ is of bounded density, therefore, $n(s)=O(s^d)$ and the integral
$$
 \int_1^\infty s^{-d-1}n(ds)=-n(1)+(d+1)\int_1^\infty\frac{n(s)ds}{s^{d+2}}
$$
is finite. Consequently the sum in \eqref{c} is absolutely convergent, and the function $(\hat\mu_k(y),\p(t-y))$ is almost periodic in $t\in\R^d$.

In the case b), by Proposition \ref{P1} ii), we get
\begin{equation}\label{p}
p_k(\l)=O(|\l|^M) \quad\mbox{as}\quad \l\to\infty.
\end{equation}
 Since $\check\p(x)=o(|x|^{-M-d-1})$ for each $H$, we can repeat the previous arguments and obtain
that $\hat\mu_k$ are almost periodic distributions.

Further, by \eqref{f2}, we have
$$
  \hat f=\sum_{k_1=0}^K (2\pi i)^{k_1}y_1^{k_1}g_{k_1}, \quad\mbox{where}\quad g_{k_1}(y)=\sum_{k_2+\dots+k_d\le K-k_1}(2\pi i)^{k_2+\dots+k_d}y_2^{k_2}\dots y_d^{k_d}\hat\mu_{k_1,\dots,k_d},
$$
and for any $\p\in\C^\infty$ with compact support
$$
(g_{k_1}(y),\p(t-y))=\sum_{k_2+\dots+k_d\le K-k_1}(2\pi i)^{k_2+\dots+k_d}(\hat\mu_{k_1,\dots,k_d},y_2^{k_2}\dots y_d^{k_d}\p(t-y)).
$$
Note that each term of the last sum can be rewritten as
$$
  \sum_{m_2\le k_2,\dots,m_d\le k_d}c_{m,k}t_2^{k_2-m_2}\dots t_d^{k_d-m_d}\left[(\hat\mu_k,\,(t_2-y_2)^{m_2}\dots (t_d-y_d)^{m_d}\p(t-y))\right]
$$
with some constants $c_{m,k}$. Since $\mu_k$ are almost periodic distributions, we get that the expressions in square brackets are almost periodic functions in $t\in\R^d$, and hence in $t_1\in\R$.
Therefore the functions $(g_{k_1}(y),\p(t-y))$ are almost periodic in $t_1\in\R$ for any fixed $(t_2,\dots,t_d)\in\R^{d-1}$. Applying  Lemma \ref{L1} to distributions $g_{k_1},\,k_1=0,\dots,K$,
we get  they have uniformly discrete supports and $\eta(\supp g_{k_1})\ge\eta(\G)$.

 For a fixed $k_1$ we have
 $$
 g_{k_1}=\sum_{k_2=0}^{K-k_1}(2\pi i)^{k_2}y_2^{k_2} g_{k_1,k_2},
  $$
where
$$
 g_{k_1,k_2}(y)=\sum_{k_3+\dots+k_d\le K-k_1-k_2}(2\pi i)^{k_3+\dots+k_d}y_3^{k_3}\dots y_d^{k_d}\hat\mu_{k_1,\dots,k_d}.
$$
The functions $(g_{k_1,k_2}(y),\p(t-y))$ are almost periodic in $t_2\in\R$ for any fixed $(t_1,t_3\dots,t_d)\in\R^{d-1}$. Applying  Lemma \ref{L1} to distributions $g_{k_1,k_2},\,k_2=0,\dots,N-k_1$
with respect to the variable $y_2$, we get that these distributions have uniformly discrete supports and $\eta(\supp g_{k_1,k_2})\ge\eta(\G)$.

After a finite number of steps we get that  the support $\G_k$ of the distributions $\hat\mu_k$ for all $k\in(\N\cup\{0\})^d$ are
uniformly discrete and satisfy \eqref{et}. If the masses $p_k(\l)$ are uniformly bounded, then Proposition \ref{P1} iii), yields that $\hat\mu_k$ are measures with uniformly bounded masses.
If masses $p_k(\l)$ satisfy \eqref{p}, we have
$$
  \sum_{|\l|\le r}|p_k(\l)|\le C_1+C_2r^M n(r)=O(r^{M+d}) \quad\mbox{as}\quad r\to\infty.
$$
Applying Proposition \ref{P1} iii) with $K=0$,  we obtain that the coefficients $q_{k,j}(\g)$ in \eqref{m2} are uniformly bounded. \bs
\medskip

{\bf Remark 3}. Let $\L$ be uniformly discrete. If we replace the condition  $p_k(\l)=O(1)$ by $p_k(\l)=o(|\l|)$, then we get for any $\e>0$ and appropriate $r(\e)$
\begin{equation}\label{b1}
  \sum_{|\l|\le r}|p_k(\l)|\le \sum_{|\l|\le r(\e)}|p_k(\l)|+\sum_{r(\e)<|\l|\le r}|p_k(\l)| \le C(\e)+\e r\,n(r).
\end{equation}
Hence, \eqref{b1} is $o(r^{d+1})$ as $r\to\infty$. Applying Proposition \ref{P1} iii) with $K=0,\,M=1$ and taking into account Remark 1,
we obtain that $\hat\mu_k$ (and, of course, $\check\mu_k$) are measures with uniformly bounded masses. Applying
 Proposition \ref{P1} iii) to the measures $\check\mu_k$, we obtain that the coefficients $p_k(\l)$ are uniformly bounded.

\section{Proofs of the main theorems}\label{S7}

{\bf Proof of Theorem \ref{T8}}. By Theorem \ref{T11}, the measures $\mu_k$ from \eqref{c2} have uniformly discrete spectra. Applying Theorem \ref{T3} to these measures,
we get that $\supp\mu_k$ is contained in a finite union of translates of a full-rank lattice.
 To obtain the last assertion of the theorem,  we  take the measures $e^{-i\xi}\mu_k$ instead of the measures $\mu_k$.  \bs
\medskip

{\bf Proof of Theorem \ref{T9}}.
Notice that   masses of the measures $\mu_k$ from \eqref{c2} are not assumed to be separated from zero, therefore we cannot make use of the result of Theorem \ref{T5}.

Let $\p(y)$ be nonnegative $C^\infty$-function with support in the unit ball such that \newline $\int\p(t)dt=1$. Pick $k\in(\N\cup\{0\})^d$, and set $\rho_\e=\e^d\p(\e y)\check\mu_k(y)$,
where $\check\mu_k$ is the inverse Fourier transform of the measure $\mu_k$. We have
\begin{equation}\label{j}
 \hat\rho_\e(s)=\int_{\R^d}\hat\p\left(\frac{s-x}{\e}\right)\mu_k(dx)=\left(\int_{|s-x|<1}+\int_{|s-x|\ge1}\right)\hat\p\left(\frac{s-x}{\e}\right)\mu_k(dx).
\end{equation}
Since $\hat\p\in S(\R^d)$, we get $\hat\p(y)=o(|y|^{-d-1})$ as $|y|\to\infty$ and $\hat\p((s-x)/\e)\to0$ for $s\neq x$ as $\e\to0$. Also, $\hat\p(0)=1$. Hence, by the Dominated Convergence Theorem,
$$
\lim_{\e\to0}\int_{|s-x|<1}\hat\p\left(\frac{s-x}{\e}\right)\mu_k(dx)=\begin{cases}\mu_k(\{\l\})=p_k(\l), & s=\l\in\L,\\  0, & s\not\in\L.\end{cases}
$$
Note that the masses $\mu_k(\{\l\})$ are uniformly bounded. Also,
$$
n_s(t):=\#(\L\cap B(s,t))=O(t^d)\qquad(t\to\infty).
$$
 Therefore,
$$
\left|\int\limits_{|s-x|\ge1}\hat\p\left(\frac{s-x}{\e}\right)\mu_k(dx)\right|\le C\int\limits_1^\infty(t/\e)^{-d-1}n_s(dt)\le\e^{d+1}(d+1)C\int\limits_1^\infty t^{-d-2}n_s(t)dt,
$$
and the left-hand side tends to zero as $\e\to0$.

Furthermore, by Theorem \ref{T11}, $\hat\mu_k$ has uniformly bounded masses and support of bounded density. Hence, $|\hat\mu_k|(B(0,r))=O(r^d)$ as $r\to\infty$ and the same is valid for the measures $\check\mu_k$.
Therefore,
$$
|\rho_\e|(B(0,r))\le\e^d|\check\mu_k|(B(0,1/\e)=O(1)\qquad (\e\to0).
$$
Since support of the measure $\rho_\e$ is bounded for each fixed $\e$, we see that the measures $\rho_\e$ form a bounded family of linear functionals on the space of continuous bounded functions on $\R^d$
 and, in particular, on its subspace $C(\hR)$. A closed  ball in the conjugate space $C^*(\hR)$ is a compact set in the weak-star topology. Hence there exists a measure $\hr_k$ on $\hR$ with a total variation
 $\|\hr_k\|<\infty$ such that for every $f\in C(\hR)$ there is a subsequence $\e'\to0$, for which we have $\la\rho_{\e'},f\ra\to\la\hr_k,f\ra$.
 Applying this to every character $x\in\R^d$ in the place of $f$, we obtain from \eqref{j}
 $$
 \hat\hr_k(x)=\lim_{\e'\to0}\hat\rho_{\e'}(x)=\begin{cases}p_k(\l), & x=\l\in\L,\\  0, & x\not\in\L.\end{cases}
 $$
 Note that $\hat\hr_k(x)$ is a continuous function with respect to the discrete topology on $\R^d$, and $|p_k(\l)|\le\|\hr_k\|$ for all $\l\in\R^d$.

Let $\psi$ be a $C^\infty$-function with compact support.  Since the inverse Fourier transform of the function $\psi(t-x)$ is $e^{2\pi i\la t,y\ra}\hat\psi(y)$, we get
\begin{equation}\label{f0}
(\psi\star\mu_k)(t)=\int_{\R^d}\psi(t-x)\mu_k(dx)=\int_{\R^d}e^{2\pi i\la y,t\ra}\hat\psi(y)\hat\mu_k(dy)=\sum_{\g\in\supp\hat\mu_k}\hat\mu_k(\g)\hat\psi(\g)e^{2\pi i\la\g,t\ra}.
\end{equation}
Clearly, $\hat\psi\in S(\R^d)$ and $\hat\psi(\g)=o(|\g|^{-d-1})$ as $\g\to\infty$. Also, the measure $\hat\mu_k$ has uniformly bounded masses and uniformly discrete support, therefore the series in the right-hand side is absolutely converges and $(\psi\star\mu_k)(t)\in W$. In particular, the measure $\mu_k$ is almost periodic.

 Furthermore, the function
$$
F(t)=\sum_{\|k\|\le K}(\psi\star\mu_k)(t)\overline{(\psi\star\mu_k)(t)}
$$
belongs to $W$ as well. Now suppose that $\psi$ has support in the ball $B(0,\eta(\L)/2)$ and $\psi(0)=1$. Taking into account that
\begin{equation}\label{p0}
(\psi\star\mu_k)(\l)=p_k(\l)\qquad\mbox{for}\quad \l\in\L,
\end{equation}
and using \eqref{c3}, we get
$$
 F(\l)=\sum_{\|k\|\le K} |p_k(\l)|^2\ge\a>0\qquad \forall\, \l\in\L.
$$
Using Proposition \ref{P3} with $h(z)=1/z$, we  construct the function
$$
g(t)=\sum_nc_ne^{2\pi i\la t,\tau_n\ra}\in W
$$
 with the property $g(\l)=1/F(\l)$ for $\l\in\L$.

Consider the dual pair $(\hR,\R^d_{discr})$. Set
$$
\ha=\sum_{\|k\|\le K}\hr_k(\t)\star\overline{\hr_k(-\t)},\quad \t\in\hR.
$$
 We get
 $$
 \hat\ha(\l)=F(\l)\quad \forall \l\in\L,\quad\mbox{and}\quad  \hat\ha(x)=0\quad\forall x\not\in\L.
 $$
Since  $\sum_n|c_n|<\infty$, we see that the series $\sum_nc_n\ha(y+\tau_n)$ converges with respect to the norm to a measure $\hb(y)$ such that
 $$
 \hat\hb(x)=\sum_n c_ne^{2\pi i\la x,\tau_n\ra}\hat\ha(x)=\begin{cases}1, & x\in\L,\\ 0, & \mbox{otherwise}. \end{cases}
 $$
By Theorem \ref{T6}, $\L$ belongs to the coset ring of $\R^d_{discr}$. By Theorem \ref{T7},
\begin{equation}\label{b2}
\L=\cup_{n=1}^N(A_n\setminus\cup_{s=1}^{S_n} B^s_n),
\end{equation}
where $A_n,\,B^s_n$ are cosets of discrete lattices.

Let $A,\,A'$  be two arbitrary cosets in $\R^d$ and $L,\,L'$ be the corresponding lattices. Clearly, $A\cap A'$ is a coset of $L\cap L'$. If $r(A\cap A')=d$, then the factor-groups $L/(L\cap L')$ and $L'/(L\cap L')$ are finite.
Therefore, $A\setminus A'=A\setminus(A\cap A')$ is a finite union of disjoint cosets of $L\cap L'$, and the same is valid for
$$
A\cup A'=(A\setminus(A\cap A'))\cup(A'\setminus(A\cap A'))\cup(A\cap A').
$$
We repeat this transformation for each pair of cosets with dimension $d$ of their intersection. After a finite number of steps, we get a representation similar to \eqref{b2} such that  $r(B^s_n)<d$
and $r(A_n\cap A_{n'})<d$ for $n\neq n'$.

Put $\d_A=\sum_{x\in A}\d_A$. Using the equalities $\d_{A\setminus A'}=\d_A-\d_{A\cap A'}$ and $\d_{A\cup A'}=\d_A+\d_{A'}-\d_{A\cap A'}$, we get
$$
   \d_\L=\sum_{j=1}^J\d_{D_j}+\sum_{j=1}^{J_1}\d_{E_j}-\sum_{j=1}^{J_2}\d_{F_j},
$$
where  $D_j,\,E_j,\,F_j$ are cosets such that $r(D_j)=d$ and $r(E_j)<d,\,r(F_j)<d$.

 Every coset $D_j$ is a translation of full-rank lattice,  hence $\d_{D_j}$ is $d$-periodic and $\sum_{j=1}^J\d_{D_j}$ is almost periodic.
Above we showed that all measures $\mu_k$ are almost periodic. By Proposition \ref{P2} ii), the measures $|\mu_k|$ are almost periodic, hence the measure $\sum_k|\mu_k|$
with support $\L$ is almost periodic as well.
 Using \eqref{c3} and Proposition \ref{P2} iii), we get that the measure $\d_\L$ is almost periodic. Therefore the left-hand side of the equality
 \begin{equation}\label{b}
 \sum_{j=1}^{J_1}\d_{E_j}- \sum_{j=1}^{J_2}\d_{F_j}=\d_\L-\sum_{j=1}^J\d_{D_j}
 \end{equation}
is an almost periodic measure as well. But its support  is contained in a finite union of hyperplanes and isn't relatively dense. Therefore the measure in the left-hand side of \eqref{b} is identically zero, and $\d_\L=\sum_{j=1}^J\d_{D_j}$.
Since  the measure $\d_\L$ has the unit masses at every point of $\L$, we see that the cosets $D_j$ are pairwise disjoint and  for some full-rank lattices $L_j$ and $\l_j\in\R^d$
\begin{equation}\label{m3}
\L=\cup_{j=1}^J D_j=\cup_{j=1}^J(\l_j+L_j).
\end{equation}

Recall that $\psi\star\mu_k\in W$. It follows from \eqref{p0} and \eqref{m3} that
\begin{equation}\label{m0}
 \mu_k=\sum_{j=1}^J (\psi\star\mu_k)\d_{L_j+\l_j}=\sum_{j=1}^J\sum_s a_{s,k}e^{2\pi i\la x,\g_{s,k}\ra}\d_{L_j+\l_j}
\end{equation}
for some $\g_{s,k}\in\R^d$ and $a_{s,k}\in\C$ such that $\sum_s|a_{s,k}|<\infty$.
  For every $\g\in\R^d$ there is $\b$ inside the parallelepiped generated by corresponding $L_j^*$ such that $\g-\b\in L_j^*$. Therefore,
$e^{2\pi i\la x,\g_{s,k}\ra}=e^{2\pi i\la x,\b_{s,k,j}\ra}$ for $x\in L_j$, and the set of all points $\b_{s,k,j}$ is bounded.
Therefore \eqref{m0} can be rewritten as
$$
\sum_{j=1}^J\sum_s\sum_{x\in L_j} b_s(k,j)e^{2\pi i\la x,\b_{s,k,j}\ra}\d_{x+\l_j}\quad\mbox{with}\quad b_s(k,j)= a_{s,k}e^{2\pi i\la \l_j,\g_{s,k}-\b_{s,k,j}\ra}.
$$
By \eqref{f2}, we obtain the statement of Theorem \ref{T9}.  \bs

\medskip

Here we actually proved the following statement:
\begin{Pro}\label{P4}
Let $\{\nu_n\}$ be a finite set of measures from $S^*(\R^d)$ such that for each $n$

a) $\supp\nu_n$ is uniformly discrete,

b) $\supp\hat\nu_n$ is locally finite of bounded density,

c) $\hat\nu_n$ is a measure with uniformly bounded masses.
\smallskip

\noindent Set $\hL=\cup_n\supp\nu_n$. If
$$
\inf_{\l\in\hL}\max_n|\nu_n(\l)|>0,
$$
then $\hL$ is  a  finite union of translates of several full-rank lattices.
\end{Pro}
Note that the boundedness of masses $\nu_n(\l)$ follows from \eqref{f0} with $\nu_k$ instead of $\mu_k$ and $\psi$ such that $\supp\psi\subset B(0,\eta(\supp\nu_n))$ and $\psi(0)=1$.

{\bf Remark 4}. It is not hard to check that conditions b) and c) can be replaced by

b') $\hat\nu_n$ is a purely point measure,

c')  $|\hat\nu_n|(B(0,r))=O(r^d)$ as $r\to\infty$.

\medskip
Our proof of Theorem \ref{T10} uses the following lemma
\begin{Le}\label{L2}
Let  measures
$$
\s_m=\sum_{\l\in\L_m} S_m(\l)\d_\l \quad\mbox{for}\quad m\in\N\cup\{0\},\,\|m\|\le M,
$$
with uniformly discrete supports $\L_m$ be almost periodic in $\R^d$.
If there are  constants $C$ and $h$  such that for all $\l\in\R^d$
\begin{equation}\label{p1}
p(\l)=\sum_{\|m\|\le M} \l^mS_m(\l)\quad\mbox{and}\quad |p(\l)|(1+|\l|)^{-h}<C,
\end{equation}
then $\s_m\equiv0$ for $\|m\|>h$.
 \end{Le}
Here we set $S_m(\l)=0$ for $\l\not\in\L_m$.
\smallskip

{\bf Proof of Lemma \ref{L2}}. Without loss of generality suppose  $M=\max\{\|m\|:\,\s_m\not\equiv0\}$.  Suppose the opposite $M>h$. Set
$$
A=\{m:\,\|m\|=M,\s_m\not\equiv0\},\quad N=\# A.
$$
 Let $\p$ be $C^\infty$-function such that $\p(0)=1,\ 0\le\p\le1,$ and $\supp\p\subset B(0,\rho)$ with $\rho<\eta(\supp\s_m)/2$. Set $\l\in\R^d$ such that
 $$
 \left|\sum_{m\in A}\l^mS_m(\l)\right|=\a\neq0
 $$
 (If the above sum  vanishes  for all $\l\in\R^d$, we can reduce the number $M$). Applying Proposition \ref{P2}, v) to the measures $\s_m$ with $\e<\frac{\a}{3N\max_{m\in A}(1+|\l|)^M}$, we find $R<\infty$
  such that every ball $B(s\l,R),\ s>0$, contains  a finite number of points $\{\l^{(j)}\}_{j=1}^J,\ J\le N$ with properties
 $$
     \forall m\in A\quad\exists j=\iota(m) \quad\mbox{such that}\quad |S_m(\l^{(j)})-S_m(\l)|<\e.
  $$
 Therefore, $|\l^m S_m(\l^{(j)})-\l^m S_m(\l)|<\a/3N$. Hence,
  $$
   \left|\sum_{m\in A}\l^mS_m(\l)-\sum_{j=1}^J\sum_{m:\iota(m)=j}\l^mS_m(\l^{(j)})\right|<\a/3, \quad\mbox{and}\quad\left|\sum_{j=1}^J\sum_{m:\iota(m)=j}\l^mS_m(\l^{(j)})\right|>2\a/3.
  $$
Here $J$ and $\l^{(j)}$ depend on $s$, but $R$ does not. Since $|\l^{(j)}-s\l|<R+\rho$, we see that $\l^{(j)}/s\to\l$ as $s\to\infty$, and $(\l^{(j)})^m s^{-M}\to\l^m$ for all $m\in A$. Therefore,  for $s$ large enough
 \begin{equation}\label{c1}
   \left|\sum_{j=1}^J\sum_{m:\iota(m)=j}(\l^{(j)})^ms^{-M}S_m(\l^{(j)})\right|>\a/3.
 \end{equation}
 On the other hand, $(\l^{(j)})^m s^{-M}\to0$ for $\|m\|<M$ and, by \eqref{p1}, $s^{-M}p(\l^{(j)})\to0$ as $s\to\infty$. Also, $S_m(\l)$ are uniformly bounded for all $m$ and $\l\in\R^d$.
  So we have
 $$
  \left|\sum_{m:\iota(m)=j}(\l^{(j)})^m s^{-M}S_m(\l^{(j)})\right|\le s^{-M}|p(\l^{(j)})|+\sum_{\|m\|<M}s^{-M}|(\l^{(j)})^mS_m(\l^{(j)})|\to0\quad (s\to\infty)
$$
uniformly with respect to $j$ and $\l^{(j)}$. Consequently,
$$
   \left|\sum_{j=1}^J\sum_{m:\iota(m)=j}(\l^{(j)})^ms^{-M}S_m(\l^{(j)})\right|\to0\quad (s\to\infty),
 $$
 which contradicts \eqref{c1}. \bs

\medskip
{\bf Proof of Theorem \ref{T10}}. By Theorem \ref{T11} b), $\hat f$ has the form
$$
\hat f(y)=\sum_k (2\pi iy)^k\hat\mu_k,\quad\mbox{where}\quad\hat\mu_k=\sum_{\|m\|\le H}\sum_{\g\in\G_k}q_{k,m}(\g)D^m\d_\g
$$
with uniformly bounded coefficients $q_{k,m}(\g)$ and uniformly discrete support $\G_k$, which satisfies \eqref{et}. Set
$$
  \nu_{k,m}=\sum_{\g\in\G_k}q_{k,m}(\g)\d_\g
$$
and apply Theorem \ref{T11} a) to every distribution
$$
\hat\mu_k=\sum_{\|m\|\le H}D^m\nu_{k,m},
$$
using the inverse Fourier transform instead of the direct one. We get
$$
  \check\nu_{k,m}=\sum_{\l\in\L_{k,m}}S_{k,m}(\l)\d_\l,
$$
with uniformly discrete $\L_{k,m}=\supp\check\nu_{k,m}$ and uniformly bounded $S_{k,m}(\l)$. Arguing as at the beginning of the proof of Theorem \ref{T11},
we obtain that the measures $\check\nu_{k,m}$ are almost periodic. Further,
$$
   \mu_k=\sum_{\|m\|\le M}(-2\pi i\l)^m\sum_{\l\in\L_{k,m}}S_{k,m}(\l)\d_\l,\quad\mbox{hence,}\quad p_k(\l)=\sum_{\|m\|\le M}(-2\pi i\l)^m S_{k,m}(\l).
$$
Using Lemma \ref{L2} with $\sigma_m=\check\nu_{k,m}$, we get $p_k(\l)=\sum_{\|m\|\le h(k)}(-2\pi i\l)^m S_{k,m}(\l)$. Therefore, for all $k$
$$
\frac{|p_k(\l)}{(1+|\l|)^{h(k)}}\le\sum_{\|m\|\le h(k)}\frac{(2\pi|\l|)^m}{(1+|\l|)^{h(k)}}\max_{\|m\|\le h(k)} |S_{k,m}(\l)|\le C\max_{\|m\|\le h(k)} |S_{k,m}(\l)|.
$$
Taking into account \eqref{c0}, we get the measures $\check\nu_{k,m}$ satisfies conditions of Proposition \ref{P4}. The statement of Theorem \ref{T10} follows from the embedding
 $\L\subset\cup_{k,m}\supp\check\nu_{k,m}$. \bs

\bigskip
I am very grateful to the referee for carefully reading my article and giving numerous remarks.