text:
\section{Introduction}
The widespread use of Convolutional Neural Networks (CNNs), as a go-to solution for a wide range of AI-related problems, combined with the high computational load typically associated with their execution, has led researchers to extensively work on the development of hardware accelerators for CNN inference \cite{inmem}\cite{eyeriss}\cite{hyperdrive}\cite{CNN-P}. Availability of this kind of hardware is key in embedded use-cases involving near-sensor processing of data, according to the edge computing paradigm \cite{vestias2019survey}. Among the different solutions available in literature, an important role is played by FPGA-based architectures, and, in more detail, by solutions exploiting the cooperation between general-purpose processors and FPGAs available in modern All-programmable SoCs, e.g. Zynq and Zynq Ultrascale+ devices by Xilinx, that take profit from the efficient implementation of Multiply-And-Accumulate (MAC) operations on the large amount of DSP Slices available \cite{survey}.
CNNs have been employed for computer vision applications, such as image recognition \cite{img_rec_He_Zhang,imagenet_Krizhevsky,deepImage}, object detection \cite{deepFace} or video frame classification \cite{videoClass}. However, convolutional networks have also been extended to deal with time sequences. These CNN variations are usually indicated as Temporal Convolutional Networks (TCNs) \cite{lea2017temporal}. Multiple TCN architectures have been proposed, reaching impressive performance on tasks such as sentence classification \cite{sentClass}, speech recognition \cite{deepSpeech}, text understanding \cite{textUnde}, Natural Language Processing tasks \cite{NLP} and, more recently, machine translation \cite{machineTras}, audio synthesis \cite{wavenet}, language modeling \cite{lang_model} or signal sequence analysis in the healthcare domain, such as action detection \cite{3D_skeleton} or ECG classification \cite{ecg}.
Research work of Bai et al. \cite{bai2018empirical} demonstrates that the exploitation of a Temporal Convolutional Network for typical sequence modelling tasks often outperforms older and better-known Recurrent Neural Networks \cite{lstm} \cite{gru}. \newline
Thus, the rapidly increasing interest in TCNs pushes for investigating on acceleration for these networks. Especially in the embedded domain, where (near) real-time analysis of sequences of data samples acquired by sensors is a common case, accelerating this kind of workload on reconfigurable devices is a very appealing approach.
In this work we explore the capabilities of a state-of-the-art CNN inference accelerator \cite{neu2017} specifically enriched to provide the flexibility needed in TCNs, to support freely selectable \emph{kernel sizes} and \emph{dilated} convolutions, with freely selectable \emph{dilation rates}. We focus on an implementation on low-cost and low-power all-programmable SoCs, more suitable for the integration of edge-computing and IoT processing nodes, considering two widely accessible devices in the Zynq and the Zynq Ultrascale+ families.
Performance is evaluated over various benchmark TCNs, reporting on absolute execution time as well as on the efficiency of the execution with respect to the peak performance imposed by the device resources.
We propose an optimization method relying on \emph{batch processing} to improve efficiency in cases where operational intensity is critical. We also assess the efficiency of the FPGA-based acceleration, comparing with software execution and with state of the art accelerators on bi-dimensional CNNs.

The reminder of this paper is organized as follows: Section \ref{sec:related} discusses the related work. Section \ref{sec:tcn_gen} analyses the TCN model. Section \ref{sec:ref_arch} gives an overview of the accelerator architecture taken as reference for this work. Section \ref{sec:CE} describes implementation details and strategies. The experimental results are presented in Section \ref{sec:hw_impl_eval} and Section \ref{sec:exp_res}. Conclusions are exposed in Section \ref{sec:concl}.
\section{Related Work}
\label{sec:related}

The landscape of FPGA-based accelerators for CNN is crowded and multifaceted \cite{guo2019dl}. Several approaches have been proposed in recent years, focusing both on the embedded domain and on architectures aimed to speed-up execution on cloud servers.
However, work on FPGA acceleration has mainly focused on classic CNN networks.

Yu et al. \cite{yu2019data} developed an FPGA acceleration platform that leverages a unified framework architecture for general-purpose CNN inference acceleration at a data center achieving a throughput comparable with the state-of-the-art GPU in this field, with less latency. This work exploits on-chip DSPs, on a Kintex KU115, arranged as supertile units (SUs), to overcome the computational bound and, together with dispaching-assembling model and broadcast caches, to deal with the memory bound.

Zhang et. al. \cite{Caffeine} proposed Caffeine, a hardware/software library to efficiently accelerate CNNs on FPGAs, leveraging a uniformed convolutional matrix multiplication representation targeting both computation-intensive convolutional layers and communication-intensive fully connected layers of CNN which maximizes the underlying FPGA computing and bandwidth resources utilization. Ma et. al. \cite{RTLCompiler} presented an RTL-level CNN compiler that generates automatically customized FPGA hardware for the inference tasks of CNNs from software to FPGA. The approach proposed by \cite{RTLCompiler} relies on a template accelerator architecture described in Verilog including all the main functions employed by CNNs such as convolutions, pooling, etc, which are automatically customized at design time to match the requirements of the target CNN model.
The proposed methodology is demonstrated with end-to-end FPGA implementations of complex CNN models such as NiN, VGG-16, ResNet-50, and ResNet-152 on two standalone Intel FPGAs, Stratix V and Arria 10, providing average performance up to 720 GOps.

These two frameworks provide huge performance gains when compared to state-of-the-art accelerators and general-purpose CPUs and GPUs. However, both leverage large FPGA devices such as Virtex7 and Arria 10, they mainly target server applications exploiting batching to improve memory access performance and bandwidth utilization.

This approach is less suitable for embedded applications where cheap and compact SoCs integrating embedded processors and FPGAs are desirable, and images have to be processed in real-time.
In this domain, Venieris et. al. \cite{LatencyDriven} presented a latency-driven design methodology for mapping CNNs on FPGAs. As opposed to previously presented approaches mainly intended for bandwidth-driven applications, this work targets real-time applications where the batch size is constrained to one, relying on Xilinx high-level synthesis tools for mapping (i.e. Vivado HLS), demonstrated on relatively simple CNN such as AlexNet, and on a very regular one such as VGG16 featuring only 3$\times$3 kernels, providing a peak performance of 123 GOps. Other work focuses on a template-based approach based on programmable or customizable RTL accelerators proposed in architectures \cite{RTLCompiler}\cite{Snowflake}\cite{GoingDeeper}, more similar to the one that is used in this paper.

SnowFlake \cite{Snowflake} exploits a hierarchical design composed of multiple compute clusters. Each cluster is composed of four vectorial compute units including a vectorial MAC, vectorial max, a maps buffer, weights buffers and trace decoders. SnowFlake provides a computational efficiency of 91\%, and an operating frequency of 250 MHz (best-in-class for CNN accelerators on Xilinx Zynq Z-7045 SoC). However, although the vector processor-like nature of the accelerator is very flexible, delivering significant performance also for 1$\times$1 kernels, it prevents to fully exploit of spatial computation typical of application-specific accelerators, which leads to overheads due to load/store operations necessary to fetch weights and maps from the buffers. This is highlighted by the low utilization of the DSP slices available on the FPGA (i.e. only 256 over 900), and by the performance when executing end-to-end convolutional neural networks, which is lower than that of other architectures including the proposed one even though the operating frequency of the CNN engine is significantly higher.

Several approaches tackling FPGA architectures for image-processing CNN, have explored the reduction of the precision of arithmetic operands to improve energy efficiency. Although most of the architectures available in literature feature a precision of 16-bit (fixed-point)\cite{LatencyDriven, Snowflake, RTLCompiler} numerous reduced-precision implementations have been proposed recently, relying on 8-bit, 4-bit accuracy for both maps and weights, exploiting the resiliency of CNNs to quantization and approximation \cite{GoingDeeper}.

Qiu et. al. \cite{GoingDeeper} proposed a CNN accelerator implemented on a Xilinx Zynq platform exploiting specific hardware to support 8/4 bit dynamic precision quantization, at the cost of 0.4\% loss of classification accuracy.
Other extreme approaches to quantization exploit ternary \cite{Ternary} or binary \cite{FINN} neural-networks accelerators for FPGA. This approach significantly improves the computational efficiency of FPGA Accelerators, allowing to achieve performance level as big as 8 TOPS \cite{Ternary}.

Recent work by Rasoulinezhad et al. \cite{pir-dsp}, starting from the Xilinx DSP slices, proposed an optimized DSP block called PIR-DSP to efficient map 9, 4 and 2 bits data precision MAC operations. It is implemented as a parameterized module generator targeting both FPGAs and ASICs reaching an estimate run time energy decrease up to 31\% for a MobileNet-v2 implementation compared with a standard DSP mode.
Other works, like Wang et al. \cite{wang2020lutnet}, leverage FPGA LUT blocks as inference operators for Binary Neural Network (BNN) achieving up to twice area efficiency compared to state-of-the-art binarized NN implementation and against several standard networks models.

While small networks like MNIST, CIFAR10, SVHN, GTSRB can reach good classification accuracy, the training is still a big challenge for larger networks such as VGG or ResNet \cite{Courbariaux2015a}. The usability of extreme quantization is also not demonstrated for TCN-related tasks, sequence classification and modelling.

Probably the most powerful currently available FPGA-based acceleration engine is the proprietary one offered by Xilinx, which provides an integrated framework, called VitisAI \cite{VitisAI}, that helps designers in mapping CNNs on a templated soft IP called Deep Learning Processing Unit (DPU) \cite{DPU}. The DPU provides impressive performance on CNNs, using quantization and high clock frequency in DSP slices. Quantization is required and can be applied automatically using a dedicated tool included in VitisAI. DSP slices are clocked at very high frequency, using a \emph{DSP Double Data Rate (DDR) technique} \cite{DPU_guide}, which uses a 2x frequency domain to increase peak performance. However, the support for TCN is missing, since the DPU does not support arbitrary dilation, stride and kernel sizes and the the VitisAI quantization process does not support 1D convolutions.
To the best of our knowledge, there are no published FPGA-based accelerators tuned to speed-up inference for generic Temporal Convolutional Networks.

As main novel contributions of this work, we propose:
\begin{itemize}
    \item an enriched architectural template supporting efficient TCN inference on FPGA;
    \item a methodology for the optimal execution/scheduling of data-transfers exploiting the specific sequence-based structure of data in TCNs
    \item a methodology for improving efficiency based on \emph{sample batching} (sequence buffering)
    \item the first (to the best of our knowledge) experimental evaluation of the usability of FPGA-based acceleration for TCNs, based on different end-to-end benchmarks and two APSoC devices
\end{itemize}

\section{TCN model generalities}
\label{sec:tcn_gen}

A dilated convolution operation $F$ on element $s$ of a sequence \cite{bai2018empirical} can be defined as:

\begin{equation}
     F(s) = (x\ast_d f)(s) = \sum_{i=0}^{k-1} \ f(i)\cdot x_{s-d\cdot i}
     \label{eq:tcn_eq}
\end{equation}

where $x\in \mathbf{R}^n $ is a 1-D input sequence, $f:\{0,...,k-1\} \in \mathbf{R}$ is a kernel of size $k$ and $d$ is the $dilation\ rate$.

The sequence of samples that constitute the input of a TCN can be processed both off-line or in a real-time streaming. The second case is very useful in application cases requiring continuous analysis of the input sequence, e.g. aimed at the identification of specific events and/or at promptly closing the loop on data-triggered actuations. This implies that the sequence must be analyzed at  every time step, after being updated with a new sample. It is possible to identify the minimal sequence
size to produce a valuable output sample as the \emph{receptive field} that depends on convolutional layer parameters such as the \emph{kernel\_size} and the \emph{dilation rate}:

\begin{equation}
     receptive field = 1+ \sum_{l=1}^L \ [k(l) - 1]\times d(l)
     \label{eq:rec_field}
\end{equation}

where $l \in {1,2 ... L}$ is a layer of the network.
This can be thought of as a sliding processing window for the input sequence.

\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/tcn_rec_field_2.pdf}
    \caption{TCN execution graph. For this example, see (\ref{eq:rec_field}), $receptivefield=1+(2-1)\times 1+(3-1)\times 2+(4-1)\times3=15$}
    %
    \label{fig:tcn_rec_field}
\end{figure}
Figure \ref{fig:tcn_rec_field} highlights these concepts.
It is worth noticing that increasing the dilation parameter leads to an increase of the network's memory without affecting its depth.

\section{Reference Accelerator Architecture}
\label{sec:ref_arch}
\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/NEU_arch_simpler.pdf}
    \caption{NEURAghe architecture}
    \label{fig:neu_arch}
\end{figure}
Our TCN accelerator is an extension of NEURAghe \cite{neu2017}, a CNN inference accelerator architecture that exploits the cooperation between the ARM Cortex-A9 Processing System (PS) and the Programmable Logic (PL) in Xilinx Zynq devices. Communication at the PS-PL interface is allowed by the PS high-performance general purpose ports. In NEURAghe, as can be seen in Figure \ref{fig:neu_arch}, the Programmable Logic hosts a Convolution Specific Processor (CSP), while the processing system acts as a General Purpose Processor (GPP) dealing with tasks hard to accelerate by parallelization on programmable logic, such as data marshalling or non-Conv layers.
NEURAghe integrates a RISC-V lightweight processor inside the CSP, dedicated to the execution of a firmware that schedules data transfers and convolutions without PS intervention, leaving the latter available for actual computation workload.
In order to support TCN execution, we have enhanced the CPS IP, described in System Verilog and available as open source \footnote{\urlstyle{tt}\url{https://github.com/neuraghe/NEURAghe}}. We have added new features to improve flexibility, requiring substantial modification of the main computational core dedicated to Multiply and Accumulate (MAC) operations execution, called Convolution Engine, and an improvement of the circuitry and procedures managing transfers to/from the DDR memory.
In particular, the Convolution Engine (Figure \ref{fig:ce}) is composed by a MAC Matrix of $N_{cols}$ columns by $N_{rows}$ rows of Sum of Product (SoP) units in charge to calculate the contribution of $N_{cols}$ input features to $N_{rows}$ output features. $N_{rows}$ Shift Adder modules sum together partial result from SoPs in each row with data values resulted from the $N_{cols}$ previously computed input feature partial results read from on-chip memory, enabling successive accumulation over multiple CE runs.
\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/CE.pdf}
    \caption{Convolution Engine. $N_{rows}\times N_{cols}$ MAC Matrix}
    \label{fig:ce}
\end{figure}

\section{TCN supporting hardware features}\label{sec:CE}

Considering the typical features of TCNs, we designed the CE according to several design principles. The accelerator:
\begin{itemize}
    \item has to be $kernel\_size$ agnostic;
    \item must execute convolutions with multiple stride values without performance overhead;
    \item must support freely selectable dilation values.
\end{itemize}

\subsection{Freely selectable kernel sizes}

To support arbitrary kernel sizes, we have chosen to dedicate one single DSP cell to compute an entire convolution kernel, reusing it over a number of cycles depending on the kernel size. A new sample of the output feature under production is thus produced after $kernel\_size$ cycles and is ready to be sent to the Shift Adder module. Using one single DSP cell per kernel can easily require the instantiation of a very high number of SoPs, to the aim of exploiting as many resources as possible among those available on the target device.
To keep the MAC Matrix growth feasible, we designed SoPs to be composed by $4$ Xilinx DSP48E primitives, performing $4\ MACs/cycle$, operating in parallel on $4$ different 16-bit samples, as visualized in Figure \ref{fig:sop_elab}, from $4$ neighbour convolution windows in an input feature.
\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/SoP_elaboration.pdf}
    \caption{SoP elaboration scheme}
    \label{fig:sop_elab}
\end{figure}

Figure \ref{fig:sop} represents the organization of a SoP.
Considering the template proposed in Figure \ref{fig:ce}, the MAC Matrix implementation requires a deterministic number of DSP48E primitives, as indicated by Equation \ref{DSPscount}.
\begin{equation}\label{DSPscount}
    N_{DSPs}= N_{rows}\times N_{cols}\times 4
\end{equation}
\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/sop_dsp.pdf}
    \caption{Sum of Product Unit}
    \label{fig:sop}
\end{figure}

\subsection{Flexible activations and weights fetching}

\begin{figure*}[]
    \centering
    \includegraphics[width = \textwidth]{img/act_n_wei_fetch.pdf}
    \caption{Memory transfers}
    \label{fig:act_fetch}
\end{figure*}

To enable arbitrary stride and dilation values, the fetching of input samples from the internal memory has been designed to be very flexible. Each CE port dedicated to input samples is endowed by a programmable Activation Source module, while weight kernels are fetched from Weight Memory banks by means of Weights Source modules. Such source modules can be programmed at start-up according to stride, dilation, aspect ratio and size of activations and weight kernels. Fetching of bi-dimensional memory sections is enabled, to support generic CNN execution. \par\noindent
The memory subsystem has been designed to enable conflict-less loading of neighbour convolution windows. The Activation Source module controls $N_{cols}$ ports of the convolution engine. Each one loads samples from a dedicated BRAM module. Considering that four samples, belonging to different windows, can be loaded in the same cycle, to support stride values up to 3 samples (which is sufficient to support most of the TCN use-cases available in literature), each module of the activation memory has to be composed of at least 8 different independently accessible RAMB18 modules. Figure \ref{fig:conflict_bram} represents an example where a different configuration of RAMB18 modules can determine a conflict.\par\noindent

\begin{figure}[H]
    \centering
    \includegraphics[width = \columnwidth]{img/conflict_bram.pdf}
    \caption{BRAM read conflict example. Samples are stored in BRAM modules using interleaving. Consecutive samples are stored in adjacent RAMB18 banks. 4 DSP slices in a SoP units, with stride 2, in the first cycle of the convolution, load respectively samples 0,2,4,6. With four RAMB18 banks, samples 0-4 and 2-6 are on the same bank, creating a conflict.}
    \label{fig:conflict_bram}
\end{figure}

Moreover, the Weight source module controls one port per each SoP in the matrix, which has to be implemented by at least one RAMB18 module. \par\noindent
Finally, the CE has a set of ports that are used to write results and to load previously computed partial results, when a convolution requires to accumulate over several accelerator operations. These ports are controlled by a Partial Result Source module and by an Output Sink module. Each of these modules controls $N_{rows}$ ports, each one writing/reading four samples simultaneously. Thus BRAM modules in the corresponding memory region are composed by at least 8 RAMB18 modules each. \par\noindent Samples and weights, in the experiments presented in this paper, are all using a 16-bit data format, thus RAMB18 modules are configured to expose two 16-bit addressable ports and can be 1024 words deep. \par\noindent
Considering the described organization, a given architectural configuration requires a number of RAMB18 primitives that can be deterministically estimated as indicated in Equation \ref{BRAMScount}.
\begin{equation}\label{BRAMScount}
    N_{BRAMs}= N_{rows}\times N_{cols} \\
    + N_{cols} \times 8 \\
    + (N_{rows}\times 2)\times 8 + \\
    32
\end{equation}
The first component corresponds to weight memory, the second to the modules storing input activations, the third to output and partial results memories. 32 blocks are used to implement the RISC-V scheduler instruction memory and private memory.

\subsection{TCN support in firmware}\label{sec:SCHED}

The programming model used in NEURAghe envisions the ARM-based processing system in the Zynq platforms to execute a C program implementing the neural network inference. When the accelerator implemented in the programmable logic has to be used. The program in the PS sends commands describing the layer to be executed. The RISC-V soft-core in the accelerator decodes the command and decomposes the layer in sub-operations, namely partial convolutions in the accelerator and data transfers from/to the off-chip memory, executing an optimized firmware which is also coded in C. The firmware uses a double-buffering technique, to allow the accelerator to overlap transfers phases with convolutions, as represented in Figure \ref{fig:scheduling}, reducing as much as possible idle times in the CE to exploit the processing capabilities of DSP slices with maximum efficiency.

\begin{figure}[H]
    \centering
    \includegraphics[width = \columnwidth]{img/scheduling.pdf}
    \caption{Scheduling scheme}
    \label{fig:scheduling}
\end{figure}

When considering TCN executions, the previously described paradigm has to be applied to the characteristics of the algorithm. For every Convolutional Layer in a TCN, given its \emph{kernel\_size} and a \emph{dilation}, it is possible to consider a local \emph{receptive field}, indicated as $RF_{local}$ in Equation \ref{loc_rec_field}, that is the minimum amount of layer's input samples per channel needed to produce a valuable output sample.

\begin{equation}\label{loc_rec_field}
    RF_{local}= 1 + (k - 1)\times d
\end{equation}

\begin{figure}[h]
    \subfloat[\label{fig:B=1}]{\includegraphics[width = 0.5\columnwidth]{img/hwce_feed_B=1.pdf}}
    \hfill
    \subfloat[\label{fig:B=B}]{\includegraphics[width = 0.5\columnwidth]{img/hwce_feed_B=B.pdf}}
    \caption{TCN execution on NEURAghe}
    \label{fig:tcn_exec}
\end{figure}

Figure \ref{fig:B=1} shows how input and output transfers are implemented for TCNs.
At every new time step, input to a layer is updated by adding one new sample to input features. In order to execute the layer, the firmware triggers DMA transfers to load to the activation input memory $RF_{local}$ samples per each input feature. After the execution, one output sample is produced per output feature. Output samples are sent to DDR using an output DMA transfer, to be stored until the next time step.

\subsection{Improving through batch processing} \label{batch proc}

Although the previous approach provides the minimum classification/recognition latency, executing the network every time a new sample is available to update the input sliding window, it can determine performance to be bandwidth limited.
This is because all of network parameters/weights must be loaded for every layer. So, despite the double-buffered scheduling strategy, transfer and computation phases hardly overlap, affecting the \emph{operational intensity} of the application.
\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/roofline_model_ECG_3.pdf}
    \caption{Use case benchmark's Roofline Model for the Convolution Specific Processor (12x4 MAC Matrix in Z-7020 SoC) with respect to different batch sizes}
    \label{fig:roofline}
\end{figure}
The roofline model in Figure \ref{fig:roofline} shows performance trend starting from the sample-by-sample processing (leftmost red cross), on a use-case that will be presented in the following.

If the latency constraint is not extremely tight, it is possible to pre-buffer input samples in order to process longer sample sequences (batch) and produce more output with every execution.
Figure \ref{fig:B=B} shows the transfer scheduling when \emph{batch size} is increased.
As the \emph{batch size} increases the architecture gets closer and closer to the computational limit (rightmost red cross on the roofline plot of Figure \ref{fig:roofline}) because of the growing number of operations performed. In this way, it is possible to increase the utilization of computing resources and to gain efficiency.
As may be noticed in Figure \ref{fig:roofline}, when sample batch size is small, besides being on the badwidth-limited region of the plot, points in the roofline model are also distant from the theoretical achievable performance. This is due to overheads related with CE programming/warm-up,  and to initial and final input/output transfers, which cannot overlap with convolutions. The impact of this overhead is limited when the operational intensity increases, reducing the distance between points and the theoretical roofline model.

\section{Hardware Implementation Evaluation}
\label{sec:hw_impl_eval}

\subsection{Design Space Exploration}

A designer willing to use the NEURAghe template, on a given target SoC, has multiple possible architectural configurations available. In order to perform a careful selection, it is possible to perform a simple design space exploration and to choose a near-optimal setup from the performance point of view. To select the architectures presented in this paper, we have used a simple grid search that evaluates multiple configurations, featuring different values of $N_{rows}$ and $N_{cols}$, to maximize the number of SoPs, while keeping the number of used DSPs and BRAMs in the range of availability imposed by the target SoC.
Table \ref{tab:board_utiliz} shows utilization numbers estimated using the Equations \ref{DSPscount} and \ref{BRAMScount}. Light-gray coloured cells in the table indicate which configurations are not implementable in a Z7020 SoC, due to excessive DSP or BRAM utilization. Darker-coloured cells indicate configurations that are not feasible in an Ultrascale+ ZU3EG device.

\begin{table}[h]
    \caption{RAMB18 and DSP utilization in NEURAghe architecture with respect to the MAC Matrix shape}
    \begin{center}
    \setlength{\arrayrulewidth}{0.3mm}
    \renewcommand{1} \begin{tabular}{*{1}{c}|*{1}{c}*{1}{c|}*{1}{c}*{1}{c}*{1}{c}*{1}{c}}{1.2}
    \begin{tabular}{*{1}{|c}*{1}{|c}|*{9}{c|}}
                \cline{1-11}
                    &   \multicolumn{9}{c|}{$N_{cols}$}  & \\
                \cline{1-11}
                                            &    \tikzmark{startbram} 144 \tikzmark{endbram}    &    156    &    168    &    180    &    192    &    204    &    216    &    228    &    240    &    \multirow{2}{0.8em}{4}\\
                                            &    \tikzmark{startdsp} 64 \tikzmark{enddsp}    &    80    &    96    &    112    &    128    &    144    &    160    &    176    &    \tikzmark{startone} 192 \tikzmark{endone}    &    \\
                \cline{2-11}
                                            &    164    &    177    &    190    &    203    &    216    &    229    &    242    &    255    &    268    &    \multirow{2}{0.8em}{5}\\
                                            &    80    &    100    &    120    &    140    &    160    &    180    &    200    &    \tikzmark{starttwo} 220 \tikzmark{endtwo}    &    \cellcolor{lightgray}240    &    \\
                \cline{2-11}
                                            &    184    &    198    &    212    &    226    &    240    &    254    &    268    &    \cellcolor{lightgray}282    &    \cellcolor{lightgray}296    &    \multirow{2}{0.8em}{6}\\
\multirow{9}{0.8em}{\rotatebox{90}{$N_{rows}$}}    &    96    &    120    &    144    &    168    &    192    &    216    &    \cellcolor{lightgray}240    &    \cellcolor{lightgray}264    &    \cellcolor{lightgray}288    &    \\
                \cline{2-11}
                                            &    204    &    219    &    234    &    249    &    264    &    279    &    \cellcolor{lightgray}294    &    \cellcolor{lightgray}309    &    \cellcolor{lightgray}324    &    \multirow{2}{0.8em}{7}\\
                                            &    112    &    140    &    168    &    196    &    \cellcolor{lightgray}224    &    \cellcolor{lightgray}252    &    \cellcolor{lightgray}280    &    \cellcolor{lightgray}308    &    \cellcolor{lightgray}336    &    \\
                \cline{2-11}
                                            &    224    &    240    &    256    &    272    &    \cellcolor{lightgray}288    &    \cellcolor{lightgray}304    &    \cellcolor{lightgray}320    &    \cellcolor{lightgray}336    &    \cellcolor{lightgray}352    &    \multirow{2}{0.8em}{8}\\
                                            &    128    &    160    &    192    &    \cellcolor{lightgray}224    &    \cellcolor{lightgray}256    &    \cellcolor{lightgray}288    &    \cellcolor{lightgray}320    &    \cellcolor{lightgray}352    &    \cellcolor{gray}384    &    \\
                \cline{2-11}
                                            &    244    &    261    &    278    &    \cellcolor{lightgray}295    &    \cellcolor{lightgray}312    &    \cellcolor{lightgray}329    &    \cellcolor{lightgray}346    &    \cellcolor{lightgray}363    &    \cellcolor{lightgray}380    &    \multirow{2}{0.8em}{9}\\
                                            &    144    &    180    &    216    &    \cellcolor{lightgray}252    &    \cellcolor{lightgray}288    &    \cellcolor{lightgray}324    &    \tikzmark{startthree} \cellcolor{lightgray}360 \tikzmark{endthree}    &    \cellcolor{gray}396    &    \cellcolor{gray}432    &    \\
                \cline{2-11}
                                            &    264    &    \cellcolor{lightgray}282    &    \cellcolor{lightgray}300    &    \cellcolor{lightgray}318    &    \cellcolor{lightgray}336    &    \cellcolor{lightgray}354    &    \cellcolor{lightgray}372    &    \cellcolor{lightgray}390    &    \cellcolor{lightgray}408    &    \multirow{2}{0.8em}{10}\\
                                            &    160    &    200    &    \cellcolor{lightgray}240    &    \cellcolor{lightgray}280    &    \cellcolor{lightgray}320    &    \cellcolor{lightgray}360    &    \cellcolor{gray}400    &    \cellcolor{gray}440    &    \cellcolor{gray}480    &    \\
                \cline{2-11}
                                            &    \cellcolor{lightgray}284    &    \cellcolor{lightgray}303    &    \cellcolor{lightgray}322    &    \cellcolor{lightgray}341    &    \cellcolor{lightgray}360    &    \cellcolor{lightgray}379    &    \cellcolor{lightgray}398    &    \cellcolor{lightgray}417    &    \cellcolor{gray}436    &    \multirow{2}{0.8em}{11}\\
                                            &    176    &    220    &    \cellcolor{lightgray}264    &    \cellcolor{lightgray}308    &    \cellcolor{lightgray}352    &    \cellcolor{gray}396    &    \cellcolor{gray}440    &    \cellcolor{gray}484    &    \cellcolor{gray}528    &    \\
                \cline{2-11}
                                            &    \cellcolor{lightgray}304    &    \cellcolor{lightgray}324    &    \cellcolor{lightgray}344    &    \cellcolor{lightgray}364    &    \cellcolor{lightgray}384    &    \cellcolor{lightgray}404    &    \cellcolor{lightgray}424    &    \cellcolor{gray}444    &    \cellcolor{gray}464    &    \multirow{2}{0.8em}{12}\\
                                            &    192    &    \cellcolor{lightgray}240    &    \cellcolor{lightgray}288    &    \cellcolor{lightgray}336    &    \cellcolor{gray}384    &    \cellcolor{gray}432    &    \cellcolor{gray}480    &    \cellcolor{gray}528    &    \cellcolor{gray}576    &    \\
         \hline
                                            &    4        &    5        &    6        &        7    &    8        &    9        &    10    &    11    &    12    & \\
         \hline
    \end{tabular}
    \end{center}
    \label{tab:board_utiliz}
    \begin{center}
    \begin{tabular}{*{1}{c}*{4}{|c}}
                &   avail.  &   avail. &                                        &   out of        \\
                &   RAMB18  &   DSP &   selected                                &   resources        \\
\hline
         Z7020  &   280     &   220 &   \tikzmark{startlone}     \tikzmark{endlone}    &   \cellcolor{lightgray}   \\
\hline
         ZU3EG  &   432     &   360 &   \tikzmark{startltwo}      \tikzmark{endltwo}    &   \cellcolor{gray}        \\
    \end{tabular}
    \end{center}

\begin{tikzpicture}[remember picture,overlay]

\draw[rounded corners,black,thick]
  ([shift={(-0.5\tabcolsep,-0.5ex)}]pic cs:startbram)
    rectangle
  ([shift={(0.5\tabcolsep,2ex)}]pic cs:endbram);

\draw [ -stealth ]
    ([shift={(0\tabcolsep,0)}]pic cs:startbram)
        --
    ([shift={(-11\tabcolsep,0.1)}]pic cs:startbram) node [above] {RAMB};

\draw[rounded corners,black,thick]
  ([shift={(-0.5\tabcolsep,-0.5ex)}]pic cs:startdsp)
    rectangle
  ([shift={(0.5\tabcolsep,2ex)}]pic cs:enddsp);

\draw [ -stealth ]
    ([shift={(0\tabcolsep,0)}]pic cs:startdsp)
        --
    ([shift={(-8\tabcolsep,-0.1)}]pic cs:startdsp) node [left] {DSP};

\foreach \Val in {one,two}
{
\draw[rounded corners,green,thick]
  ([shift={(-0.5\tabcolsep,-0.5ex)}]pic cs:start\Val)
    rectangle
  ([shift={(0.5\tabcolsep,4.8ex)}]pic cs:end\Val);
}

\draw[rounded corners,red,thick]
  ([shift={(-0.5\tabcolsep,-0.5ex)}]pic cs:startthree)
    rectangle
  ([shift={(0.5\tabcolsep,4.8ex)}]pic cs:endthree);

\draw[rounded corners,green,thick]
  ([shift={(-0.5\tabcolsep,-0.5ex)}]pic cs:startlone)
    rectangle
  ([shift={(1\tabcolsep,1ex)}]pic cs:endlone);

\draw[rounded corners,red,thick]
  ([shift={(-0.5\tabcolsep,-0.5ex)}]pic cs:startltwo)
    rectangle
  ([shift={(1\tabcolsep,1ex)}]pic cs:endltwo);
\end{tikzpicture}
\end{table}

\subsection{Implementation on different SoCs}
We have implemented the NEURAghe configurations with different Convolution Engine's MAC Matrix shapes, selected after the DSE process, on two target platforms: a Xilinx Zynq Z-7020 and a Xilinx Zynq Ultrascale+ ZU3EG. For the Z-7020 we have selected two configurations, featuring a similar number of DSP slices and similar clock frequencies. Table \ref{tab:occup_12x4} shows resource occupation on the reconfigurable logic of a first configuration implemented on the Z-7020, featuring a 12$\times$4 MAC matrix of SoP units, that can be clocked up to 120 MHz, providing peak performance of 46 GOPS/s. Table \ref{tab:occup_11x5} shows results related with a similar configuration that integrates an 11$\times$5 MAC matrix, using slightly more DSPs but clockable up to 110 MHz, with a peak performance of 48.4 GOPS/s. Finally, Table \ref{tab:occup_ultra} describes the results of the implementation of a configuration, featuring a 9$\times$10 MAC matrix of SoP modules, on the ZU3EG. This design uses all the DSP slices on the chip and can be clocked at 180 MHz, providing a peak of 129.6 GOPS/s.
\begin{table}[h!]
    \caption{Resource occupation on a Xilinx Zynq Z-7020 (12x4 MAC matrix)}
    \centering
    \begin{tabular}{*{1}{c}||*{5}{c}}
                    &  DSP      & BRAM  & LUTs  & LUTs  & Regs \\
                    &           &       & (logic) & SR   &   \\
         \hline
         \hline
         Used       &  192   & 120    & 47230   & 259    & 26942 \\
         \hline
         Available  &  220    & 140   & 53200   & 53200    & 106400 \\
         \hline
         \%         &  87.27    & 85.71 & 88.78   & 1.49   & 25.32 \\
         \multicolumn{6}{c}{} \\
    \end{tabular}
    \label{tab:occup_12x4}
\end{table}
\begin{table}[h!]
    \caption{Resource occupation on a Xilinx Zynq Z-7020 (11x5 MAC matrix)}
    \centering
    \begin{tabular}{*{1}{c}||*{5}{c}}
                    &  DSP      & BRAM  & LUTs  & LUTs  & Regs \\
                    &           &       & (logic) & SR   &   \\
         \hline
         \hline
         Used       &  220   & 128    & 42964   & 283    & 26962 \\
         \hline
         Available  &  220    & 140   & 53200   & 53200    & 106400 \\
         \hline
         \%         &  100    & 91.4 & 80.76   & 1.63   & 25.34 \\
         \multicolumn{6}{c}{} \\
    \end{tabular}
    \label{tab:occup_11x5}
\end{table}
\begin{table}[h!]
    \caption{Resource occupation on a Xilinx Zynq UltraScale+ ZU3EG (9x10 MAC matrix)}
    \centering
    \begin{tabular}{*{1}{c}||*{5}{c}}
                    &  DSP      & BRAM  & LUTs  & LUTs  & Regs \\
                    &           &       & (logic) & SR   &   \\
         \hline
         \hline
         Used       &  360   & 354    & 47857   & 159    & 26463 \\
         \hline
         Available  &  360    & 432   & 70560   & 70560    & 141120 \\
         \hline
         \%         &  100    & 81.94 & 67.82   & 0.4   & 18.75 \\
         \multicolumn{6}{c}{} \\
    \end{tabular}
    \label{tab:occup_ultra}
\end{table}

\section{Experimental Results}
\label{sec:exp_res}
We tested the actual level of performance that can be achieved using the accelerator on three different benchmarks, trying to cover as much as possible the landscape of TCNs presented in literature:
\begin{itemize}
    \item a \emph{plain} TCN network for ECG monitoring and classification (Goodfellow et al. \cite{ecg}), that performs classification over single lead ECG waveforms and reaches around $90\%$ average accuracy. We call this benchmark \emph{ECG} in the following. This benchmark exposes real-time constraints and can be used to assess the usability of the accelerator in this kind of context.
    \item A network, called hereafter Res-TCN, based on residual units, with a structure taken from the Resnet family presented in \cite{resnet}. The TCN is presented in (\cite{3D_skeleton}) authors started from a 3D human action recognition dataset with 3D full skeleton annotations to extract a 1D feature representation per frame resulting in a 150-dimensional vector.
    \item  A more complex network, based on WaveNet  (Van  DenOord et al.  [14]),   for  Polyphonic Note  Transcription of Time-Domain Audio Signal ([19]). We indicate this use-case as WN-PNT.
\end{itemize}
We have considered three system implementations using NEURAghe, two of them, 11$\times$5 and 12$\times$4, implemented on a Zedboard development board, with the MAC matrix respectively clocked at 70 and 80 MHz, and one implemented on the Ultra96 development board, with MAC Matrix clocked at 180 MHz.

Table \ref{tab:mem_foot} shows the comparison between on-chip memory capabilities for each system implementation and the memory footprint of each use-case network that regards the overall occupation of weight kernels and input features through all layers, considering the sample batch size values that will be used in the following. As may be noticed, all the networks must be adequately managed with  the scheduling strategy presented in section \ref{sec:SCHED}, since their activation and weight memory footprint exceeds the memory available on the considered low-cost hardware platforms.

\begin{table}[h!]
    \caption{Implementation related on-chip memory capabilities and use-case networks memory footprint}
    \centering
    \begin{tabular}{*{1}{c}|*{3}{c}*{1}{c}*{1}{c}}

                    &   \multicolumn{3}{c}{input features [kB]}   & weight kernels [kB]   \\
         \hline
\multirow{2}{8em}{12x4 on Z-7020 on-chip memory}    & \multicolumn{3}{c}{\multirow{2}{1em}{192}}    &  \multirow{2}{1em}{96}   \\
            & & \\
         \hline
\multirow{2}{8em}{11x5 on Z-7020 on-chip memory}    & \multicolumn{3}{c}{\multirow{2}{1em}{176}}    &  \multirow{2}{1em}{110}   \\
            & & \\
         \hline
\multirow{2}{8em}{9x10 on ZU3EG on-chip memory}  & \multicolumn{3}{c}{\multirow{2}{1em}{144}}    &   \multirow{2}{1em}{180}  \\
            & & \\
         \hline
         ECG:    &  \textbf{B=1}  & \textbf{B=8} & \textbf{B=348}    &               \\
         memory footprint   &  216,8    &   250,8     &   1696,5    &     11495,6  \\
         \hline
         Res-TCN:    &  \textbf{B=1} &  \multicolumn{2}{c}{\textbf{B=144}}    &    \\
         memory footprint   &      37,3         &  \multicolumn{2}{c}{511,5}    &  5424,8  \\
         \hline
         WN-PTN:     &  \textbf{B=1}   &  \multicolumn{2}{c}{\textbf{B=504}}      &   \\
         memory footprint   &      553,5     &   \multicolumn{2}{c}{5846,49}  &   3328,8    \\

    \end{tabular}
    \label{tab:mem_foot}
\end{table}

\subsection{ECG classification use-case}
The network consists of several computational blocks mostly made of a 1D Convolutional layer, a batch normalization layer, a ReLU and a dropout stage, operating on streams of 16-bit samples, acquired at 300 Hz frequency.
In this case, we have assessed three different operating modes:
\begin{itemize}
\item \textit{minimum latency mode},
\item \textit{maximum throughput mode},
\item \textit{real-time execution}.
\end{itemize}
The first one provides classification in output after minimum latency. The network is executed as soon as possible per every input sample.
In the second operating mode, sample batching is used extensively, to maximize throughput without considering latency as an optimization objective. In the third mode, sample batching is exploited just enough to reach a throughput that allows respecting the real-time constraint posed by the input sampling frequency (300 Hz).

Figure \ref{fig:effic_compar_zed} shows the performance achieved on this benchmark by configurations implemented on the Zedboard, while Figure \ref{fig:time_compar_zed} shows execution times.

As expected, without sample batching, performance is significantly bandwidth-limited. In minimum latency mode ($B=1$ in the Figure), efficiency is very low for every layer, and consequently on the whole network. DSPs have very long idle times and actual performance is very far from the peak. Execution time is around 17 ms with the 12$\times$4 matrix, and a bit higher with the 11$\times$5.

To design the maximum throughput mode, we have iteratively tested different batch size values, to identify which value saturates the benefits achievable with sample batching. For this benchmark, such value is B=348, corresponding to a latency of around 1.2$s$. In this case, both configurations can operate very near to the peak performance, with an average efficiency of around 0.9. Some layers, especially Type 1, are still less efficient, however, their contribution to the overall execution time is limited. In maximum throughput mode, each network execution takes around 140 ms, producing in output classification of 348 sample consecutive sample sequences, corresponding to one sample every 0.4$ms$.

Finally, the real-time constraint requires to process one sequence in less than 3.3 $ms$. To design the real-time operating mode, by exploring the batch size, we identified $B=8$, to be the lowest value enabling that respects such requirement. Execution time is around 17 $ms$, corresponding to around 2 $ms$ per sample.

Since 12$\times$4 is more efficient than 11$\times$5 in all modes, in this case higher frequency is more important than the number of MACs, due to better utilization with the specific layer characteristics (the matrix is partially unused in the final operations of a convolution when the number of output features is not a multiple of 5).

\begin{table}[h!]
    \caption{Convolutional Layer characteristics for Network ECG \cite{ecg}}
    \centering
    \scriptsize

    {\def1} \begin{tabular}{*{1}{c}|*{1}{c}*{1}{c|}*{1}{c}*{1}{c}*{1}{c}*{1}{c}{1} \begin{tabular}{*{1}{c}||*{1}{c}|*{1}{c}|*{1}{c}|*{1}{c}|*{1}{c}|*{1}{c}|*{1}{c}|*{1}{c}}

                                                & type    & type    & type   & type  & type  & type   & type   & type  \\
                                                &   1     &  2      &   3    &   4   &   5   &   6    &    7   &  8     \\
    \hline
                             $input features$            &  1      &   320   & 256    & 256   & 128   & 128    & 128    & 64  \\
                             $output features$             &  320    &   256   & 256    & 128   & 128   & 128    & 64     & 64  \\
                             $Kernel\_size$     &  24     &   16    & 16     & 8     & 8     & 8      & 8      & 8  \\
                             $dilation\ rate$   &  1      &   2     & 4      & 4     & 6     & 8      & 8      & 8  \\
    \end{tabular}}
    \label{tab:CL_charact}
\end{table}

\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/effic_on_goodfellow_compar.pdf}
    \caption{Efficiency comparison on ECG \cite{ecg} for NEURAghe 12x4 and 11x5 MAC matrix configurations}
    \label{fig:effic_compar_zed}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/time_on_goodfellow_compar.pdf}
    \caption{Execution Time comparison on ECG \cite{ecg} for NEURAghe 12x4 and 11x5 MAC matrix configurations}
    \label{fig:time_compar_zed}
\end{figure}
The results are similar on the Ultra96 board, with increased performance, obviously, due to the higher clock frequency and the higher number of SoP modules. Efficiency and execution time are shown respectively in Figure   \ref{fig:effic_compar_ultra}  and in \ref{fig:time_compar_ultra}. Using minimum latency, efficiency is very low due to the bandwidth limits. Using $B=348$ we have around 0.9 efficiency corresponding to around 0.1 $ms$ per sample. To respect real-time constraint with minimum latency we can use $B=4$.

\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/effic_on_goodfellow_ultra.pdf}
    \caption{Efficiency on ECG \cite{ecg} for NEURAghe 9x10 MAC matrix configuration in ZU3EG}
    \label{fig:effic_compar_ultra}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[width = \columnwidth]{img/time_on_goodfellow_ultra.pdf}
    \caption{Execution Time on ECG \cite{ecg} for NEURAghe 9x10 MAC matrix configuration in ZU3EG}
    \label{fig:time_compar_ultra}
\end{figure}

\subsection{Res-TCN use-case}
This use case considers execution of the network presented in \cite{3D_skeleton}, made of stacked residual units, which perform 1D convolutions.
Table \ref{tab:skeleton_char_perf} shows the characteristics for the different types of Convolutional Layer in the network.
\begin{table}[]
    \caption{Convolutional Layer characteristics for Res-TCN \cite{3D_skeleton}}
    \centering
    \scriptsize

    {\def1} \begin{tabular}{*{1}{c}|*{1}{c}*{1}{c|}*{1}{c}*{1}{c}*{1}{c}*{1}{c}{1} \begin{tabular}{*{1}{c}||*{1}{c}|*{1}{c}|*{1}{c}|*{1}{c}|*{1}{c}|*{1}{c}}

                    & type    & type   & type & type  & type  & type  \\
                    &   1     &  2     &   3  &   4   &   5   &   6   \\
    \hline
                $ input features  $ & 150 & 64 & 64   & 128   & 128   & 256   \\
                $ output features $ & 64  & 64 & 128  & 128   & 256   & 256   \\
                $ Kernel\_size   $ & 8   & 8  & 8    & 8     & 8     & 8     \\
                $ stride  $ & 1   & 1  & 2    & 1     & 2     & 1     \\
    \end{tabular}}
    \label{tab:skeleton_char_perf}
\end{table}

Figure \ref{fig:effic_time_3D_hum} shows efficiency and execution time on this benchmark, highlighting contributions of the single layers.
Behaviour is similar to the ECG use case on both boards. Executing the processing after each sample ($B=1$), the efficiency is very limited for all the layers. Increasing $B$, the accelerator provides much better throughput. The layers executed less efficiently are the early layers, especially the first type, that pays for a significant underutilization of the MAC Matrix, due to the low number of input features. However, its contribution to the overall efficiency is marginal. Layers that show longer execution time are placed at the end of the TCN graph. These layers account for the highest contribution to the overall workload and, as shown in Figure \ref{fig:time_3D_hum} and \ref{fig:time_3D_hum_ultra}, are executed quite efficiently when sample batching is used ($B=144$). Sample batching is more effective on the Zedboard, that integrates a MAC Matrix with less input and output ports, thus requires longer runs to complete convolution, which reduces the impact of data-transfer and accelerator warm-up overheads.

\begin{figure}[h]
    \subfloat[\label{fig:effic_3D_hum}]{\includegraphics[width = \columnwidth, height = 0.55\columnwidth]{img/effic_3D_human.pdf}}
    \hfill
    \subfloat[\label{fig:time_3D_hum}]{\includegraphics[width = \columnwidth, height = 0.55\columnwidth]{img/time_3D_human.pdf}}
    \caption{Efficiency trend and Execution Time on Res-TCN \cite{3D_skeleton} for different batch sizes for NEURAghe 12x4 and 11x5 matrix configuration in Z-7020 SoC}
    \label{fig:effic_time_3D_hum}
\end{figure}

\begin{figure}[h]
    \subfloat[\label{fig:effic_3D_hum_ultra}]{\includegraphics[width = \columnwidth, height = 0.5\columnwidth]{img/effic_3D_human_ultra.pdf}}
    \hfill
    \subfloat[\label{fig:time_3D_hum_ultra}]{\includegraphics[width = \columnwidth, height = 0.5\columnwidth]{img/time_3D_human_ultra.pdf}}
    \caption{Efficiency trend and Execution Time on Res-TCN \cite{3D_skeleton} for different batch sizes for NEURAghe 9x10 matrix configuration in ZU3EG SoC}
    \label{fig:effic_time_3D_hum_ultra}
\end{figure}

\subsection{WN-PNT use-case}
This case considers a network derived from WaveNet (Van Den Oord et al. \cite{wavenet}) for Polyphonic Note Transcription of Time-Domain Audio Signal (\cite{polyWaveNet}). It is made of $20$ stacked residual blocks composed by a $1\times 1$ skip connection and a Dilated Convolutional block of $128$ channels each. \emph{Filter Size} is $2$ and the \emph{dilation factor} grows like $2^{k}$ where the residual block index $k \in \{0,1,2 ... 9,0,1,2 ... 9\}$.

\begin{figure}[h]
    \subfloat[\label{fig:effic_poliph_ultra}]{\includegraphics[width = \columnwidth, height = 0.5\columnwidth]{img/effic_poliph_ultra.pdf}}
    \hfill
    \subfloat[\label{fig:time_poliph_ultra}]{\includegraphics[width = \columnwidth, height = 0.5\columnwidth]{img/time_poliph_ultra.pdf}}
    \caption{Efficiency trend and Execution Time on WN-TCN \cite{polyWaveNet} for different batch sizes for NEURAghe 9x10 matrix configuration in ZU3EG}
    \label{fig:effic_time_poliph}
\end{figure}

This case is especially challenging, due to the small size of kernels ($kernel\_size=2$ in most layers) and because the sample acquisition frequency is $16KHz$, posing hard constraints about real-time execution. Thus we focus on the Ultra-96 board, relying on its higher clock frequency to achieve the required throughput.
As may be noticed in Figure \ref{fig:effic_poliph_ultra}, in general, this use case is executed less efficiently on the platform, none of the layers reaches more than 0.56 efficiency. This is because the execution of the actual convolution kernels takes only two cycles and thus hardly overlaps with input data transfers. Batch size can be used to increase the reuse of weights, once they are loaded to the weight memory region, but, at the same time, has an impact on the duration of input and output transfers. To compensate this issue, considering that the total size of the local receptive fields in the network allows for the continuous storage on the on-chip memory for almost all layers, (unless for types 8, 9 and 10), we have slightly modified the scheduling described in Figure \ref{fig:scheduling}. In this use-case, we transfer to/from DDR only new output/input samples, while keeping the rest of the local receptive fields in the on-chip memory. As shown in Figure \ref{fig:time_poliph_ultra}, using $B=504$ we can process 504 samples in around $19 ms$, thus respecting the real-time constraint posed by the use-case.

\subsection{Assessment of hardware vs. software speed-up}
In order to evaluate the benefits obtained by on-FPGA acceleration, we have compared the execution of the convolution-related load of the three benchmarks on NEURAghe, with the execution on an A53 quad-core ARM processor, using the ARM Compute Library optimized functions \footnote{Since ARM-CL does not support dilated 1D convolution, for the sake of comparison, we have evaluated execution time on the A53 on analogous convolution layers, without dilation. Considering that in dilated convolutions input samples are not adjacent in memory, the exploitation of the vector processing in the ARM-CL could be suboptimal, thus the execution time reported in Figure \ref{fig:neu_a53_time} for the A53 could be underestimated. Our own software implementation (unoptimized) of a dilated convolution performs one order of magnitude slower than what reported for ARM-CL.}. As may be noticed, NEURAghe provides up to 10.8x, 10.7x and 5.7x speed-up on the three reference benchmarks, and considering the power consumption of the two platforms (around $0.9\ W$ for the A53 cores and around $3.3\ W$ for NEURAghe), PL-based acceleration provides improvement, in terms of power efficiency, corresponding to 33,8 GOPS/s/W , 26,3 GOPS/s/W and 15 GOPS/s/W respectively.

\begin{figure}[h]
    \subfloat[\label{fig:neu_a53_time}]{\includegraphics[width = \columnwidth, height = 0.5\columnwidth]{img/NEUvsA53_time.pdf}}
    \hfill
    \subfloat[\label{fig:neu_a53_watt}]{\includegraphics[width = \columnwidth, height = 0.5\columnwidth]{img/NEUvsA53_gopss_watt.pdf}}
    \caption{Execution time and power efficiency comparison  between software execution on a Cortex-A53 quad core and NEURAghe (ultra-96).}
    \label{fig:neu_a53}
\end{figure}

\subsection{Comparison to other APSoC based accelerators}
As mentioned, literature is lacking TCN evaluations on FPGA-based accelerators. The only attempt available is \cite{waveFPGA}, which is strictly customized for a autoregressive TCN and is implemented on a high-end board non usable in the embedded domain. However, when designing the TCN-supporting version of NEURAghe, we have considered compatibility with 2D convolutions for classic CNN acceleration. Moreover, the improved flexibity of the architecture, in terms of kernel size and stride, allows for some improvements with respect to other approaches in the literature targeting low-to-mid All-programmable SoCs.
In Table \ref{tab:comp_previous}
we report comparative results with state-of-the-art on two well known networks for image classification, ResNet-18 and VGG-16. We compare to other accelerator architectures, \cite{neuESL}, \cite{sharma2016high} and \cite{venieris2017latency}, that are implemented on the same kind of hardware and use the same 16-bit data precision.
On VGG-16, that exposes quite regular kernel sizes and stride values, our work shows comparable performance with respect to the alternatives. It executes convolutions slightly faster than \cite{neuESL} and \cite{sharma2016high} and around 13\% slower than \cite{venieris2017latency}. On ResNet-18, which exposes more  variable $kernel\ sizes$ and $strides$, we can reduce a lot of overheads that must be payed by more \emph{static} architectures, executing the whole convolution workload 40\% faster than \cite{neuESL}.
\begin{table}[h]
    \caption{Comparison between this work, previous version (\cite{neuESL}) and other works on ResNet-18 and VGG-16. Xilinx Zynq Z-7020}
    \centering
    \scriptsize
     {\def1} \begin{tabular}{*{1}{c}|*{1}{c}*{1}{c|}*{1}{c}*{1}{c}*{1}{c}*{1}{c}{1} \begin{tabular}{*{1}{c}|*{1}{c}*{1}{c|}*{1}{c}*{1}{c}*{1}{c}*{1}{c}}
                & \textbf{This}   &     & \textbf{This}  & &   &    \\
                & \textbf{Work}   & \cite{neuESL}  & \textbf{Work} & \cite{neuESL} &  \cite{venieris2017latency} &  \cite{sharma2016high} \\
    \hline
                & \multicolumn{2}{c|}{ResNet-18} & \multicolumn{4}{c}{VGG-16}   \\
    \hline
            Xilinx Zynq SoC & \textbf{Z-7020} & Z-7020   & \textbf{Z-7020} & Z-7020 & Z-7020 & Z-7020 \\
            Freq. [MHz]     & \textbf{120}    & 120      & \textbf{120}    & 120    & 125    & 150 \\
            GOPS/s          & \textbf{26,5}  & 16,1    & \textbf{42,62}  & 42.48  & 48.53  & 31.38 \\

    \end{tabular}}
    \label{tab:comp_previous}
\end{table}

\section{Conclusions and future work}\label{sec:concl}

In this work, we have presented an accelerator architecture supporting Temporal Convolutional Networks, implemented on FPGA-based SoCs. We have presented the accelerator features serving this specific computing pattern, such as the capability of supporting arbitrary \emph{kernel\_size}, \emph{dilation rate} and \emph{stride values} without overhead.
We have also shown how data transfers from-to off-chip memory can be managed in TCNs and how performances improve by changing the computational paradigm, going from a \emph{latency constrained} approach to a \emph{batched} approach, that trades latency for throughput.
We applied this method to three notable TCNs and using two different SoCs as a target, analyzing throughput, latency and efficiency figures, and the capabilities of the system to respect real-time constraints.
Results show that using sample batching, we can achieve efficiency up to 0.96, 0.86 and 0.57 on the three use-cases. In the two use-cases with real-time requirements, an adequate sample batching can be used to achieve sufficient throughput to timely process all input samples.
We also compared the execution of the use-cases on a Cortex A53 quad-core, to evaluate the achievable speedup, noticing up to 10x execution time reduction, with an improvement in power efficiency that ranges from 2x to 3.5x depending on the benchmark.
Finally, we validated the compatibility of proposed architectural solutions with state-of-the-art CNNs used in the image processing domain, by direct comparison with previous work on APSoC-based accelerator, showing similar performance with respect to CNN-targeting alternatives when regular network topologies are targeted and 40\% improvement when targeting more irregular patterns.
\balance
\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


A classical Italian seasoning for Bruschetta and pasta sauces.
Borgo de Medici ORGANIC BRUSCHETTA MIXED SPICE.
Perfect blend of Italian Organic ingredients: Garlic, Salt, Chilli Pepper, Oregano, Tomato ,Onion and Carrot.


text:
sed -i 's|DATABASE_HOST|'"${DBHOST}"'|g' src/main/resources/hello/server.properties

fw_depends java8 maven

mvn clean compile assembly:single
cd target
java -jar undertow-example-0.1-jar-with-dependencies.jar &

redpajama_set_name: RedPajamaGithub


text:
Congratulations to Dr Arlene Holmes-Henderson on becoming a Knowledge Exchange Fellow
The Faculty warmly congratulates Dr Arlene Holmes-Henderson on her selection as a TORCH Knowledge Exchange Fellow in 2020-2021, working with the Department for Education. This partnership allows officials with responsibility for humanities and languages curriculum policy in England to engage with Dr Holmes-Henderson's longitudinal research evidence on the likely benefits of teaching ancient languages in primary schools, given these are part of the national curriculum. In considering the long-term impact that the study of an ancient language can have on subsequent language study, this knowledge exchange partnership has the potential to inform future policy development on both the teaching of ancient and modern foreign languages in schools in England.
Via her Knowledge Exchange Fellowship, Dr Holmes-Henderson is sharing the strength of the existing evidence from her award-winning Classics in Communities project, and wider research, on the impact of learning ancient languages on KS2 children's cognitive development. The feedback, insight and questions from policy colleagues will directly contribute to Dr Holmes-Henderson's evolving research agenda, for example by making it more responsive to gaps in policy delivery.
Knowledge will be shared widely throughout this Fellowship. Dr Holmes-Henderson is the Outreach Officer of the Classical Association (the national subject association for Classics), she sits on the British Academy's National Languages Advisory Group alongside colleagues representing modern languages, she chairs the Classics Development Group, she is a member of the British Curriculum Forum, a member of the OCR Classics consultative forum and a Schools' committee member of the Roman Society. She aims to improve communication between these organisations as part of this Fellowship, and will help to connect the Department for Education with a more diverse network of stakeholders in language and humanities education.

redpajama_set_name: RedPajamaCommonCrawl


This project involved a new build extension to an existing house to accommodate a bar, gym and deluxe swimming pool. The scheme also included the conversion of garage into new living accommodation.
Gilling Dod Interiors worked closely with the client to achieve a high end specification throughout using quality fixtures and fittings which complimented and contrasted with the oak structure.


Weekend Short-Term Loans "Cashsland Com". We are looking into cash advance. It isn't that we need extra money all of the time but this past month has been hard for us. A friend told us to look into it. That is what we plan to do very soon to see if it is the right thing for us to do. You can get cash loans with low credit score by using Cashsland Com, and read reviews.
Seeking for Cashsland Com. Require Income Right this moment?. Certainly not Examine A bad credit score. 100% Approval inside 60 minutes. Earn money Today.
Cashsland Com, It is exactly what we want to do very soon to ascertain if this is the right thing for all of us to perform.
Whenever we could get any additional money just for this next month we need to be okay and are able to get back to normal. I needed for taking 14 days off from work as a consequence of a health problem therefore we got behind. Normally we have sufficient money to have our bills paid punctually.
We are hoping by using the www.MyCashNow.com advance loan we could pay down all of the bills which are due and after that pay it back later on. Like that nothing is late and that we don't need to bother about getting a little bit more behind on things. It is far from an excellent feeling to find out you will be behind.
Some time ago I lost my job so we did get behind. It took a while to acquire back from the mess we had been in. My lovely wife was working two jobs and this didn't even get us everything we needed. Luckily I did have a job ahead of the bank took anything away from us.
When we knew we might not get money because I had been sick we started to get worried. That is when we reached over to our friend and so they helped us find out about the money advance. When it had not been on their behalf we never might have known regarding this.
We plan to check out them this week. We would like to realize how to sign up for one and what it means when you are getting one. We are also interested in exactly how much we can easily get them for. We realize we will need to pay each of the cash back but we have been uncertain how that works. There is still a lot of research that we must do.
In terms of money I don't like to be stressed regarding it. I work tirelessly therefore i don't need to be but that isn't always possible. Some days are harder as opposed to others and in some cases an unexpected bill can throw us off. We now have learned to never worry excessive and focus of what we could do in order to fix things rather than worrying rather than making our situation better.


text:
Q: Tomcat error: Element type "Connector" must be followed by either attribute specifications, ">" or "/>" I have implied basic authentication on tomcat server and configured to have ssl encryption. Then my tomcat is not starting shows following error

Could not load the Tomcat server configuration at /Servers/Tomcat v7.0 Server at localhost-config. The configuration may be corrupt or incomplete.
  Element type "Connector" must be followed by either attribute specifications, ">" or "/>".

any help ?

A: I had the same problem when configuring a proxy connector in Jira after copying sample tag from their support page. Nothing wrong with the XML syntax according to my editor.
I solved it by going over the tag replacing all " with new " and all blanks with space. Looking at it in a hex editor to compare the difference between non-working and working config I see that the non-working one had had 20 09" (space tab) in the blank/space areas. The working one has only space.

A: Sounds like your server.xml file is syntactically incorrect. It is located in $CATALINA_BASE/conf. Simply check its XML syntax, there must be something wrong with the <Connector> element.
Try use an XML editor if you can't spot the mistake.

A: I had a similar problem. I missed a space between 2 attributes.
Changing
protocol="org.apache.coyote.http11.Http11NioProtocol"SSLEnabled="true"

to
protocol="org.apache.coyote.http11.Http11NioProtocol" SSLEnabled="true"

solved my problem.

redpajama_set_name: RedPajamaStackExchange


Planning the perfect spa honeymoon doesnt have to be as stressful as organizing the wedding. Here are some tips that will make planning a getaway perfect for newlyweds as relaxing as a trip to the spa!
Make a Check List: The first step to picking the perfect destination for your spa honeymoon is deciding what you do and dont want to do while celebrating your new marriage. Key things to consider are the size of the spa and the specialty of the spa. These are particularly important as it is a great way to narrow your initial search. It is important to decide whether or not you and your mate are interested in isolation or enjoying a more communal experience. Many getaway spas have private villas, like Sundara Inn & Spas, designed for ultimate privacy, while some resorts facilities include many rooms.
Spa Cuisine: is an important aspect to consider, as some luxury spas have restricted menus, and often limit or dont offer alcohol. Double check the spa menu to be sure that you wont be limited by a vegetarian diet or a no-booze zone.
Activites: Researching activities offered at a resort of spa is another great way to narrow down your search for the perfect post-wedding escape. While some newlyweds may want simply to relax, undisturbed on a beach or garden, others may want to supplement their spa treatments with activities like horseback riding, hiking or Pilates. Some resorts even offer courses in cooking or gardening. Similarly, attractions outside of the spa may be a draw for some couples. Readers Choice Award Winner, The Spa of Colonial Williamsburg, is just steps away from the Historic Area, a living history museum.
Couples Treatments: Many spas, particularly the romantic ones, offer treatments like the couples massage during which newlyweds can enjoy relaxing massages side-by-side simultaneously. This is a fantastic way to enjoy the spa with your partner. Another way to enjoy tranquility together is during hydrotherapy or other water treatments. Water treatments include dips in mineral baths and cold plunges, underwater massage, jets, hot tubs and the like. Water treatments encourage circulation through the use of hot and cold contrasts. It is a great way to lay back and unwind, while floating close to your mate, as most hydrotherapy or water treatment is co-ed. See a list of spas that offer couple"s massages.
Do Some Research: Check out Spafinders Readers Choice Award winners for Best for Romance. Readers Choice Awards are picked by spa-goers and are a great resource for picking a spa that fits whatever criteria is most important for you. In the case of picking a spa to honeymoon at, Spafinders Best for Romance list is your best bet for picking a spa that really sets the mood. View a list of all the Reader"s Choice Spas.


Our Set-Rite® Toilet Flange Extension Kits can correct flange elevations ranging from 1/4" to 1-5/8". With additional Set-Rite® Spacers a maximum of 2-1/4" in elevation can be achieved. These products can be used with existing PVC, ABS, and Stainless Steel or metal flange rings, as well as cast iron closet flanges. Eliminate leak paths with the only water-tight and air-tight flange extension product on the market.


text:
Alan Miller est un concepteur de jeux, cofondateur d'Activision et d'Accolade.

Biographie

Miller est diplômé de l'université de Californie à Berkeley, avec la spécialité génie électrique en 1973.

Notes et références

Liens externes

Concepteur américain de jeux vidéo
Programmeur de jeux vidéo
Personnalité d'Activision

redpajama_set_name: RedPajamaWikipedia


Interaction between biblical study and the practical work of the church receives attention in this book. The author seeks biblical perspective on the problem of racial conflict. In New Testament times, the deepest conflict between groups was that between Jews and Gentiles. Ephesians 2:11-12 summarizes this conflict and its reconciliation in Jesus Christ. The book traces the history of the passage's interpretation from the early church to the present in order to clarify the current situation. It illustrates the significance of biblical scholarship for the practice of ministry. William H. Rader of Dauphin, PA, is a retired United Church of Christ pastor.


text:
Volcano Erupts on the Island Where MH370 Debris Washed Up
Rocky PlanetBy Erik KlemettiAug 25, 2015 1:30 AM
Lava flows and lava fountain from the August 25 eruption of Piton de la Fournaise on Reunion Island.
Webcam capture - OBSERVATOIRE VOLCANOLOGIQUE DU PITON DE LA FOURNAISE
Piton de la Fournaise on Reunion Island has started erupting again---for the second time in the last two months. The small island in the Indian Ocean gained some notoriety after a piece of the missing Malaysian Airlines Flight 370 washed up on its shores, but prior to that, it was really the volcanism from the aptly named Piton de la Fournaise ("Furnace Peak") for which Reunion was best known. As is typical for Piton de la Fournaise, a new fissure vent opened overnight after a brief period of earthquakes (likely caused by the magma making its way to the surface) and lava flows have been issues from the fissure ever since (see GIF below). A fairly impressive lava fountain is the source of these flows that are centered at Piton de Bert on the main volcano. The lava flows are occurring far up the slopes of the volcano, so far from anyone living on the island itself, so they are really a spectacle rather than a major hazard.
#volcan#volcano#PitondelaFournaise 4ème éruption de l'année en cours/4th eruption of the year began pic.twitter.com/Sz652Dkk13
— CultureVolcan (@CultureVolcan) August 24, 2015
Right now you can get a great view of the eruption on the OVPF webcam pointed at Piton de Bert
, along with a bit of an obscured view from Piton de Basaltes
. If this eruption follows the script of many of Piton de la Fournaise's eruptive events, it might last a few days to a week before the supply of lava peters out again.
Earth's First Stable Dose of Oxygen Nearly Ended All Life
3 Facts to Know About The Strange Spinning of Earth's Inner Core
Volcanic Activity Isn't Increasing Across the Planet


Discussion in 'Tips/Tricks/Mods' started by clpetroff, Oct 8, 2007.
How can a *site/user's* web directory be set up so when their url is access ed the list of the files/folders are displayed instead of a home or index.htm file?
I am assuming .htaccess is to be in the web dir with the files to be displayed; and there is no ISP Config changes that need to be made.
Ieem to be missing something.
in the Apache Directives field in ISPConfig instead.
both brought the index.html page, and when index.html was removed brought up the error: You don't have permission to access / on this server.
Can you post the vhost configuration of web25?


text:
Catocala semirelicta hippolyta is a subspecies of moth of the family Erebidae first described by Strecker in 1874. It is found in the US state of California.

The wingspan is 72–76 mm. Adults are on wing from June to October depending on the location. There is one generation per year.

The larvae feed on Populus species.

References

External links
Species info

semirelicta hippolyta
Moths described in 1874
Moths of North America
Subspecies


Woodland Hills Mobile Home Park Michigan City is free HD wallpaper. This wallpaper was upload at April 20, 2019 upload by admin in .You can download it in your computer by clicking resolution image in Download by size:. Don't forget to rate and comment if you interest with this wallpaper.


text:
Watch Sex and Death 101
The film Sex And Death 101 is a film about Roderick Blank, an executive who is about to be married to his fiancé. At his bachelor party he receives an email with list of 101 women. He realized his fiancé was number 29. He thinks it is just a prank list until he meets Carlotta Valdez and sleeps with her which is number 30 on the list. He then realizes that the list is all his sexual encounters from the past to future. Roderick then cancels his wedding and starts to sleep with the ladies on the list. His friends convince him to bury the list. He then falls for Lester, a quirky veterinarian. However, she died. He digs back the list and continues his sleeping rampage. One day, Roderick gets shocked when he learns the last name on his list is Gillian de Raisx, a female vigilante better known as Death Nell. Death Nell seduces men then puts them into a coma. Roderick decides to abandon the list again. However, after an incident he ends up at a Catholic college. The women think he was sent there to deflower them. He tries to resist but fails in the end. He jumps from 82 to 100 in that single event. Roderick gets to thinking that he might not survive after Death Nell. He tries to sleep with another Gillian de Rasx in Australia. However, he decides that has to face the consequences of his doings and decides to find the real Death Nell. Roderick and Gillan meet over dinner. Gillian shares everything with Roderick, her profession, her marriage and her "side-job". She also tells Roderick that she is exhausted with her mission. In the end, they both took a sedative pill and lay on the bed with the words written on the wall 'The End'. In the epilogue of the movie, it revealed that they both survived the pills. They got married and have a child.
Own from $5.99
The film Sex And Death 101 is a film about Roderick Blank, an executive who is about to be married to his fiancé. At his bachelor party he receives an email with list of 101 women. He realized his fiancé was number 29. He thinks it is just a prank list until he meets Carlotta Valdez and sleeps with her which is number 30 on the list. He then realizes that the list is all his sexual encounters from the past to future.
Roderick then cancels his wedding and starts to sleep with the ladies on the list. His friends convince him to bury the list. He then falls for Lester, a quirky veterinarian. However, she died. He digs back the list and continues his sleeping rampage.
One day, Roderick gets shocked when he learns the last name on his list is Gillian de Raisx, a female vigilante better known as Death Nell. Death Nell seduces men then puts them into a coma.
Roderick decides to abandon the list again. However, after an incident he ends up at a Catholic college. The women think he was sent there to deflower them. He tries to resist but fails in the end. He jumps from 82 to 100 in that single event.
Roderick gets to thinking that he might not survive after Death Nell. He tries to sleep with another Gillian de Rasx in Australia. However, he decides that has to face the consequences of his doings and decides to find the real Death Nell.
Roderick and Gillan meet over dinner. Gillian shares everything with Roderick, her profession, her marriage and her "side-job". She also tells Roderick that she is exhausted with her mission. In the end, they both took a sedative pill and lay on the bed with the words written on the wall 'The End'.
In the epilogue of the movie, it revealed that they both survived the pills. They got married and have a child.
Sex and Death 101 is a 2007 comedy with a runtime of 1 hour and 40 minutes. It has received poor reviews from critics and viewers, who have given it an IMDb score of 5.9 and a MetaScore of 24.
Where to Watch Sex and Death 101
Sex and Death 101 is available to watch free on Tubi TV. It's also available to stream, download and buy on demand at Apple TV, Amazon, Google Play, YouTube VOD and Vudu. Some platforms allow you to rent Sex and Death 101 for a limited time or purchase the movie and download it to your device.
Comedy Romance Drama
Tanc Sade
Daniel Waters
Stream Movies Like Sex and Death 101
She's the Man
Drumline
Daddy's Little Girls


text:
Harriet Maltby: Sunshine is winning the day. Here's evidence that Britain's getting better – not worse
By Harriet Maltby
Follow @HarrietMaltby
WATCH: Corbyn challenges the Prime Minister's plan for social care
Darren Grimes: These devalued degrees are infantalising a generation of young people
Harriet Maltby is a Government and Economics Researcher at the Legatum Institute and a former Senior Parliamentary Assistant.
There are reasons to believe. So said David Cameron as he articulated a powerful vision of Conservative Britain in his 2009 Conference speech. A steep climb, "but the view from the summit will be worth it".
Since then, we have lost our sense of belief. Poor polling, defections, and the EU turn us on ourselves. Conservative Britain – one based on stronger families, personal responsibility, opportunity, innovation, and choice – is something that remains unrealised: a hope extinguished by the reality of coalition.
Except that it hasn't. The Legatum Prosperity Index charts prosperity in Britain from the dying days of the Brown Government in 2008/09 through to the record of the final acts of the Coalition. It shows that we have been too busy climbing to stop and take in the view. It gives us reasons to believe.
Britain is today more prosperous than it has been at any point in the past six years. This year we rank 13th, rising three places since last year. In crucial areas like education, opportunity, safety, and health we score higher today than we did in 2009. So we have also made progress towards the vision Cameron outlined.
Whilst our overall social capital has fallen slightly since 2009, our families remain strong. Marriage rates have declined from 53 per cent in 2009 to 47 per cent, though this is likely to increase as a result of same-sex marriage legislation. However, despite this decline, our families are as strong as ever, with 94 per cent reporting being able to rely on family in times of need, putting our society among the ten with the strongest families in the world.
There is evidence too that Britain has grown more philanthropic as the state has stepped back. Our rates of volunteering and charitable donation have both increased since 2009. The UK is today the third most charitable nation in the world with 74 per cent on average donating to charity in any given month. The Big Society may not yet be Big, but it is certainly growing.
Within our strong society, the Conservative-led government has not only improved opportunity, but has brought it to a greater number of people. Economic development, the harbinger of many opportunities, has grown more equal under every year of this government and is significantly more equal than it was in 2009. On a scale of 1-10 of uneven economic development, the UK scored 4.7 in 2009: today it is 3.6. That puts us among the 20 most equal in the world. This is reflected in perceptions of opportunity. Those reporting Britain to be a country where working hard gets you ahead has risen from 78 per cent in 2009 to 84 per cent.
Indeed, Britain has established its place as one of the best environments for entrepreneurship and opportunity in the world. It has always ranked in the top ten for this in the Prosperity Index, but this year posts its highest ever score and ranks 8th. Britain is the most connected nation in the world for internet. We have the third lowest business start up costs, which have more than halved as a percentage of GNI since 2009. Today it costs just £66 to start a business in Britain. We are also the world's 15th largest exporter of high-tech goods – higher now than at the last election and ahead of the USA, Japan, and Germany. Conceived of in Britain, started in Britain, made in Britain, but exported across the world.
Fundamental to opportunity and prosperity is of course freedom of choice. It is in this area where the Conservative vision has most been realised. Cameron has delivered on his promise to give people more choice over schools for their children. The free school movement has delivered this freedom. So, too, is choice being introduced into the NHS. Reforms to energy markets brings more information and choice to consumers. Innovation with personal budgets by Conservatives in local authorities such as Harrow bring choice and dignity into vital areas of our lives, such as social care.
On our civil liberties, the reversal of 42 days detention without charge, the scrapping of ID cards and control orders, and the passing of the Protection of Freedoms Act 2012 led the Economist to describe Britain as a "more liberal and principled country" than it was six years ago. It is in freedom and choice that the Conservative vision has most been realised. Our overall satisfaction with freedom of choice and civil liberties has increased from 70 per cent in 2008 to an impressive 91 per cent today. This is one reason why Britain now for ranks as one of the ten freest nations in the world. A far more fitting position for the nation of Magna Carta than the 18th position in which we languished in 2009.
There are reasons to believe, but there remains work to be done. Our economic pessimism is doing much to hold us back, despite recent impressive GDP growth. The UK ranks just 28th for its economy, largely as a result of economic perceptions. Despite some of the biggest falls in unemployment since records began, only 18 per cent think it a good time to find a job. Whilst better than the seven per cent in 2009, it is still the 106th lowest in the world, surpassed only by our confidence in our financial institutions (124th).
The Party last faced the electorate with a vision of what Conservative Britain might look like. We must look up from the polls, from towns such as Clacton and Rochester, and from disquiet over the EU and take a minute to look out across the Britain we have created. It may not be a view from the summit, but it is an impressive view nonetheless.
Britain is at her most prosperous since the eve of the financial crisis and the Prosperity Index shows us why. Freedom, opportunity, choice, innovation, and strong society. This is the Britain this government created. In this election, our job is to point to the view and articulate how we make it better still. We must have faith in our record. There are reasons to believe, it is time we shared them.
Big Society Charities Civil liberties David Cameron MP Education Employment free schools freedom gay marriage jobs Marriage Prosperity Schools Social Care Unemployment
12 comments for: Harriet Maltby: Sunshine is winning the day. Here's evidence that Britain's getting better – not worse


Have your best photos printed at Deep Sky Printing .
We use Kodak Color Metallic paper for an exquisite finish.
This page will provide you with information on over 7800 Deep Space Objects such as rise and set time, transit times, location at these times.
You are also provided the objects magnitude, diameter in Arc seconds and the distance from earth in thousands of light years as well as local sun rise and sun set times. You can select the objects by their Messier, NGC, Caldwell or IC numbers or by category or select the object from the list.
All these items are tailored to your location. So please provide you latitude and longitude. These values will be stored in a cookie so they will be available the next time you come to this page.
These values will be stored in a cookie for future use.
Help Support this site. Check out the Google ads below!


Scroll down to explore all 20 images uploded under Small Leather Accent Chairs's gallery and find brilliant inspirations and ideas. Enjoy!
this post was published by b4g under livingroom furniture on Thursday, March 30th, 2017 and tagged with small leather goods.


All acreages are estimates and are subject to revision with the final platting process.
All utilities in place and readily available.
Desirable mixed use CB zoning available.
Residential, retail, office space available.


text:
Die Droga wojewódzka 808 (DW 808) ist eine 36 Kilometer lange Droga wojewódzka (Woiwodschaftsstraße) in der polnischen Woiwodschaft Lublin, die Łuków mit Kock verbindet. Die Strecke liegt im Powiat Łukowski und im Powiat Lubartowski.

Streckenverlauf

Woiwodschaft Lublin, Powiat Łukowski
  Łuków (DK 63, DK 76, DW 806, DW 807)
 Świderki
 Helenów
 Bystrzyca
 Wojcieszków
 Hordzież
 Serokomla
Woiwodschaft Lublin, Powiat Lubartowski
 Annopol
  Kock (Kotzk) (S 19, DK 19, DK 48)

Weblinks

Droga wojewodzka
Woiwodschaftsstraße in der Woiwodschaft Lublin


text:
OpEdNews Op Eds 1/11/2015 at 2:03 PM EST
The Lash and the Sword: Hebdo, Hypocrisy and the Mob-Mind
By Chris Floyd (Page 1 of 1 pages) 3 comments
Reprinted from Empire Burlesque
Arthur Silber -- naturally -- has something of depth and insight to say about the Hebdo case, and the self-righteous mob mentality it has provoked -- on a worldwide, witless scale -- in the aftermath. As always, do yourself a favor and go read the whole thing. (Especially his startling and perceptive connection of the current mindset to the words of John Brown and Abraham Lincoln.) Meanwhile, here are just a few excerpts:
"Nothing can justify the Charlie Hebdo murders. All civilized people must condemn these murders absolutely and unequivocally.
"Endless variations of such proclamations have issued from almost everyone in recent days. These titans of virtue and proper thought offer their judgment as if its mere utterance embodies courage of ungraspable dimensions. Truly, moral giants walk among us.
"Genuine moral courage does not require the company of a mob. The contrary proposition states the truth of this particular matter: genuine courage forbids the company of a mob that includes almost everyone -- from all political leaders, including those who direct the operations of the most terrifying terrorist organization on Earth, which goes by the name 'the United States Government,' to every well-known writer, to public personalities of dubious intellect and questionable character, to the most sickeningly bigoted and hate-filled ignoramuses. ...
"But I know this: it is obscene that the Hebdo murders should be singled out for an orgy of spluttering condemnation and outrage when the West, led by the monstrous U.S. Government with able support from most European nations, routinely murders more innocents in a single day (and, often, in less than a single day) than were murdered in Paris. The United States commits its murders across the globe -- from Afghanistan, to Iraq, to Syria, to Libya, on through other countries in Africa, and Asia, and in every corner of the world. England and, yes, France, and other countries provide significant aid in this unending campaign of terror.
"I also know this: when the U.S. and its accomplices commit murderous acts of terrorism -- when the U.S. and its accomplices murder innocents -- with a regularity and on a scale that would be the envy of the most barbarous and bloodthirsty criminals in all of history, there will be resistance. 'Nothing can justify the Charlie Hebdo murders.' Nothing? This is the voice of the master, the imperialist, the slaveowner, the sadist: 'We can bomb you, we can starve you, we can torture you, we can eviscerate you, we can visit every imaginable horror on you, we can utterly destroy you -- but you are forbidden to ever attack even one of us in any manner at all.'...
"I offer this not as a justification of the Hebdo murders, or as approval of violence. The obsession with 'justification' and 'approval' in this manner -- an obsession shared, it appears, by almost every semiconscious human being, who is breathlessly eager to tell us what he thinks of world events, on the assumption that masses of idiots can't wait to hear what one additional idiot thinks of it all -- is the mark of an arrested narcissistic adolescent, who still believes at the age of 30, or 40, or 50 or more, that the world, and history, require his approval to move forward.
"Today, our enemy is a campaign of terror that encompasses the world. Do I desperately hope for a far better world, one that can be reached by only peaceful means? Of course. As I said in the earlier essay, I consider the recourse to violence to be always deeply tragic, even when it is thoroughly understandable. Today, when faced with an enemy more powerful than any the world has ever known -- when the West's ruling class continues to be ruthlessly intent on amassing ever more power and wealth, when it is determined to eliminate and murder all those who stand in its way, when there is no place on Earth to make oneself safe from the barbaric violence unleashed by the ruling class every minute, of every hour, of every day -- resistance which includes violence is not only understandable, but inevitable.
"Facts can be awful things. This is but one example, albeit on an unusually large scale, which makes the awfulness that much more terrifying. Facts do not ask for your approval, or for mine. Your unhappiness or fear will not cause them to dissolve.
"You may find comfort in the mob, with its gutter talk of 'justification' and what is 'approved,' and what kinds of resistance are permissible. Always remember: the mob that comforts you today will kill you tomorrow."
Chris Floyd Social Media Pages:
Chris Floyd is an American journalist. His work has appeared in print and online in venues all over the world, including The Nation, Counterpunch, Columbia Journalism Review, the Christian Science Monitor, Il Manifesto, the Moscow Times and many (more...)
Related Topic(s): Murder, Add Tags
Soft Machine: A Bright Ray of Hope for Bradley Manning
A Low, Dishonest Decade: New Details for the Iraq War Crime Mosaic
Blanking Bradley Manning: NYT and AP Launch Operation Amnesia
Obama's Selective Outrage: Rage Against Russia, Silence at Indian Injustice
CIA Ãœber Alles: The Spooky Loyalties of Evan McMullin
The Unplucked Eye: Sinister Machinations Behind Kentucky's Manufactured Martyr
"Bringing Rob Kall in as a consultant on making my business and its website more bottom-up was incredibly valuable. Rob's out-of-the-box member registration system suggestions led to a multi-leveled engagement process designed to maximally connect clients with the company, which took the company and the website to the next level. Rob's coaching in bottom-up thinking played a strong role in enabling me to sell my company for over a million dollars."
Don Brown, Founder, the International Wireless Industry Consortium (IWPC)


The headline and text of this story have been updated to remove misleading references to Mokriya's work with Path. A Mokriya representative initially claimed the company was the "mobile development studio behind blockbuster apps like Path" but in truth only consulted with Path on a BlackBerry app that was never released, both it and Path have confirmed.
Craigslist gets over 50 billion page views and over 60 million U.S. visitors every month — yet the classified site's design and mobile strategy appear stuck in the year 2000 (when it first expanded beyond San Francisco). Craigslist offers a barebones mobile view, and it's recently rolled out some mobile-friendly features like a map view and an image-heavy grid view. But it's never released its own smartphone or tablet app, and its API is closed.
Plenty of companies have stepped in to try to fill the void, and the most recent of those is Mokriya, a consulting company that has worked on apps for companies like Hipster.
Mokriya's new Craigslist app for iPhone (s AAPL) and Android (s GOOG), which launches Wednesday morning, and is called Mokriya Craigslist, joins third-party Craigslist apps like Craigslist Mobile and c•Mobile. What sets Mokriya Craigslist apart, the company says, is its easily navigable "two-tap" interface and the fact that it's officially licensing data from Craigslist. The basic version is free; a premium version that allows posting and other features is $0.99.
When you launch the Mokriya Craigslist app, you choose category and city. Listings are then presented in an image-heavy interface. "Browsing Craigslist should be as pleasurable as using Pinterest," Mokriya founder Sunil Kanderi told me.
To choose a new category, post a listing of your own or 'favorite' a listing; all you have to do is tap at the top of the screen. "We built this to solve the problem of having to browse through multiple categories," Kanderi said. Users can also create alerts so that they're notified when, say, an apartment that fits their criteria is listed. And the app can use GPS to identify listings near a user's location. Posting is also streamlined: A user writes a headline and description, chooses photos from his or her camera roll, adds price and category, and that's it. Browsing listings is free, but to post listings, 'favorite' listings or set alerts, you'll have to pay $0.99 for a premium version. I found a test version of the app smooth and nice to use — which sets it apart from some other unofficial Craigslist apps I've tried.
Actually just had a chance to download this app and by far it's the best I've seen!
Any word as to why Craigslist hasn't created their own app?


COST - we ask that everybody attending the gathering contributes £20 - 30 depending on income (more is welcome if you can afford it) This goes directly towards the costs incurred in putting on the Gathering, which is run entirely by volunteers, none of whom get in "for free". If you genuinely cannot afford to pay this, talk to one of this year's organisers, we don't want to turn anyone away. Payment on arrival at the gate tent.
FOOD - food tickets will be available from the Gate tent at the cost of £6 per day. This will cover Breakfast, Lunch and Dinner, plus tea and coffee through the day. All food is deliciously vegan, and organic / locally grown where possible. Please contact before the start of the Gathering us if you have any food allergies that the kitchen should know about, so that the kitchen crew can arrange the menu accordingly. You are welcome to cater for yourself if you prefer, but care should be taken with cooking equipment, and we ask that you do not use barbeques or open fires because of the fire risk.
Children will be catered for separately at the Kids kitchen, where suitable nutritious lunches and dinners will be provided. The Kids food will cost £3 daily. Parents are asked to provide in-between snacks, but cold drinks will be available throughout the day.


Le Parchemin, at Montreal, never hesitates to get you the most innovative and high quality selection of hand bags, crossbody bags, travelling bags, wallets and leather goods for men and women. Our large array of products includes such big names like Mancini, MATT & NAT, Lug and more others.
Our leather goods' section personnel will gladly help you choose the ideal product within our hundreds of items available at our store.


text:
All Six Circuits to Return in New 24H SERIES ESPORTS Season
One year after the biggest shake-up in the history of the NEO Endurance Series, the second season of the 24H SERIES ESPORTS will have a much more familiar feel, as all six circuits on last season's calendar will return to the schedule, which runs from…
TCR Recap: CraigSetupShop the Last Leader Standing at a Contact-Packed Imola
Just when it seemed like the TCR class in the 24H SERIES ESPORTS championship was becoming too one-sided, qualifying for last Sunday's 6H IMOLA brought the most closely matched field so far this season. Just 0.7 seconds separated 13 of the 14 qualifiers, with the…
991 Recap: Four-za MSI! Porsche Points Leaders Win in Imola
Earlier this season, a near-crash for MSI eSports in the first round at Spa-Francorchamps seemed to show that even top 991-class teams weren't immune to disaster, especially while racing through the often-hectic mix of GT3 and TCR traffic. But after three wins and no more…
GT3 Recap: Williams Works Its Magic at Imola
Just three weeks ago at Silverstone, it looked like Williams Esports may have lost their early-season magic. An early spin at Stowe corner and a mid-race tow to the pits left them in an also-ran eighth place rather than in winning contention. But with their…
Race Preview: 6H IMOLA
Corey Davis, 3 years ago 10 min read
The new year brings a new venue for the 24H SERIES ESPORTS. In its sixth season, NEO's endurance championship is set to visit the virtual counterpart of the Autodromo Internazionale Enzo e Dino Ferrari in Imola, Italy, for the first time. Before the field takes…