text:
\section{Introduction}

Plasma accelerators \cite{Tajima:1979, Chen:1985} enable the acceleration of charged particles over short distances due to their multi-GeV/m field gradients. Although great progress in terms of beam quality and stability has recently been achieved \cite{Lindstrom:2021,Kirchen:2021}, significant advance is still required to make plasma-accelerator-driven applications feasible. The Particle-in-Cell (PIC) method \cite{Hockney:1981,Birdsall:1985} is a reliable tool to simulate plasma acceleration, and PIC simulations play a major role in understanding, exploring and improving plasma accelerators.

Simulation of a multi-GeV plasma-based accelerator typically requires modeling sub-micron-scale structures propagating over meter-scale distances, hence full electromagnetic PIC simulations require millions of time steps due to the Courant-Friedrichs-Lewy (CFL) condition \cite{Courant:1928}, which makes them unpractical. Several methods were developed to circumvent this limitation and enable larger time steps, including running PIC in a Lorentz-boosted frame~\cite{Vay:2007} or using a quasi-static approximation~\cite{Sprangle:1990a,Mora:1996,Whittum:1997}, both of which have proved performant for modeling of high-energy plasma accelerator stages~\cite{Huang:2006,An:2013,Mehrling:PPCF:2014,Vay:2021}.

Besides algorithmic improvements, further speedup can be accomplished from hardware improvement. Accelerated computing is growing in popularity in the supercomputer landscape~\cite{Top500HPC:2020}, and in particular using GPUs (Graphics Processing Units) as accelerators enabled significant speedup in High-Performance Computing (HPC) applications including PIC~\cite{Bussmann:2013,Myers:2021}. The heterogeneity of processor architectures in HPC makes it difficult to maintain a portable codebase but, following modern HPC practices, this challenge can be efficiently addressed with performance-portability layers~\cite{CarterEdwards:2014,Beckingsale:2019,Zenker:2016}.

In this article, we present the portable, three-dimensional, open-source, quasi-static PIC code HiPACE\texttt{++}\footnote{\url{https://github.com/Hi-PACE/hipace}}~\cite{hipacepp}. HiPACE\texttt{++} is written in C++ and is built on top of the AMReX~\cite{Zhang:2019} framework, which provides field data structure, Message Passing Interface (MPI) communications, and a performance-portability layer. In particular, the quasi-static PIC algorithm was adapted to accelerated computing, and HiPACE\texttt{++} demonstrates orders-of-magnitude speedup over CPU implementations as well as near-optimal scaling up to hundreds of cutting-edge GPUs. These performances enable realistic simulations of $1024\times1024\times1024$ cells for 1000 time steps in less than two minutes on modern GPU-accelerated supercomputers. HiPACE\texttt{++} is a complete rewrite of the legacy C code HiPACE \cite{Mehrling:PPCF:2014}.

The article is organized as follows: Section~\ref{sec:algorithm} summarizes the well-known quasi-static PIC algorithm. The GPU-porting strategy is introduced in Sec.~\ref{sec:gpu}. Correctness of the code is demonstrated in Sec.~\ref{sec:correctness}. Sec.~\ref{sec:performance} presents performance results and a novel parallelization strategy improving scalability on accelerated platforms. Additional code features are highlighted in Sec.~\ref{sec:features}.

\section{The quasi-static particle-in-cell algorithm}\label{sec:algorithm}

In a plasma accelerator, a driver perturbs the plasma electrons (the ions, heavier, remain practically immobile) and drives an electron plasma wave. While the driver can be a laser pulse or a particle beam, we hereafter focus on the case of a particle beam (beam-driven wakefield acceleration) for simplicity, as this is what is currently implemented in HiPACE\texttt{++}. In the driver's wake, a witness beam of charged particles can be accelerated with a high field gradient. In most conditions (with the notable exception of witness beam self-injection), the driver and witness beams evolve on a time scale much longer than the plasma response~\cite{Esarey:2009}. The quasi-static approximation~\cite{Sprangle:1990b} (QSA) treats the beams as rigid when computing the plasma response at a given beam location, hence decoupling the beam and plasma evolutions. Under this approximation, the Maxwell equations take the form of Poisson equations, and the scheme is not subject to a CFL condition. Then, the time step is determined by the smallest betatron period of the beams, making it possible to use time steps orders of magnitude larger than conventional electromagnetic PIC~\cite{Mehrling:PPCF:2014}. The algorithm has two main parts: first, from the distributions of the beams, compute the plasma response (computationally expensive). Second, from the plasma fields, advance the beams by one time step (computationally cheap).

For a given distribution of the beams, the plasma response is computed in the co-moving frame defined by $\zeta = z - c t$, with $c$ being the speed of light in vacuum (the beams propagate in the $+z$ direction). A slice of unperturbed plasma is initialized ahead of the beams and pushed backwards along the $\zeta$ coordinate. At each longitudinal position, the wakefields are calculated as a 2D problem in the transverse plane. The 3D problem is then solved as $n_\zeta$ 2D transverse problems (called \emph{slices}), with $n_\zeta$ being the number of longitudinal grid points in the simulation domain. Figure~\ref{fig:qsa_algorithm} illustrates the algorithm. Once the fields in the whole 3D domain have been computed, the beams are advanced by one time step.
\begin{figure}
	\centering
	\includegraphics[trim={0 0 0 0},clip, width=3.375in]{porting_plot_single.pdf}
	\caption{Snapshot of the quasi-static PIC algorithm. The 3D simulation domain is calculated slice-by-slice in a loop over the longitudinal grid points from the head of the box to its tail. Only the beam particles, a 2D slice of plasma particles, and a few 2D slices of fields are required to determine the wake in the 3D simulation domain.}
	\label{fig:qsa_algorithm}
\end{figure}

From Maxwell's equations and the quasi-static approximation, the following field equations can be derived \cite{An:2013}.
The wake potential $\psi = \phi - c\,A_z$, with $\phi$ and $\bm{A}$ being the scalar and vector potential respectively, is obtained from
\begin{equation}
\nabla^2_{\perp} \psi  = -\frac{1}{\epsilon_0}\left(\rho - \frac{1}{c}J_z \right),
\label{eq:psi}
\end{equation}
where $\epsilon_0$ is the vacuum permittivity and $\rho$ and $\bm{J}$ are the total (beams + plasma) charge and current densities, respectively. The transverse fields ${E_x-c\,B_y}$ and ${E_y+c\,B_x}$ are calculated from the transverse derivatives of $\psi$:
\begin{subequations}
\label{eq:exmby}
\begin{align}
    E_x - c\,B_y &= -\frac{\partial}{\partial x} \psi \, , \\
    E_y + c\,B_x &= -\frac{\partial}{\partial y} \psi.
\end{align}
\end{subequations}
The longitudinal field $E_z$ is obtained from
\begin{equation} \label{eq:ez}
\nabla^2_{\perp} E_z = \frac{1}{\epsilon_0 c} \nabla_{\perp} \cdot  J_{\perp} \,.
\end{equation}
The components of the magnetic field are given by
\begin{subequations}
\label{eq:bxby}
\begin{align}
\nabla_{\perp}^2 B_x  &= \mu_0(- \partial_y J_z + \partial_{\zeta} J_y) \,  ,\\
\nabla_{\perp}^2 B_y  &=   \mu_0(\partial_x J_z - \partial_{\zeta} J_x) \, ,
\end{align}
\end{subequations}
and
\begin{equation}
 \nabla^2_{\perp} B_z = \mu_0(\partial_y J_x - \partial_x J_y),
\label{eq:bz}
\end{equation}
where $\mu_0$ denotes the vacuum permeability. All quantities except the longitudinal derivatives $\partial_\zeta J_x$ and $\partial_\zeta J_y$ are directly accessible after the current deposition. These derivatives can be obtained with a predictor-corrector loop~\cite{Huang:2006,Mehrling:PPCF:2014} or by explicit integration~\cite{Wang:2017,Wang:2020}. Both options are available in HiPACE\texttt{++}, hereafter referred to as predictor-corrector or explicit method respectively, and their implementations are described in Sec.~\ref{sec:implementation}. The procedure to calculate fields at slice $\zeta$ given fields at slice $\zeta+\Delta\zeta$ (where $\Delta\zeta$ is the longitudinal cell size) is the standard 2D QSA PIC routine:
\begin{enumerate}
    \item Gather fields and push plasma particles backwards from $\zeta+\Delta\zeta$ to $\zeta$;
    \item Deposit plasma currents and densities;
    \item Deposit beam currents and densities;
    \item Solve equation~\eqref{eq:psi} for $\psi$ to calculate ${E_x-c\,B_y}$ and ${E_y+c\,B_x}$ with equation~\eqref{eq:exmby};
    \item Solve equation~\eqref{eq:ez} for $E_z$;
    \item Solve equation~\eqref{eq:bz} for $B_z$;
    \item Solve equations~\eqref{eq:bxby} for $B_{x/y}$.
\end{enumerate}

\section{Porting quasi-static PIC to GPU}\label{sec:gpu}

\subsection{Porting strategy}

HiPACE\texttt{++} is written considering modern GPU architectures with tens-of-GB global memory, relatively slow transfers between host (CPU) and device (GPU) memories, and fast atomic operations. The algorithm was designed to reduce host-device transfers whenever possible. As illustrated in Fig.~\ref{fig:qsa_algorithm}, the data that needs to be allocated for computation is modest, and only consists in the beam particles, a 2D slice of plasma particles and a 2D slice of grid quantities. In a vast majority of practical cases, these quantities fit in the global memory of a single GPU. This makes it possible to use single-GPU Fast Fourier Transforms (FFTs) that are considerably faster than single- or multi-CPU FFTs, which accounts for a significant fraction of the observed speedup.

In this implementation, the fields on a slice overwrite the previous values, so the fields in the 3D domain are not known at the end of the loop over slices. For this reason, the beam operations (field gather, particle push and current deposition) are also performed per slice within the loop over slices. To that end, beam particles are sorted per slice. Although this results in many small and inefficient kernels (a slice of beam particles contains $\sim$1000 particles for typical simulation parameters), the beam operations take a negligible amount of time overall. Finally, field data in the full simulation domain can be stored for the purpose of diagnostics. In that case, the required data is stacked in host (CPU) memory until the last slice is computed, and then flushed to disk.

\subsection{Implementation}\label{sec:implementation}

Performance-portability is achieved via the AMReX framework~\cite{Zhang:2019}. The most time-consuming functions in the quasi-static PIC method are the field solver, the plasma particle pusher and the plasma current deposition. Particle operations in the PIC method were demonstrated to benefit from GPU computing~\cite{Burau:2010,Myers:2021}, and the same methods as Ref.~\cite{Myers:2021} were applied (in particular, the current deposition relies on fast atomic operations on GPU).

For the \textbf{plasma particle push}, HiPACE\texttt{++} uses a fifth-order Adams-Bashforth particle pusher, as described in \cite{Mehrling:PPCF:2014}. The transverse beam position $x_\perp$, the transverse normalized momentum $u_\perp = \frac{1}{M c} \left( p_x, p_y \right)$, and the normalized plasma wake potential $\Psi_p = \frac{e}{m_ec^2} \psi_p$ of each plasma particle (with $M$ being the mass of the pushed plasma particle, $m_e$ the mass of an electron, and $e$ the elementary charge) are updated as follows:

\begin{align}
\partial_{\zeta}x_\perp ={}& -\frac{u_\perp}{1+\Psi_p } ,  \\[10pt]
\begin{split}\label{eq:1}
\partial_{\zeta}u_\perp ={}&  -\frac{q}{M}\bigg[ \frac{\gamma_p}{1+\Psi_p} \left(
\begin{array}{l}
   E_x - c B_y \\
   E_y + c B_x
\end{array}
\right)
+
\left(
\begin{array}{l}
   \,\,\,\,c B_y \\
   -c B_x
\end{array}
\right) \\ &
+
\frac{cB_z}{1+\Psi_p} \left(
\begin{array}{l}
   \,\,\,\,u_y \\
   -u_x
\end{array}
\right)
\bigg] ,
\end{split}\\[10pt]
\partial_{\zeta}\psi_p ={}&
-\frac{q m_e}{M e} \bigg[ \frac{1}{1+\Psi_p}
\left(
\begin{array}{l}
   u_x \\
   u_y
\end{array}
\right)
\cdot
\left(
\begin{array}{l}
   E_x - c B_y \\
   E_y + c B_x
\end{array}
\right)
- E_z \bigg] ,
\end{align}
with $q$ being the charge of the particle and $\gamma_p$ the Lorentz factor given by
\begin{equation}
\gamma_p = \frac{1 + u_\perp^2 + \left( 1+\Psi_p \right)^2}
                {2\left(1+\Psi_p \right)} .
\end{equation}
The beam particles are advanced by a second-order symplectic integrator. The field gather and particle push are embarrassingly parallel operations well-suited to GPU computing.

Due to the handling per slice, the \textbf{current deposition} is so far limited to zeroth order longitudinally for both plasma and beam particles, while orders 0-3 are available in the transverse direction. On GPU, the current deposition is performed using atomic operations to global device memory.

As can be seen in Eqs.~\eqref{eq:exmby},~\eqref{eq:ez} and~\eqref{eq:bz}, most fields are computed by \textbf{solving a transverse Poisson equation} and applying finite-difference operators. The Poisson equation with Dirichlet boundary conditions is solved by means of fast Poisson solvers~\cite{VanLoan:1992}, which are based on a Discrete Sine Transform (DST) of the first type. The DST is provided by the FFTW~\cite{Frigo:2005} library on CPU and by a custom implementation using FFTs~\cite{Cooley:1970} on GPU. The FFTs on GPU are provided by vendor libraries. The capability to run 2D FFTs on a single GPU instead of parallel FFTs on many CPUs is critical to provide good performance, considering that parallel FFTs require large amount of communications. Special care is needed to compute $B_{x/y}$, because of the longitudinal derivatives $\partial_\zeta J_x$ and $\partial_\zeta J_y$ in Eq.~\eqref{eq:bxby}. The $B_{x/y}$ field solver is usually the most expensive part of the 3D QSA PIC method.

Two options are implemented for the $B_{x/y}$ field solver algorithm. The first option is a \textbf{predictor-corrector field solver}, as implemented in the legacy code HiPACE. The longitudinal derivatives $\partial_\zeta J_{x/y}$ are evaluated on slice $\zeta$ from the previously-computed slice $\zeta+\Delta\zeta$ and slice $\zeta-\Delta\zeta$ still to be computed. An initial guess is made for $B_{x/y}$, with which particles are pushed from slice $\zeta$ to $\zeta-\Delta\zeta$ where their current is deposited. The current on slice $\zeta-\Delta\zeta$ is used to calculate $B_{x/y}$ at $\zeta$, and the procedure is repeated until a convergence criterion is reached or a maximum number of iterations is attained. Each iteration involves all PIC operations for plasma particles as well as several Poisson solves.

The second option for the $B_{x/y}$ field solver is an \textbf{explicit field solver} using analytic integration, as done in Ref.~\cite{Wang:2017,Wang:2020}. A 2D non-homogeneous Helmholtz-like equation must be solved (see equation (19) in Ref.~\cite{Wang:2020}), for which HiPACE\texttt{++} uses the GPU-capable multigrid solver provided by AMReX. The multigrid solver is an expensive operation, but does not required multiple iterations of PIC operations (although an implicit solver happens inside the multigrid solver).

\section{Correctness}
\label{sec:correctness}

The reference setup used throughout this article consists in a typical beam-driven wakefield acceleration simulation containing a driver beam and witness beam with Gaussian distributions with rms sizes $k_p \sigma_{\perp, d} = 0.3$, $k_p \sigma_{\zeta, d} = 1.41$ and $k_p \sigma_{\perp, w} = 0.1$, $k_p \sigma_{\zeta, w} = 0.2$, where $k_p = \omega_p/c$ is the plasma wavenumber, $\omega_p = \sqrt{n_0 e / (m_e \epsilon_0 )}$ is the plasma frequency, and $n_0$ the ambient plasma density (subscripts $d$ and $w$ stand for driver and witness, respectively).

The driver beam is located at the origin and has a peak density of $n_{b, d}/n_0 = 10$. The witness beam is centered around longitudinal position $k_p \zeta_{0, witness} = -5$ and has a peak density of $n_{b, w}/n_0 = 100$. The electron plasma is modeled with 4 particles per cell, and the background ions are assumed to be immobile. The simulation domain in $x$, $y$, and $\zeta$ is, in units of $k_p^{-1}$, ($-8,8$), ($-8,8$), and ($-7,5$) and uses $1024\times1024\times1024$ grid points. All simulation parameters and the used software are listed in the \hyperref[sec:simparam]{Appendix}. All simulations in this section ran on the JUWELS Booster, where each node is equipped with 2 \textsc{AMD EPYC} 7402 processors with 24 cores each and 4 \textsc{NVIDIA A100} GPUs (40\,GB, NVLink3) per node.

Figure~\ref{fig:Ez_comp} shows a comparison between HiPACE\texttt{++}, the legacy code HiPACE, and the full GPU-capable 3D electromagnetic PIC code WarpX~\cite{Vay:2021}. The accelerating field $E_z/E_0$, where $E_0 = c m_e \omega_p / e$ is the cold non-relativistic wave breaking limit, shows excellent agreement between these three codes. For HiPACE\texttt{++}, both the predictor-corrector and the explicit solver were used, and both demonstrate again very good agreement, as shown in Figure~\ref{fig:Ez_comp}(b). For WarpX, the rigid beams propagated in a uniform plasma long enough for the wake to reach a steady state. Minor differences in the witness beam region and in the spike at the back of the bubble can be attributed to different physical models, numerical methods, and initialization.

\begin{figure}
	\centering
	\includegraphics[trim={0 0 0 0},clip, width=3.375in]{Ez_comparison.png}
	\caption{(a) $x$-$\zeta$ snapshot of the electric field in a beam-driven wakefield acceleration simulation using WarpX (top) and HiPACE\texttt{++} (bottom). (b) Lineout of the accelerating field from WarpX, HiPACE, and HiPACE\texttt{++}. The inset shows a zoom on the witness beam region, where flattening of the accelerating field due to beam loading is visible. Both the predictor-corrector (pred.-corr.) and the explicit field solvers are shown.} \label{fig:Ez_comp}
\end{figure}

\section{Performance and parallelization}\label{sec:performance}

\subsection{Single GPU performance}

As discussed in Sec.~\ref{sec:gpu} and illustrated in Fig.~\ref{fig:qsa_algorithm}, the amount of data that must be allocated for a 3D domain is relatively modest and consists of beam particles and 2D slices of plasma particles and fields on the grid. Due to the relatively small size of beam data, the amount of data virtually depends only on the transverse number of cells. For example, the total allocated data on the global memory of a NVIDIA A100 GPU with the explicit solver (respectively predictor-corrector method) with 2 million beam particles (accounting for $\sim 230\,\mathrm{MB}$) and 1 particle per cell for the plasma electrons is 2.0 GB (resp. 2.0 GB) for a problem of $128\times128$ cells transversely, 2.9 GB (resp. 2.6 GB) for a $1024\times1024$ problem and 19.2 GB (resp. 12.9 GB) for a $4096\times4096$ problem size (for details on all simulation parameters see the \hyperref[sec:simparam]{Appendix}). Therefore, most practical problems fit on a single GPU, and the performance of HiPACE\texttt{++} on a single NVIDIA A100 GPU is detailed below.

The benefits of fitting the problem on a single GPU is clearly demonstrated in Fig.~\ref{FIG:singleGPUvsCPUs}. Typical CPU implementations of the 3D QSA PIC method~\cite{Mehrling:PPCF:2014,Wang:2020,An:2013} accelerate the calculation by decomposing the domain transversely, resulting in large amounts of communications (in particular in the Poisson solver) that dominate the runtime and cause non-ideal scaling. The CPU runs used only the 48 CPU cores on the nodes of the JUWELS Booster. The GPU runs also used the 4 GPUs. On GPU, the simulations at medium and high resolutions take $3.8$\,sec and $37.6$\,sec and cost $2.6 \times 10^{-4}$~node-hours and $2.6 \times 10^{-3}$~node-hours, respectively. For the same simulations using 1024 cores on CPU, HiPACE requires $17.5\,$sec and $556.1\,$sec for a cost of $0.10$~node-hours and $3.3$~node-hours. At medium resolution, the run on 1 (1024) CPU cores was $138\times$ ($4.5\times$) slower and cost $11.5\times$ ($388 \times$) more node-hours than on 1 GPU. At high resolution, the run on 16 (1024) CPU cores was $120\times$ ($14.8\times$) slower and cost $161\times$ ($1262\times$) more node-hours than on 1 GPU. The number of node-hours was calculated as [number of CPU cores]/48 for CPU runs, and [number of GPUs]/4 for GPU runs, as each node has 48 CPU cores and 4 GPUs.

\begin{figure}
	\centering
	\includegraphics[trim={0 0 0 0},clip, width=3.375in]{single_A100_vs_hipace_scaling.pdf}
	\caption{Performance comparison between single GPU and MPI-parallel many-CPU, for the same setup as Sec.~\ref{sec:correctness} for a single time step with medium ($512\times512\times1024$ cells, blue lines) and high ($2048\times2048\times1024$ cells, red lines) resolutions, with predictor-corrector field solver) on the JUWELS Booster. Simulations on CPU used HiPACE (MPI-parallel, dashed lines) and HiPACE\texttt{++} (OpenMP-parallel, dotted line). Simulations on GPU used HiPACE\texttt{++}. The high resolution run with HiPACE did not fit on less than 16 nodes on CPU.}
	\label{FIG:singleGPUvsCPUs}
\end{figure}

\begin{figure}
	\centering
	\includegraphics[trim={0 0 0 0},clip, width=3.375in]{transverse_scaling.pdf}
	\caption{Runtime for different transverse resolutions on NVIDIA A100 GPUs. Left bars: using the predictor-corrector loop. Right bars: using the explicit field solver. The runtimes of $2048\times2048$ and $4096\times4096$ transverse grid points were divided by a factor of 2 and 8, respectively, to improve readability of the figure.
	\label{FIG:transversescaling}}
\end{figure}

For further insight into the performance of HiPACE\texttt{++}, we ran the reference setup presented in Sec.~\ref{sec:correctness} with increasing transverse resolution, keeping all other parameters constant (for more details see the \hyperref[sec:simparam]{Appendix}). This scan uses 1024 longitudinal grid points, and performance data is given for both the predictor-corrector loop and the explicit field solver. The predictor-corrector loop used up to 5 iterations, so the runtime would be comparable between the two solvers.

The most time-consuming function of the two solvers are shown in Fig.~\ref{FIG:transversescaling}. In both cases a vast majority of the time is spent in solving for $B_{x/y}$. While both the fast Poisson solver and particle operations dominate the predictor-corrector solver at different resolutions, the multigrid solver is always the most expensive operation for the explicit solver. As a reminder, each iteration in the predictor-corrector loop involves all PIC operations for the plasma particles (field gather, particle push, current deposition and field solve) repeated up to 5 times per slice. Note that this study is not a comparison of the two field solvers, as they have different convergence properties, but rather a performance analysis of each solver separately.

Although CPU-only computing is not the main target of HiPACE\texttt{++}, shared-memory parallelization is implemented to enable transverse parallelization when running on CPU only, using OpenMP. In that case, tiling is implemented for plasma particle operations (field gather, particle push and current deposition), and the threaded version of FFTW can be called. As shown by the dotted line in Fig.~\ref{FIG:transversescaling}, the transverse OpenMP parallelization of HiPACE\texttt{++} gives a similar scaling as the pure MPI transverse parallelization of the legacy code HiPACE up to 16 threads (running on 16 cores of the 24-core JUWELS Booster CPUs).

\subsection{Longitudinal parallelization via temporal domain decomposition}
\label{sec:parallelization}

As presented in Sec.~\ref{sec:algorithm}, the computation of the fields in the full domain at a given time step relies on a loop from the headmost slice to the tail, and consequently cannot be parallelized longitudinally by standard domain decomposition. When computing multiple time steps, longitudinal parallelization can be achieved via pipelining algorithms~\cite{Feng:2009,Sosedkin:2016}, which were first realized in the form of a spatial decomposition~\cite{Feng:2009}. Because of the combination of (i) faster computation of each slice and (ii) using a single rank per slice, the standard spatial decomposition demonstrates poor scaling with our GPU implementation. We hereafter present a temporal domain decomposition, more suitable to the GPU implementation in HiPACE\texttt{++}. This implementation has some similarities with the streaming pipeline presented in Ref.~\cite{Sosedkin:2016}. Both pipelines are summarized below, assuming the  problem is decomposed longitudinally in as many sub-domains (\emph{boxes}) as the number of ranks $n_{ranks}$, and runs for \verb!nt! time steps.

In the spatial decomposition, each rank gets assigned one sub-domain, which it consecutively calculates for every time step. The algorithm (in pseudo-code) reads:
\begin{cverbatim}
# Rank r computes box b for all time steps
for t in 0:nt-1:
  Receive last slice from box b+1 at time t
  Compute box b at time t
  Send last slice to box b-1 at time t
\end{cverbatim}
where a slice consists in field data and plasma particle data. This pipeline is represented on the left of Fig.~\ref{FIG:new_pipeline_scheme}. The number of scalars communicated per time step and per rank reads $N_s = n_xn_y(S_{cell} + n_{ppc}S_{plasma})$ where $n_{x}$ ($n_y$) is the number of cells in the transverse direction $x$ ($y$), $S_{cell}$ is the number of scalars communicated per cell (in HiPACE\texttt{++}, $S_{cell}=6$ for $J_x$ and $J_y$ of the previous slice, and $B_{x/y}$ of the two previous slices) and $S_{plasma}$ is the number of scalars communicated per plasma particle (in HiPACE\texttt{++}, $S_{plasma}=35$ due to fifth-order Adams-Bashforth pusher). Here, $n_{ppc}$ is the number of plasma particles per cell. Each rank communicates a full slice, so the amount of data communicated does not scale with $n_{ranks}$.

In the temporal decomposition, each rank computes the full domain for the subset of time steps $t$ for which ${t \equiv r \pmod{nt}}$ (where $r$ is the current rank). The algorithm (in pseudo-code) reads:
\begin{cverbatim}
# Rank r computes all boxes for time step t
for b in nb-1:0:
  Receive beam from box b at time t-1
  Compute box b at time t
  Send beam to box b at time t+1
\end{cverbatim}
This pipeline is represented on the right of Fig.~\ref{FIG:new_pipeline_scheme}. The number of scalars communicated per time step and per rank reads $N_t = n_{beam,r}\times S_{beam}$ where $n_{beam,r}$ and $S_{beam}$ denote the number of beam particles on that rank and number of scalars communicated per beam particle, respectively. As can be seen in Fig.~\ref{FIG:new_pipeline_scheme}, $n_{beam,r}$ scales with the number of ranks (\textit{i.e.}, the number of sub-domains), so this pipeline should perform better for strong scaling.

As an example, let us consider a typical problem with $n_x = n_y = 1024$, $n_{ppc} = 1$, and $n_{beam,total}=2\times 10^6$. Even in the most favorable case (exchanging as few scalars as possible), $S_{plasma}=7$ for position, momentum and particle weight (HiPACE\texttt{++} uses 35), $S_{beam}=7$ and $S_{cell}=6$ to calculate the initial guess, the amount of data (assuming IEEE 754 \verb!double! precision) exchanged per rank and per time step is $110\,\mathrm{MB}$ for the spatial decomposition. Although this is usually not the case, we assume a load-balanced beam particle distribution across ranks for simplicity, so that $n_{beam,r} = n_{beam,total}/n_{ranks}$. The temporal decomposition exchanges roughly $110\,\mathrm{MB}/n_{ranks}$ per rank and per time step. For $n_{ranks}=256$ the temporal decomposition exchanges roughly two orders of magnitude less data than the spatial decomposition, and is hence expected to show better scalability.

\begin{figure}
	\centering
	\includegraphics[trim={0 0 0 0},clip, width=3.375in]{new_pipeline_visual_per_timestep.pdf}
	\caption{Left: spatial domain decomposition. Each rank calculates a fixed sub-domain for all time steps. Plasma particles and 2D field slices need to be communicated. Right: temporal domain decomposition. Each rank calculates the full domain for a sub-set of time steps. The beam particles of a sub-domain need to be communicated.}
\label{FIG:new_pipeline_scheme}
\end{figure}

The performance of the temporal decomposition pipeline is assessed via a strong scaling of two different setups. One setup is the reference simulation setup from Sec.~\ref{sec:correctness} with $n_{steps}=1000$ time steps and the other uses $2048\times2048\times 2048$ cells and 2048 time steps (for more details see the \hyperref[sec:simparam]{Appendix}). The efficiency $\eta$ is given by $\eta(n_{ranks}) = t(1)/[n_{ranks} t(n_{ranks})]$ where $t(i)$ is the run time on $i$ ranks. Due to the filling and emptying of the pipeline, the ideal efficiency for both pipelines is not identically 1 but rather given by
\begin{equation}
    \eta_{ideal}(n_{ranks}) = \left( 1+ \frac{n_{ranks} - 1}{n_{steps}}\right)^{-1}.
\end{equation}
An efficiency of $1$ is obtained in the limit of $n_{steps} \gg n_{ranks}$.
\begin{figure}
	\centering
	\includegraphics[trim={0 0 0 0},clip, width=3.375in]{new_pipeline.pdf}
	\caption{Strong scaling for two different problems with a) $1024\times1024\times1024$ cells with $4$ plasma particles per cell for 1000 time steps and b) $2048\times 2048 \times 2048$ cells with $1$ plasma particle per cell for 2048 time steps. Both settings used two beams with $10^6$ beam particles each. The final run time is given for the maximum number of ranks used. In b) the scaling starts at $4$ GPUs due to time limit restriction on the supercomputer. The problems are parallelized in the longitudinal direction only.}
\label{FIG:strong_scaling}
\end{figure}
The results are shown in Fig.~\ref{FIG:strong_scaling}. The temporal domain decomposition (red lines) shows an efficiency close to the ideal pipeline scaling (black dashed line). The spatial decomposition (blue lines) suffers from efficiency degradation above 8 ranks. Both scalings were performed on the JUWELS Booster and the reference setup was also run on Summit (red dotted line), which is equipped with 6 NVIDIA V100 GPUs per node. The maximum number of ranks is chosen so that only 4 slices remain per sub-domain, which was the case at 256 ranks ($=256$ GPUs) for the reference setup and 512 ranks for the higher-resolution case. The absolute run time between the temporal and spatial decompositions also vary due to significant performance enhancements not related to the parallelization in the runs with the temporal decomposition.

\section{Software practice and additional features}\label{sec:features}
HiPACE\texttt{++} is a versatile open-source, 3D, quasi-static PIC software with an object-oriented design to invite the integration of new numerical methods or physics packages. HiPACE\texttt{++} uses the cross-platform build system CMake and can be installed, as well as its dependencies, with software package managers, such as Spack~\cite{Gamblin:2015}.

HiPACE\texttt{++} complies with the openPMD standard~\cite{Huebl:2015} and uses the openPMD-api~\cite{Huebl:2018} for I/O, allowing for interoperability and simple benchmarking with other codes. Both HDF5~\cite{hdf5} and ADIOS2~\cite{Godoy:2020} file formats are supported (a feature inherited from the openPMD-api), and the capability to read an external beam from file at the openPMD format is available.

Two unit systems are available in HiPACE\texttt{++}, SI units and normalized units, available as a runtime parameter. In normalized units, all lengths are re-scaled to the plasma skin depth $k_p^{-1}$, the fields to cold, non-relativistic wave breaking limit $E_0$, and all densities to the background plasma density $n_0$.

\begin{figure}
	\centering
	\includegraphics[trim={0 0 0 0},clip, width=3.375in]{precision.pdf}
	\caption{(a) Evolution of the emittance during propagation over 3000 time steps of the witness beam presented in Sec.~\ref{sec:correctness} with an initial transverse offset of the witness bunch centroid of $x_b = \sigma_x$, for the two field solvers, in normalized units. The error due to single precision is computed for each field solver with respect to the double-precision simulation; (b) Same for a simulation running in SI units, where $k_p^{-1}=10\,\mu$m.}
\label{fig:precision}
\end{figure}
The code can be compiled in either double (C++ \verb!double!) or single (C++ \verb!float!) precision, a feature inherited from AMReX. The effect of the precision on the simulation accuracy is investigated by comparing the evolution of the emittance of the witness beam of the reference setup with an initial emittance of $\epsilon_{x,0} = 0$ when an initial transverse offset of the bunch centroid $x_b = \sigma_x$ is present in Fig.~\ref{fig:precision}.
For both predictor-corrector and explicit field solvers, the error attributed to using single precision remains well below 1\% (2\%) in normalized units (SI units) after 3000 time steps. Table~\ref{tab:precision} shows the runtime in single and double precisions for the two solvers on two different architectures, a cutting-edge HPC GPU (NVIDIA A100) and a typical consumer-grade ("\emph{gaming}") GPU (NVIDIA RTX2070), easily available on a laptop. As anticipated, double-precision calculations are much faster on the HPC GPU than on the gaming GPU. However, with the capability to run high-resolution production simulations on a gaming GPU with comparable accuracy and performance as on an HPC GPU in single precision, HiPACE\texttt{++} provides useful scalability from laptops to the largest supercomputers.
\begin{table}
\begin{tabular}{|c|c||c|c|c|}
 \hline
GPU & Solver & $T_{double}$ & $T_{single}$ & speedup \\
 \hline
 \hline
A100 & P-C & 16.6 s & 12.9 s & 1.3$\times$ \\
A100 & explicit & 23.8 s & 20.6 s & 1.2$\times$ \\
 \hline
RTX2070 & P-C & 99.1 s & 36.3 s & 2.7$\times$ \\
RTX2070 & explicit & 66.6 s & 27.8 s & 2.4$\times$ \\
 \hline
\end{tabular}
\caption{Runtime for 1 time step of the simulation presented in Fig.~\ref{fig:precision} on NVIDIA A100 and NVIDIA RTX2070 GPUs. P-C stands for the predictor-corrector loop. \label{tab:precision}}
\end{table}

Though in active development, HiPACE\texttt{++} features numerous capabilities useful for production simulations including multiple beams and plasma with different species and profiles (driver and witness beam, ion motion, etc.), the possibilities to load external beams and apply external fields as well as specialized mesh refinement capabilities~\cite{Mehrling:AAC:2019}. Field ionization using the ADK-model \cite{Ammosov:1986} is available in SI units and could readily be extended to normalized units. HiPACE\texttt{++} proposes different field solvers, and uses modern software practices to make it a user-friendly and stable code (continuous integration, open development repository, extensive documentation and comments). Planned upgrades include a laser envelope model~\cite{Benedetti:2017}, full mesh refinement, and the support of more GPU architectures from other providers. The code is fully operational on CPU and on modern NVIDIA GPUs, and simplified simulations are already possible on modern AMD GPUs.

\section{Conclusion}

This paper presented the open-source, performance-portable, 3D quasi-static PIC code HiPACE\texttt{++}. The main adjustments required to port the quasi-static PIC loop to accelerated computing consist in (i) ensuring that all operations, including the beam operations, are performed within the loop over slices so little data needs to be stored on device memory and (ii) proposing a different longitudinal parallelization (pipeline) with which the amount of data communicated per rank scales down with the number of ranks, enabling excellent scalability for production simulations up to hundreds of GPUs.

Focusing on runtime rather than scalability, HiPACE\texttt{++} is not MPI-parallel transversally: each slice is computed on a single GPU, enabling orders-of-magnitude speedup with respect to CPU implementations. Transverse parallelization will be considered if it provides significant speedup without impacting the code clarity. Benchmarks show excellent agreement with legacy code HiPACE and full electromagnetic PIC code WarpX.

HiPACE\texttt{++} is built on top of cutting-edge libraries (in particular AMReX and openPMD-api) to harness top performance-portability and encourage open science, while improving sustainability. It enables production simulations of plasma acceleration from laptops to supercomputers.

\section*{Acknowledgements}
The authors gratefully acknowledge helpful discussions with T.J.~Mehrling, C.B.~Schroeder, J.~Osterhoff, T.~Wetzel, and B.~Diederichs. We gratefully acknowledge the Gauss Centre for Supercomputing e.V. (www.gauss-centre.eu) for funding this project by providing computing time through the John von Neumann Institute for Computing (NIC) on the GCS Supercomputer JUWELS Booster at J\"ulich Supercomputing Centre (JSC). We acknowledge the Funding by the Helmholtz Matter and Technologies Accelerator Research and Development Program. This research was also supported by the Exascale Computing Project (No. 17-SC-20-SC), a collaborative effort of the U.S. Department of Energy's Office of Science and National Nuclear Security Administration. We acknowledge the support of the Director, Office of Science, Office of High Energy Physics, of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

redpajama_set_name: RedPajamaArXiv


One of the best ways to promote your tour today is through online channels. Our event ticketing software, Virtual Ticketer, makes it easy to handle all of your online traffic in one easy-to-use software solution. Now you don't have to worry about how to handle online transactions; not only will our platform even work with mobile devices, it also makes it easy to add links to purchase tickets for use on nearly any website.
If you are interested in driving more traffic to your website, here are three great ways you can promote your business on the internet. A great online presence is about more than just having a website, but using popular social media channels to drive more traffic to your website and converting those visitors into sales.
Pinterest is an often overlooked, but incredibly valuable, tool for online promotion for tours. People love to build boards on Pinterest filled with pictures of their dream vacations or ideas for an upcoming getaway. By putting gorgeous photos of your tour on Pinterest with links to your website, you'll get free marketing as people share your photo and add it to their vacation boards.
Facebook is a great place to connect with potential customers. While Facebook ads can be a great source of marketing, so can your business's Facebook page. The key is to post interesting, relevant content on your Facebook page that gets potential customers excited to learn more and maybe even share your content with their family and friends.
Twitter is all about engaging your followers. Twitter is a great place to share your business's personality and engage with people. There is a huge travel community that is actively engaged on Twitter, making it a perfect place for tour companies to connect and share.
No matter where your traffic is coming from or what method you are using to sell tickets to your tours, our event ticketing software can handle it. We make it easy to capture sales from websites, affiliate sales, walk-up sales and more. You'll be able to focus your attention on online promotions to drive more sales instead of compiling information from different sales platforms. Learn more about our software and learn how easy it is to integrate into your business today. Connect with our team today to get started.


This may seem like a simple question, but the answer is bigger than we think it is. We've all grown somewhat immune to the mocking of those in charge – you see the President of the United States and other world leaders mocked on TV and other media all the time. But really, what about the leadership of your company? Do you have confidence in them? Do you think they have your back? Do you feel like they're leading you, your co-workers … even your customers in the right direction?
Project Management Made Easier - Use Templates Everyone!
Having Trouble Closing the Project?
In my next installment of "5 Reasons Why" I'd like to refer you to Triskell-PPM Factory project portfolio management solution. Do you need a PPM or EPM? That may depend on your organization's size, client base, types of projects you undertake and sizes of projects you are engaged on. But if you think you may need such a tool – and most organizations of any size will benefit greatly from this – or if you already have a PPM tool, you owe it to yourself to check out Triskell's offering.
With this PPM solution you will be able to break any business strategy down into multiple portfolios and organizations. Flexibility and user-friendliness is at the forefront of their software design for this product.
With this PPM solution, you will be able to dynamically prioritize, fund and staff initiatives and actions according to strategic targets, as well as available budgets and resources. Everything is configurable and customizable for you, but the incredible out of the box functionality means you will end up with a great full-featured fit for your company's needs.
You can plan work and schedule your resources in a standard or agile mode. You won't be locked into waterfall or agile...the product will flex with your projects.
You will be able to dynamically control execution and progress on several axes and identify the potential gaps with the original targets. Getting a 1000 foot view of your projects as well as a 360˚ view means better information and, thus, better decision-making.
Re-align periodically with strategy and execution. Again, the product will flex with the changes and changing directions and needs of your organization – fast, efficiently and effectively – and not with a lot of redesign or configuration.
And keep in mind...these highlights are on top of what those familiar with PPM solutions would consider more standard PPM functionalities and highlights. Triskell-PPM Factory has all of those and more offering a clear strategic advantage.
Convinced yet? Don't let me stop you from moving forward or seeking more info on your own. The people behind the product are top-notch and very quick responders for information, help and support. Request a live demo and check it out for your self with the experts at Triskell. Remember, unlike other tools Triskell is an enterprise solution that is configured to meet the specific needs of your company, which makes it very easy to use and fit each user in many critical areas including project portfolio management and planning / resource optimization. Go to Triskell to learn much more about what they have to offer.


This update follows other recent changes by HRA, such as their policy update to no longer include gender markers on their Common Benefit Identification Cards. While all benefit cards do not have a gender marker as of Spring 2015, the prior update did not formalize a system to update your gender with HRA and the State.
Many of these changes would not have been possible without years of organizing by trans, gender non-conforming, and intersex (TGNCI) communities, and by all the allies and advocates who fought for this change. We got here after some notable attempts by HRA to affirm and assist TGNCI communities, especially with the creation of the LGBTQIA Liaison, which is led by former SRLP Survival and Self- Determination Project Director, Elana Redfield.
This new policy also is important because clients who are TGNCI will be given a Restriction/Exception Code to fix many billing issues. Since some treatments are coded by gender in insurance plans like Medicaid, members of the TGNCI community often face billing issues when obtaining the treatment they need. The Restriction/Exception Code helps to fix this problem! For example, a transgender or intersex person considered a male in the system can be covered for necessary OB-GYN appointments.
To request a gender change and the Restriction/Exception code, TGNCI folks should bring the appropriate documents to their local HRA benefit office. Clients who receive Medicaid through the New York State Department of Health must request the code and gender change through the New York State of Health Marketplace.
If you face discriination or have trouble updating your gender and name with HRA, please feel free to come to SRLP during our Legal Drop-in Clinic, currently Thursdays from 1:00-4:30 p.m. You may also e-mail HRA LGBTQIA Affairs at LGBTQI@hra.nyc.gov or report the issue to the Commision on Human Rights by calling 311 from any New York number.


Two life long mates, born a month apart - Zeus and Xena were with me for 7 years. I got them when they were 3.
Except for them I lived alone. They were my friends and companions, they both loved to chase a red dot laser, squirrels, rabbits, and ground hogs. Zena killed a skunk once and that was great fun - lol. I miss them a lot as they died last year, Xena in Nov and Zeus in Dec.


When diagnosed with a chronic or life-threatening illness not only do you have the shock of the diagnosis, but now you're required to make all these decisions about your care. Our age of technology sends most of us running to the internet for information and options and in many cases it helps and in others it increases the confusion. Do you think it's possible to discuss with your doctor all the options to come up with a feasible treatment plan? What if the doctor talked to you and not at you, would that make a difference? Wouldn't you rather be a partner in your care than have a plan super-imposed on you like clothing bought off-the-rack?
The only way you, the patient, should be making decisions about your own care is if you're thoroughly informed. Making off-the-cuff decisions isn't good for you and will leave you angry at the medical community. The internet may provide a foundation of information, but it can't go through a decision-making process for you so there has to be a better plan. Becoming an informed patient means having a dialogue with your doctor and coming to an understanding about the pros and cons of each treatment option and how the two of you can tackle this health challenge.
There is a group called "The Foundation for Informed Medical Decision Making" (http://www.informedmedicaldecisions.org) that focuses on how patients make decisions and is looking at outcome studies when patients are fully informed about their options. I've spoken in the past about the fact that if you go to a surgeon they are going to want to do surgery, as an example. Each provider has some stake in the treatment plan because it's their specialty, it's what they know, and it's how they know to solve the problem.
Business week had an article discussing ways to reduce healthcare costs and they shared a scenario from the Spine Center at Dartmouth-Hitchcock Medical Center. They said, "patients with back problems are show a video that walks them through various procedures and provides data showing that outcomes are similar whether or not they have surgery. Once the program started, spinal surgery rates dropped 30%." A 30% reduction in spinal surgeries, so how does the domino effect apply? Thirty-percent fewer spinal surgeries means 30% fewer chances for surgical errors that result in medical malpractice suits. Thirty-percent fewer spinal surgeries means less chance for the patient to acquire a hospital based infection like MRSA or Cdiff. These are big things to consider.
It doesn't matter what your diagnosis, informed decision-making is crucial to your well-being. It will empower you and that in and of itself increase your confidence in the treatment and the provider. When you feel empowered it increases your level of hope and that's important for your recovery. Informed decision-making means that your voice is heard and your place in this world as an individual, not a number is honored!


text:
The Kaleidoscope of Pageantry
The Kaleidoscope of Pageantry: Redefining Beauty!
The Pageant Guide
Miss Grand International 2016
Peru Awaits Homecoming of Maria Jose Lora Post Her Crowning as Miss Grand International 2017!
January 26, 2018 / Vagisha Mishra	/ 12 Comments
Peru Awaits Homecoming of Maria Jose Lora Maria Jose Lora, who was crowned as Miss Grand International 2017 in Vietnam last year in October, will be visiting her country, Peru first time post her crowning as Miss Grand International. The homecoming ceremony of the beauty queen will last for three days. Maria will reach Peru … Continue reading Peru Awaits Homecoming of Maria Jose Lora Post Her Crowning as Miss Grand International 2017!
Found it awesome? Share with your friends!
Ariska Putri Pertiwi from Indonesia is Miss Grand International 2016
October 26, 2016 October 26, 2016 / Vagisha Mishra	/ 20 Comments
Indonesia Wins Miss Grand International 2016 Ariska Putri Pertiwi from Indonesia was crowned as Miss Grand International 2016 on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. She succeeds Claire Parker from Australia as the new Miss Grand International and will now continue to serve as an ambassador of peace and goodwill to spread … Continue reading Ariska Putri Pertiwi from Indonesia is Miss Grand International 2016
Top 10 Best National Costumes of Miss Grand International 2016 Announced!
October 20, 2016 October 20, 2016 / Vagisha Mishra	/ 1 Comment
Meet The Top 10 Best National Costumes of Miss Grand International 2016 The fourth edition of Miss Grand International 2016 is scheduled to take place on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. At the conclusion of the pageant, Claire Parker from Australia will pass on her crown to … Continue reading Top 10 Best National Costumes of Miss Grand International 2016 Announced!
Miss Grand International Evening Gown Portraits: The Battle of Poise is On!
October 19, 2016 / Vagisha Mishra	/ Leave a comment
Miss Grand International 2016 Contestants Dazzle in Evening Gown Portrait The fourth edition of Miss Grand International 2016 is scheduled to take place on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. At the conclusion of the pageant, Claire Parker from Australia will pass on her crown to her successor. … Continue reading Miss Grand International Evening Gown Portraits: The Battle of Poise is On!
Miss Grand International Beauties Showcase Heritage in National Costume Round!
October 18, 2016 October 18, 2016 / Vagisha Mishra	/ Leave a comment
Miss Grand International 2016: National Costume Presentation The fourth edition of Miss Grand International 2016 is scheduled to take place on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. At the conclusion of the pageant, Claire Parker from Australia will pass on her crown to her successor. Last night the … Continue reading Miss Grand International Beauties Showcase Heritage in National Costume Round!
Miss Grand International 2016- Welcome Ceremony Shimmers in Las Vegas! (PHOTOS)
October 11, 2016 October 11, 2016 / Vagisha Mishra	/ 2 Comments
Miss Grand International 2016 Press Conference and Welcome Ceremony Kicks Off! The fourth edition of Miss Grand International 2016 is scheduled to take place on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. At the conclusion of the pageant, Claire Parker from Australia will pass on her crown to her … Continue reading Miss Grand International 2016- Welcome Ceremony Shimmers in Las Vegas! (PHOTOS)
Indonesia Impresses Again in National Costume Round at Miss Grand International 2016
October 11, 2016 / Vagisha Mishra	/ 1 Comment
Indonesia Sends a Dazzling National Costume at Miss Grand International 2016 The fourth edition of Miss Grand International 2016 is scheduled to take place on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. At the conclusion of the pageant, Claire Parker from Australia will pass on her crown to her … Continue reading Indonesia Impresses Again in National Costume Round at Miss Grand International 2016
Miss Grand International 2016- Departure & Arrival Pictures
October 9, 2016 October 10, 2016 / Vagisha Mishra	/ 3 Comments
Miss Grand International 2016 Delegates Arrive in Las Vegas The fourth edition of Miss Grand International 2016 is scheduled to take place on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. At the conclusion of the pageant, Claire Parker from Australia will pass on her crown to her successor. The … Continue reading Miss Grand International 2016- Departure & Arrival Pictures
VOTE NOW! Who Should Win Miss Grand International 2016?
September 21, 2016 October 25, 2016 / Vagisha Mishra	/ 559 Comments
Who Should Be The Next Miss Grand International?: People's Opinion in Spotlight! The fourth edition of Miss Grand International 2016 is scheduled to take place on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. At the conclusion of the pageant, Claire Parker from Australia will pass on her crown to … Continue reading VOTE NOW! Who Should Win Miss Grand International 2016?
Miss Grand International 2016: Meet The Contestants
September 21, 2016 September 21, 2016 / Vagisha Mishra	/ 10 Comments
Meet The Delegates of Miss Grand International 2016 The fourth edition of Miss Grand International 2016 is scheduled to take place on 25th October 2016 in Las Vegas, Nevada, USA at WestGate Las Vegas Resort & Casino. At the conclusion of the pageant, Claire Parker from Australia will pass on her crown to her successor. This year … Continue reading Miss Grand International 2016: Meet The Contestants
Happy Readers So Far!
Follow The Kaleidoscope of Pageantry on WordPress.com
How To Participate in Miss India: Your Step By Step Guide! August 1, 2020
BREAKING NEWS: Aishwarya Rai Tests Positive! July 12, 2020
Miss Universe Philippines 2020: Hotpicks and Predictions July 4, 2020
Miss South Africa 2020- Meet The Contestants! June 26, 2020
Pageant Talk With Melvyn Noronha June 9, 2020
Открыто голосование… on VOTE NOW! Who Should Win Miss…
Ioanna barnett on VOTE NOW: Who Will Win Miss US…
Beverly Spinosa on VOTE NOW: Who Will Win Miss Te…
Randy Joyce on VOTE NOW: Who Will Win Miss Te…
John Joseph Matthews on VOTE NOW: Who Will Win Miss Te…
Categories Select Category Contact Editorials Fashion Femina Miss India Interviews Make-up Tutorials Male Pageant Male Pageants Miss America Miss Diva Miss Earth Miss Earth India Miss Grand International Miss Intercontinental Miss International Miss Supranational Miss Supranational India Miss Teen USA Miss Universe Miss USA Miss World Miss World India News Other Pageants Poll The Pageant Guide Uncategorized
©Vagisha Mishra | The Kaleidoscope of Pageantry|

redpajama_set_name: RedPajamaCommonCrawl


text:
Stark County, Ohio
County in the United States
Stark County Courthouse
Location within the U.S. state of Ohio
Ohio's location within the U.S.
581 sq mi (1,505 km2)
5.3 sq mi (14 km2), 0.9%
Population (est.)
• (2018)
653/sq mi (252/km2)
Eastern: UTC−5/−4
www.starkcountyohio.gov
Stark County is a county located in the U.S. state of Ohio. As of the 2010 census, the population was 375,586.[1] Its county seat is Canton.[2] The county was created in 1808 and organized the next year.[3] It is named for John Stark, an officer in the American Revolutionary War.[4]
Stark County is included in the Canton-Massillon, OH Metropolitan Statistical Area, which is also included in the Cleveland-Akron-Canton, OH Combined Statistical Area.
2.1 Adjacent counties
2.2 Major Highways
2.3 National protected area
3.1 2000 census
5.1 Elected officials[19]
6.1 Colleges and universities
6.2 Community, junior, and technical colleges
6.3 Public school districts
6.3.1 High schools
7.1 Cities
7.2 Villages
7.3 Townships
7.4 Census-designated places
7.5 Other unincorporated communities
Stark County was named in honor of American Revolutionary War General John Stark. John Stark (August 28, 1728 – May 8, 1822) was a general who served in the American Continental Army during the American Revolutionary War. He became widely known as the "Hero of Bennington" for his exemplary service at the Battle of Bennington in 1777.
In the 1760s Moravian missionaries from Pennsylvania attempted to establish missions aimed at converting the native people. The earliest of these were Christian Frederick Post and John Heckewelder.
The first permanent settlements were established in 1805, beginning with Canton. Possibly 80% of the early settlers were German-speakers from Pennsylvania, although others came from Virginia, New York, and New England. Lumbering and sawmills were important early industries, to cater to the enormous demand for lumber from the incoming settlers. Stark County was originally part of Columbiana County, but was split off in 1807.
At the start of the Civil War the men of Stark County were quick to volunteer to preserve the Union. As of 1862 over 1,100 had enlisted.[5]
During the early 20th century, Stark County was an important location in the early development of professional football. The rivalry between the Massillon Tigers and Canton Bulldogs helped bring the Ohio League to prominence in the mid-1900s (decade) and again in the late 1910s. The Bulldogs ended up a charter member of the National Football League, where it played for several years. (The role Stark County had in developing the game is part of the reason the Pro Football Hall of Fame is located in Canton.) Two relatively large football stadiums, Fawcett Stadium in Canton and Paul Brown Tiger Stadium in Massillon, are still in use (albeit now mostly for high school football), with Fawcett Stadium hosting the NFL's annual Pro Football Hall of Fame Game each year.
In the later 20th century, Stark County's voting record swung from one party to another, closely tracking the winner of the U.S. Presidential election. Even within the swing state of Ohio, Stark County is regarded as a quintessential bellwether, and thus presidential candidates have typically made multiple visits to the region. Major media outlets typically pay close attention to the election results in the county. The New York Times in particular has covered the county's citizens and their voting concerns in a series of features each election cycle for over a decade.
According to the U.S. Census Bureau, the county has a total area of 581 square miles (1,500 km2), of which 575 square miles (1,490 km2) is land and 5.3 square miles (14 km2) (0.9%) is water.[6]
Adjacent counties[edit]
Portage County (north)
Mahoning County (northeast)
Columbiana County (east)
Carroll County (southeast)
Tuscarawas County (south)
Holmes County (southwest)
Wayne County (west)
Summit County (northwest)
Major Highways[edit]
National protected area[edit]
First Ladies National Historic Site
1910 122,987 29.8%
1940 234,887 5.9%
1990 367,585 −3.0%
Est. 2018 371,574 [7] −1.1%
U.S. Decennial Census[8]
1790-1960[9] 1900-1990[10]
1990-2000[11] 2010-2017[1]
2000 census[edit]
As of the census[12] of 2000, there were 378,098 people, 148,316 households, and 102,782 families residing in the county. The population density was 656 people per square mile (253/km²). There were 157,024 housing units at an average density of 272 per square mile (105/km²). The racial makeup of the county was 90.28% White, 7.20% Black or African American, 0.24% Native American, 0.54% Asian, 0.02% Pacific Islander, 0.29% from other races, and 1.43% from two or more races. 0.92% of the population were Hispanic or Latino of any race.
There were 148,316 households out of which 31.00% had children under the age of 18 living with them, 54.20% were married couples living together, 11.50% had a female householder with no husband present, and 30.70% were non-families. 26.10% of all households were made up of individuals and 10.90% had someone living alone who was 65 years of age or older. The average household size was 2.49 and the average family size was 3.00.
In the county, the population was spread out with 24.80% under the age of 18, 8.30% from 18 to 24, 27.80% from 25 to 44, 24.00% from 45 to 64, and 15.10% who were 65 years of age or older. The median age was 38 years. For every 100 females there were 92.40 males. For every 100 females age 18 and over, there were 88.40 males.
The median income for a household in the county was $39,824, and the median income for a family was $47,747. Males had a median income of $37,065 versus $23,875 for females. The per capita income for the county was $20,417. About 6.80% of families and 9.20% of the population were below the poverty line, including 12.90% of those under age 18 and 6.60% of those age 65 or over.
As of the 2010 census, there were 375,586 people, 151,089 households, and 100,417 families residing in the county.[13] The population density was 652.9 inhabitants per square mile (252.1/km2). There were 165,215 housing units at an average density of 287.2 per square mile (110.9/km2).[14] The racial makeup of the county was 88.7% white, 7.6% black or African American, 0.7% Asian, 0.3% American Indian, 0.5% from other races, and 2.2% from two or more races. Those of Hispanic or Latino origin made up 1.6% of the population.[13] In terms of ancestry, 33.6% were German, 15.5% were Irish, 10.1% were English, 10.1% were Italian, and 7.7% were American.[15]
Of the 151,089 households, 30.2% had children under the age of 18 living with them, 49.2% were married couples living together, 12.7% had a female householder with no husband present, 33.5% were non-families, and 28.1% of all households were made up of individuals. The average household size was 2.42 and the average family size was 2.96. The median age was 41.1 years.[13]
The median income for a household in the county was $44,941 and the median income for a family was $55,976. Males had a median income of $44,238 versus $31,896 for females. The per capita income for the county was $24,015. About 9.5% of families and 12.7% of the population were below the poverty line, including 19.5% of those under age 18 and 6.7% of those age 65 or over.[16]
Stark County formerly was a Republican Party stronghold, but since 1992 it has become a swing county that tilts Democratic. In 2016, Donald Trump won the county by the largest margin of any presidential candidate since Ronald Reagan in 1984.
Presidential election results
Presidential election results[17]
2016 55.9% 98,388 38.7% 68,146 5.5% 9,631
2004 48.9% 92,215 50.6% 95,337 0.5% 907
1996 38.0% 60,212 46.4% 73,437 15.6% 24,697
1980 55.9% 87,769 37.6% 59,005 6.6% 10,332
1964 37.7% 53,632 62.3% 88,704
1912 22.6% 6,033 37.1% 9,908 40.4% 10,788[18]
1904 65.5% 15,695 28.9% 6,919 5.7% 1,361
1892 44.8% 9,231 49.6% 10,227 5.6% 1,150
1888 47.3% 8,763 49.1% 9,094 3.6% 674
1872 52.3% 5,817 47.2% 5,250 0.5% 55
Government[edit]
See also: Ohio county government
Elected officials[19][edit]
Commissioners: Janet Weir Creighton, Bill Smith, Richard Regula
Auditor: Alan Harold
Clerk of Courts: Louis P. Giavasis
Judges of the Court of Common Pleas: Hon. Kristin Farmer, Hon. John G. Haas, Hon. Taryn L. Heath, Hon. Francis G. Forchione, Hon Chryssa Hartnett
Coroner: P.S. Murthy M.D.
Engineer: Keith Bennett
Family Court: Hon. Rosemarie Hall, Hon Jim D. James, Hon David R. Nist
Probate Court: Hon. Dixie Park
Prosecutor: John D. Ferrero
Recorder: Rick Campbell
Sheriff: George Maier
Treasurer: Alexander Zumbar
Colleges and universities[edit]
Kent State University at Stark
Stark State College
Walsh University
Community, junior, and technical colleges[edit]
R. G. Drage Career Technical Center
Public school districts[edit]
Alliance City School District
Canton City School District
Canton Local School District
Fairless Local School District
Jackson Local School District
Lake Local School District
Louisville City School District
Marlington Local School District
Massillon City School District
Minerva Local School District
North Canton City School District
Northwest Local School District
Osnaburg Local School District
Perry Local School District
Plain Local School District
Sandy Valley Local School District
Tuslaw Local School District
High schools[edit]
Alliance High School
Canton McKinley High School
Canton South High School
Central Catholic High School
East Canton High School
Fairless High School
GlenOak High School
Hoover High School
Indian River High School
Jackson High School
Jackson School for the Arts
Lake Center Christian School
Lake High School
Louisville High School
Marlington High School
Massillon Christian School
Minerva High School
Northwest High School
Communities[edit]
Map of Stark County, Ohio with municipal and township labels
Cities[edit]
Canal Fulton
Canton (county seat)
Villages[edit]
Beach City
East Canton
East Sparta
Hills and Dales
Limaville
Meyers Lake
Townships[edit]
Prior to 1815, Stark County consisted of only eight large townships.[20] After a number of partitions and a few transfers between counties, the townships are:
Nimishillen
Osnaburg
https://web.archive.org/web/20160715023447/http://www.ohiotownships.org/township-websites
Census-designated places[edit]
North Lawrence
Perry Heights
Richville
Robertsville
Other unincorporated communities[edit]
Middlebranch
North Industry
Pigeon Run
Sippo
National Register of Historic Places listings in Stark County, Ohio
^ a b "State & County QuickFacts". United States Census Bureau. Retrieved February 11, 2015.
^ "Find a County". National Association of Counties. Archived from the original on 2011-05-31. Retrieved 2011-06-07.
^ "Ohio: Individual County Chronologies". Ohio Atlas of Historical County Boundaries. The Newberry Library. 2007. Retrieved February 14, 2015.
^ "Stark County data". Ohio State University Extension Data Center. Retrieved 2007-04-28. [dead link]
^ Perrin, William Henry (1881). History of Stark County, with an outline sketch of Ohio. Chicago: Baskin & Battey. Retrieved June 26, 2019.
^ "2010 Census Gazetteer Files". United States Census Bureau. August 22, 2012. Archived from the original on May 4, 2014. Retrieved February 11, 2015.
^ "American FactFinder". Retrieved April 19, 2019.
^ "U.S. Decennial Census". United States Census Bureau. Archived from the original on April 26, 2015. Retrieved February 11, 2015.
^ "Historical Census Browser". University of Virginia Library. Retrieved February 11, 2015.
^ Forstall, Richard L., ed. (March 27, 1995). "Population of Counties by Decennial Census: 1900 to 1990". United States Census Bureau. Retrieved February 11, 2015.
^ "Census 2000 PHC-T-4. Ranking Tables for Counties: 1990 and 2000" (PDF). United States Census Bureau. April 2, 2001. Retrieved February 11, 2015.
^ "American FactFinder". United States Census Bureau. Archived from the original on 2013-09-11. Retrieved 2008-01-31.
^ a b c "DP-1 Profile of General Population and Housing Characteristics: 2010 Demographic Profile Data". United States Census Bureau. Retrieved 2015-12-27.
^ "Population, Housing Units, Area, and Density: 2010 - County". United States Census Bureau. Retrieved 2015-12-27.
^ "DP02 SELECTED SOCIAL CHARACTERISTICS IN THE UNITED STATES – 2006-2010 American Community Survey 5-Year Estimates". United States Census Bureau. Retrieved 2015-12-27.
^ "DP03 SELECTED ECONOMIC CHARACTERISTICS – 2006-2010 American Community Survey 5-Year Estimates". United States Census Bureau. Retrieved 2015-12-27.
^ Leip, David. "Dave Leip's Atlas of U.S. Presidential Elections". uselectionatlas.org. Retrieved 2018-05-03.
^ The leading "other" candidate, Progressive Theodore Roosevelt, received 6,802 votes, while Socialist candidate Eugene Debs received 3,606 votes, Prohibition candidate Eugene Chafin received 309 votes, and Socialist Labor candidate Arthur Reimer received 71 votes.
^ Stark County Elected Officials Archived September 27, 2006, at the Wayback Machine
^ U.S. GenWeb: Ohio, Stark County, accessed February 2018.
Stark County official website
Stark County Sheriff's Office official website
Canton Stark County Convention & Visitors Bureau official website
Lehman, John H. (ed.) (1916). A standard history of Stark County, Ohio (3 vol). Chicago: Lewis Publishing. Retrieved June 26, 2019. CS1 maint: Extra text: authors list (link)
Canton, Ohio and the surrounding communities in Stark County
Ohio and Erie Canal National Heritage Corridor, a National Park Service Discover Our Shared Heritage Travel Itinerary
Stark County Weather and Weather News
Stark County Employment Classifieds
Places adjacent to Stark County, Ohio
Summit County Portage County Mahoning County
Columbiana County
Holmes County Tuscarawas County Carroll County
Municipalities and communities of Stark County, Ohio, United States
County seat: Canton
Alliance‡
Magnolia‡
Minerva‡
CDPs
Oval City
‡This populated place also has portions in an adjacent county or counties
Columbus (capital)
Allegheny Plateau
Appalachian Ohio
The Bluegrass
Glacial till plains
Great Black Swamp
Lake Erie Islands
Miami Valley
Northwest Ohio
Western Reserve
Findlay-Tiffin
Huntington-Ashland
Youngstown-Warren
Auglaize
Geauga
Muskingum
Pickaway
Scioto
Coordinates: 40°49′N 81°22′W / 40.81°N 81.37°W / 40.81; -81.37
Retrieved from "https://en.wikipedia.org/w/index.php?title=Stark_County,_Ohio&oldid=905369561"
Ohio counties
1809 establishments in Ohio
Populated places established in 1809
Articles with dead external links from June 2016
CS1 maint: Extra text: authors list


"Lauren Sowa portrays a focused and powerful Ellen Tree, eager to devour fresh acting methods and take a risk with her new costar."
"The relationship between Aldridge and his Desdemona, played by the famous actress Ellen Tree (Lauren Sowa, in a lovely and subtle performance) is further complicated by the controversy about acting styles: the exaggerated classical English style, and the more naturalistic, emotional method of the American stage."
"Four years ago, Forrest McClendon made a captivating Othello at Philadelphia Shakespeare Theatre. Near the end of the first act, McClendon and Lauren Sowa (playing Ellen Tree, Aldridge's Desdemona) step before a red velvet curtain and play a snippet of the handkerchief scene from Othello. Sowa was actually McClendon's Desdemona in the Philadelphia Shakespeare Theatre production, and I momentarily felt transported back to that satisfying theatrical experience."
"Lauren Sowa as Gwendolyn wears her chin very high indeed."
"The Walnut's Earnest is, in the best tradition of English farces, a grand time."
"Sowa's Gwendolen is a nice blend of practical and absurd."
"Sowa's and Smith's scene in Act II, when the girls discover they are both engaged to Earnest Worthington, is priceless."
"Lauren Sowa as Nurse Kelly and Ian Merrill Peakes as Dr. Sanderson handle the love/hate relationship between these two characters with refinement. Sowa and Peakes steal the scenes they are in with their banter."
"The superb ensemble displays perfect pacing, impeccable comedic timing, and flawless synchronicity."
"Walnut's production is funny from before the curtain goes up, until the final bow."
"Lauren Sowa, as a sanitorium assistant .. you want to run on stage and kiss her."
"The production was a delight, pure fun, well-acted, and well-conceived."
"[The show] is full of sparkle and passion, and has a cast that will delight audiences.."
"Laugh out loud funny moments.."
"Lauren Sowa is excellent as Elyot's petulant new wife Sibyl."
"Lauren Sowa is canny at combining Sibyl's sweet and conventional ways with a tendency to want to prevail and dominate. Sowa's Sibyl is not a silly flibbertigibbet or run-of-the-mill Mayfair-Hertfordshire deb. She is a woman of substance."
"The production is ambitious, full, and rich, and every second is thoroughly engaging. Director Kathryn MacMillan captures all the detailed observations, upper-class spectacle, and lively characters of the novel with satiric wit and 19th-century British decorum. A light and airy pleasure in every way, and a brilliant start to the Lantern's landmark season."
"Lauren Sowa...makes an excellent Emma, transmitting all the coyness, naiveté and judgmental qualities the character embodies. She's just a girl who can't stay out of other people's business although she hasn't really got her own in order – a paradox you can see in Sowa's body language, but mostly in her eyes."
"Exquisitely detailed performance turned in by Sowa. ... Sowa puts up a valiant fight as Othello strangles and smothers her, making for one of the most disturbingly violent Desdemona deaths I've ever seen."
"Desdemona's decision to marry Othello against her father's wishes seems more mature than usual, particularly in Lauren Sowa's smart performance. Her girlish Desdemona has a mature steadiness that complements this cool-headed Othello. Not every production makes their connection as plausible and pleasant as director Carmen Khan does here, and its dissolution is even more tragic because the two are so perfect together."
"Lauren Sowa's Desdemona is lively and feisty. Sowa's performance deftly portrays a woman who, though sheltered and privileged, nevertheless possesses the passionate and adventurous spirit to elope with Othello and follow him to war."
"A welcome surprise comes from the play's other romantic duo, Claudio and Hero: Isaiah Ellis and Lauren Sowa, respectively, engender their characters with enough personality to almost rival Much Ado's most famous lovers. Sowa takes a role notable for an importance to the plot disproportionate to the number of allotted lines, and makes Hero a significant presence on the stage regardless of whether she has lines or not. Her performance is pleasingly multi-layered, with Hero's shy young love of Claudio supplemented by a prim propriety, frequent familial exasperation, and no little backbone whenever her integrity is called into question."
"The accomplished ensemble delivers all the playfulness and tension demanded of their roles. Isaiah Ellis (Claudio) and Lauren Sowa (Hero) are charmingly sentimental and starry-eyed before their nuptials are thwarted. ... As with OTHELLO, MUCH ADO ABOUT NOTHING is a show to be seen, in a season that ranks among the company's very best."
"I'm sorry I waited until nearly the end of the run to see R&J at High Point Theatre last night. Otherwise, I could have given you more notice of the jewel NCSF landed in Lauren Sowa as Juliet. The young actress was utterly charming as the passionate, impetuous star-cross'd teenage lover. Playing a character whose emotions continually bounce from joy to anguish to impatience to peevishness, Sowa totally nailed it. She'll be one to watch."
"The third act allows some terrific acting to shine through. Ms. Sowa and Mr. Weinert finally come to resemble any couple who share a life under duress. Great divide indeed."
"[Michael] Hardart is fortunate to have the splendid Lauren Sowa at the center of his production. Sowa's Ruth is a memorable combination of radiance, grit, and masochistic self-punishment."
"Handling Moody's shifting tones and words with grounded expertise is Hardart's cast, with especially engaging performances by Tony Zazella (Philip) and Lauren Sowa (Ruth)."
"Lauren Sowa's Sam will definitely remind you of your sister or niece or some brat that you know."
"Charmingly done by a cast of five adults who channel their inner child.."
"Stinky Flowers does a lot to bring the magic back into good old-fashioned storytelling."
"An impression of childhood is captured precisely without being childish.."
"The seduction scene is played as a clever battle of the sexes, with the woman clearly in charge. Ms. Sowa teases and Mr. Coughtry complies, looking ever so adolescently foolish in his yearnings. ...Each character in this production speaks Shakespeare's words clearly -- a trademark of this season, and a distinct credit to the company."
"Suitably, Lauren Sowa is enticing as the lady in question and assured daughter of Celia Howard's fine Widow Capilet."
"Sowa, as battered but still beautiful Audrey, manages to make this ditzy, damaged, but warmly honest soul, wholly real."
"and as his would-be paramour, Sowa mixes a nice cocktail of sweet, ditzy, and affecting."
"Lauren Sowa, in the role as cast, was reminiscent of a young Meryl Streep in her spunky strength."
"At the end of act one when Rosalind, played with confidence and subtle comedic prowess by Lauren Sowa, begins to instruct Orlando the show really hits its stride. Sowa's casual cadence and fluidity enabled her to seamlessly flow in and out of Rosalind's disguise, endearing her to the audience."
"Lauren Sowa (Susy Hendrix/the recently blinded wife) gives a superb performance. While masterfully capturing the physical, mental and emotional life of a visually impaired person, Sowa also plays her role smart and sensitive and perfectly paced. Her refusal to give the doll up — thereby somehow implicating her husband in a murder — is inspiring and horrifying. Sowa's energy, intuitiveness and capacity as a listening actor, creates an empowered, multilayered performance by everyone she shares a scene with."
"The famous, frightening, final scene, much of it in complete darkness, is the big pay off; executed brilliantly by Lauren Sowa as Susy and John McCluggage as Roat. You just can't say enough about Sowa's impeccable performance as Susy. She's clearly rattled and unnerved at times, and later strong — someone to be reckoned with. Susy's physical limitation is never questioned, nor is the skill of this actress that imbues her with believability and class."
"The force of the play that most definitively rouses the chills is Susy herself. In Sowa's hands she is reed-like in sensitivity, and so convincing in the mannerisms of her blindness that I often found myself pulled into the dark. At once fierce and vulnerable, Sowa's excellent Susy is the formidable linchpin of Seacoast Repertory Theatre's beautifully produced production of the classic thriller Wait Until Dark."


text:
On Pole
AAC 815
Ferraris with Colombo V12 engine
Ferraris with Lampredi V12 engine
Ferraris with Lampredi 4 and 6 cylinder…
V12 Race Cars
Testa Rossas (1956 – 1962)
GTOs and LMBs (1961 – 1964)
Sport Prototypes
CanAms
512 Prototypes
V12 Grantourismos over 3-litres…
V12 Berlinettas and Cabriolets up to…
V12 Granturismos up to 3-litres…
275 GTBs, GTB/Cs and GTS/4 NART Spyder
Daytonas
V12 Mid Engine Grantourismos
Modern V8 Mid Engine Cars
Modern Front Engine V8 Models
The Supercars
Design and Technology Highlights
Mid Engine Dinos
Front Engine Dinos
The ASAs
The Alfa Romeo Years
ALMS – American Le Mans Series
ALMS 2013
Australian GT Championship
BPR Global GT Series
BPR 1996
British GT Championship
British GT 2016
Carrera Panamericana 1950 – 1955
European Le Mans Series – ELMS
ELMS 2014
Asia-Pacific 2017
Ferrari F1 Clienti and XX Programmes
FIA WEC 2017
Formula A Races 1949
French GT Championship
IMSA WeatherTech SCC
ICGTC 2016
Intercontinental Le Mans Cup
ILMC 2011
Italian GT Championship – CIGT
CIGT 2016
Italian Hill climb Championship –…
CIVM 2016
Pirelli World Challenge
Super GT Series Japan
Autobacs SGTS 2016
Tudor USCC
VLN Nürburgring
VLN 2014
World Sports Car Championship
WSCC 1961
Ferrari Racing Activities 2015
RACING DRIVERS
Racing Drivers A
Racing Drivers B
Racing Drivers C
Racing Drivers D
Racing Drivers E
Racing Drivers F
Racing Drivers G
Racing Drivers H
Racing Drivers I
Racing Drivers J
Racing Drivers K
Racing Drivers L
Racing Drivers M
Racing Drivers N
Racing Drivers O
Racing Drivers P
Racing Drivers Q
Racing Drivers R
Racing Drivers S
Racing Drivers T
Racing Drivers U
Racing Drivers V
Racing Drivers W
Racing Drivers X
Racing Drivers Y
Racing Drivers Z
Official Memorabilia – Numbered
Official Memorabilia – Unnumbered
Ferrari SA APERTA »
« HY-KERS vettura laboratorio
Published 7. January 2013
Revealed on www.ferrari.com: The new 599 GTO
The fastest ever road-going Ferrari will be unveiled to the public at the Beijing Motor Show
Ferrari 599 GTO presented at Modena's Military Academy.
Montezemolo "A wonderful and extraordinary car"
Source: Ferrari
Maranello, 15 April – The 599 GTO world premiere, held exclusively for Ferrari clients, ended around midnight last night.The 12-cylinder Berlinetta, produced in a limited edition of 599, was presented at a symbolic place of the city of Modena: the Military Academy's Ducal Palace. A choice, as Ferrari Chairman Luca di Montezemolo underlined, to "confirm the deep relationship between Ferrari with the terroir and especially with Modena. The Academy is a proper centre of excellence, a historical place, with culture and passion. The Academy and Ferrari are an expression of Italian excellence of this region."The evening, with the participation of Ferrari Vice Chairman Piero Ferrari and CEO Amedeo Felisa, started with a welcome address from Montezemolo and the Academy's Commander General Roberto Bernardini.A few moments later three 599 GTOs arrived through the Ducal Palace's entry gate, accompanied by impressive light beams in red, illuminating the Academy's courtyard. The cars were driven by Luca Badoer, Giancarlo Fisichella and long-time test driver Dario Benuzzi, with three cadets as co-drivers, underlining the strong relationship with the Academy.The three Berlinettas were then admired by the 500 guests from all over the world, followed by enthusiastic comments on the 599 GTO, while some felt sorry for not being amongst the 599 clients of this exceptional model. The 599 GTOs of this special limited series, which will be produced, are already sold.The expectations for this car with such an important name are extremely high, as Montezemolo explained: "We've created a wonderful car, the most powerful in our history, an exclusive car I wanted to give a historical name like GTO, because it deserves it".The 599 GTO's next official appearance will be in Beijing, at the International Automotive Exhibition. A decision taken, considering the growing market for Ferrari in the Asia Pacific region and in China.
Ferrari 599GTO debuts at Modena's military Academy
Maranello, 14 April – First outing for the 599 GTO, which will be the centre of attention during an exclusive gala for Ferrari clients from all over the world at Modena's impressive Military Academy, in the presence of Ferrari Chairman Luca di Montezemolo.Around 500 invited guests will be able to admire the most powerful car ever built by the Prancing Horse, sporting the initials GTO (Gran Turismo Omologata), in a preview. So far there are only two GTOs in the history of the Prancing Horse: the 1962 250 GTO and the 1984 GTO.The 599 GTO, a special series limited to 599, is already a record holder: the fastest lap on the Fiorano race track amongst the road cars produced by Ferrari: 1 minute 24 seconds.
Maranello, 8 April – Now available on www.ferrari.com are the first photos and information on the new 599 GTO, an extreme V12 berlinetta developed to a specific performance-oriented brief. The 599 GTO is, in fact, the company's fastest ever road car. It is an exclusive limited edition special which, in true Ferrari tradition, is a completely new concept, albeit inspired by a production car. In fact the 599 GTO is based on the 599XX, the advanced experimental track car, and can be considered almost a road-going version.The 599 GTO is reserved for just 599 clients who seek the maximum expression of high-performance driving. The 599 GTO benefits directly from the technological transfer from racing and set a record lap time at Fiorano in 1'24".The Fiorano lap time provides just one indication of the potential of this car. Just as significant are the technical specifications – 670 hp in a 1495kg car represents a weight-to-power ratio of just 2.23 kg/hp, and ensures a 0-100 km/h acceleration time of just 3.35" as well as a top speed of over 335 km/h.Fundamental to the GTO's performance is the innovative approach to chassis development which, for the first time on a production car, saw the integration between a handling set-up tuned for a level of responsiveness that is close to the limit and highly sophisticated electronic controls. The result is the almost complete absence of understeer and a truly communicative chassis.As is Ferrari's policy, every new Ferrari features new solutions for a road car. Thus the 599 GTO is equipped with the latest, second-generation carbon-ceramic brakes which are lighter and offer better performance, new aerodynamic innovations, such as the wheel doughnuts which increase aerodynamic efficiency as well as improve brake cooling, and the Supersport tyres developed by Michelin include a wider front tyre for greater roadholding. The driver-car interface is also new with the adoption of the Virtual Race Engineer (VRE) which provides the driver with instantaneous information on performance.The 599 GTO's more aggressive character is also apparent in a number of styling elements that recall the 599XX and from the powerful sound of the V12.The GTO (Gran Turismo Omologata) moniker instantly calls to mind two Ferraris that have entered the collective imagination as symbols of performance. After the 1962 250 GTO, which swept the boards in GT racing categories in the 1960s and is now a highly prized collector's car, came the iconic 1984 GTO, which basically invented the entire modern supercar genre.The web special includes photos, videos and interviews on the new car, while an exclusive preview for a small number of Ferrari clients will be held on April 14th at the Military Academy in Modena. The public debut of the 599 GTO will instead be at the Beijing International Motor Show at the end of April.
599 GTO TECHNICAL CONTENTS
ENGINE AND GEARBOX
The 599 GTO's engine is directly derived from the 599XX unit implementing, however, the necessary modifications for road-going homologation. It thus complies with Euro 5 and LEV 2 standards. The 5999 cc 65-degree V12 engine punches out 670 CV at 8250 rpm with maximum torque of 620 Nm at 6500 rpm and there is a smooth, constant rush of power all the way to the redline with no loss of flexibility even at medium and low revs. This result was obtained by working on the fluid-dynamics and components to reduce internal friction and by adopting, amongst other things, the 599XX's redesigned crankshaft. The car also has a racing-type intake system with a new manifold with diffuser-type intake geometry and short inlet tracts designed to improve power delivery at high revs and reduce losses. To maximise volumetric efficiency per cylinder, a connection between the two plenums at the front compensates for variations in the volume. This is how the engineers managed to achieve maximum performance at high engine speeds. The engine sound inside the car is carefully controlled to balance the intake sound with the exhaust, which features a 599XX-derived 6-into-1 manifold.The 599 GTO features the same, lower 60 ms shift times and the possibility to make multiple downshifts as on the 599XX.
One of the most significant innovations on the 599 GTO is the close correlation between the chassis set-up, which is close to the handling limit, and the input from the electronic controls which are developed to increase overall levels of performance.Since the very earliest states of the two cars' development, Maranello's engineers worked to ensure that these two areas of the car would be seamlessly integrated thus pushing responsiveness to the limit. The result is, of course, superlative driving involvement as well as faster lap times. Apart from new springs and a stiffer rear anti-roll bar, the car also features a second generation magnetorheological suspension control system (SCM2). The suspension works in tandem with the VDC (Vehicle Dynamic Control) and latest-generation F1-Trac traction control. This makes the car extremely responsive to driver inputs – thanks in part to the adoption of a very direct steering ratio – but also very stable under braking, sharper on turn-in, more precise in cornering and quicker out of corners.
The 599 GTO inherits much of the development work – using the same principles as employed in F1 – that went into making the 599XX such an extreme performance car. Reducing weight was a vital objective and the result was the widespread use of composites and components manufactured with technologies more akin to racing specifications. The areas involved include the bodywork and greenhouse (with thinner gauge aluminium and thinner glass), the brakes, transmission and exhaust system. The result is a dry weight of 1495 kg and a weight-to-power ratio of just 2.23 kg/hp, a very significant figure that underlines the GTO's performance potential.
The 599 GTO's aerodynamics have benefited significantly from Ferrari engineers' experience in F1 and with the 599XX which allowed downforce to be greatly increased without impacting on drag. Thanks to solutions transferred from the track car to the road-going version, the GTO generates downforce of 144 kg at 200 km/h. The entire car was honed, including the front, the sides, the flat underbody and cooling flows. In the latter instance, the GTO can count on improved ducting to the brake discs and pads, and the adoption of wheel doughnuts – a disc positioned outside the brake disc that ensure that hot air exiting the wheelarch stays as close to the body of the car as possible to reduce drag.Work on the nose of the car was aimed at reducing the width of the wake generated by the front and thus reduce drag. The front spoiler incorporates a separate lower wing that increases downforce at the front of the car and increases the flow of cooling air to the oil radiator. On the flanks there's a new sill design with a more pronounced leading edge that improves the efficiency of the central section of the underbody. The underbody itself incorporates a new, lower front section with diffusers ahead of the front wheels to optimise downforce, and a new double-curve rear diffuser.
WHEEL RIMS AND TYRES
Here the difference in size between the front and rear tyres has been changed. The GTO's front tyres are now 285/30 on a 9.5'' channel with 315/35 on an 11.5'' channel at the rear. The 599 GTO has 20'' rims. Roll rigidity is greater at the rear to minimise understeer. These solutions guarantee improved lateral grip and quicker turn in.
The new CCM2 braking system is lighter and even more consistent in high performance situations. Its consistent coefficient of attrition meant that the ABS could be calibrated to a particularly high performance level, further reducing lap times, thanks to improved deceleration and shorter stopping distances. In fact, the 599 GTO boasts an excellent 100 to 0 km/h braking distance of just 32.5 metres. As well as their role in improving aerodynamics, the Formula 1-derived wheel doughnuts also improve braking efficiency by optimising brake cooling.
CAR-DRIVER INTERFACE
The car-driver interface was designed to maximise car and driver performance with a layout of the main commands that ensures absolute efficiency and minimum distraction. The Racing manettino also puts the emphasis firmly on sporty, track-specific driving settings by offering the driver full choice with regard to the electronic control parameters. The ICE position on the 599 GTB Fiorano has been replaced by CT-Off (traction control off). The GTO is also fitted with bespoke, longer carbon-fibre F1 paddles for easier use in high-speed driving. In addition the GTO also features the Virtual Race Engineer, a system that monitors the status of the car and gives the driver immediate visibility of vehicle performance.
599 GTO technical specificationsDIMENSIONS AND WEIGHT
Length: 4710 mm (185.4 in)
Width: 1962 mm (77.2 in)
Height: 1326 mm (52.2 in)
Wheelbase: 2750 mm (108.3 in)
Front track: 1701 mm (67.0 in)
Rear track: 1618 mm (63.7 in)
Dry weight*: 1495 kg (3296 lbs)
Kerb weight*: 1605 kg (3538 lbs)
Weight distribution: 47% front – 53% rear
Fuel tank capacity: 105 litres (27.7 US gal/23.1 UK/gal)
Boot volume: 320 litres (11.3 cu ft)
Type: V12 – 65°
Bore & stroke : 92 x 75.2 mm (3.62 x 2.96 in)
Unitary displacement: 499.9 cc (30.51 cu in)
Total displacement: 5999 cc (366.08 cu in)
Maximum power: 500 kW (670 CV) at 8250 rpm
Maximum torque : 620 Nm (457 lbs/ft) at 6500 rpm
F1 6–speed + reverse
SCM2 – Magnetorheological Supension Control
Front: 285/30 ZR20''
Rear: 315/35 ZR20''
CARBON-CERAMIC BRAKES
Front: 398 x 38 mm (15.7 x 1.5 in)
Rear: 360 x 32 mm (14.2 x 1.3 in)
CST with F1-Trac: Traction and stability control
TPTMS: Tyre pressure and temperature monitoring system
Maximum speed: over 335 km/h (over 208 mph)
0-100 km/h (0-62 mph) : 3.35 sec
Combined cycle (ECE)*: 17.5 l/100 km
Combined cycle (ECE)*: 411 g/km*European version
Mick Schumacher to drive father's F2004 at Hockenheim 30. June 2019
Formula 1 myWorld Grand Prix of Austria 2019 26. June 2019
Formula 1 Pirelli Grand Prix of France 2019 18. June 2019
Michelin Le Mans Cup 2019 – Round 3 – Road to Le Mans 12. June 2019
Ferrari Challenge Europe 2019 – Round 4 – Le Mans 11. June 2019
Ferrari Museum Maranello – "90 Years Scuderia Ferrari Exhibition" 5. June 2019
FIA WEC Super Season 2018 – 2019 – Round 8 – 24 Hours of Le Mans 2019 5. June 2019
Ferrari takes the Red Dot: Best of the Best award for the Monza SP1 4. June 2019
Ferrari Challenge North America 2019 – Round 4 – Montreal 4. June 2019
Formula 1 Pirelli Grand Prix of Canada 2019 3. June 2019
Support Mitorosso.com
Archives Select Month June 2019 May 2019 April 2019 March 2019 February 2019 January 2019 December 2018 November 2018 October 2018 September 2018 August 2018 July 2018 June 2018 May 2018 April 2018 March 2018 February 2018 January 2018 December 2017 November 2017 October 2017 September 2017 August 2017 July 2017 June 2017 May 2017 April 2017 March 2017 February 2017 January 2017 December 2016 November 2016 October 2016 September 2016 August 2016 July 2016 June 2016 May 2016 April 2016 March 2016 February 2016 January 2016 December 2015 November 2015 October 2015 September 2015 August 2015 July 2015 June 2015 May 2015 April 2015 March 2015 February 2015 January 2015 December 2014 November 2014 October 2014 September 2014 August 2014 July 2014 June 2014 May 2014 April 2014 March 2014 February 2014 January 2014 December 2013 November 2013 October 2013 September 2013 August 2013 July 2013 June 2013 May 2013 April 2013 March 2013 February 2013 January 2013 December 2012 March 2012 November 2011 September 2011 June 2011 May 2011 March 2011 November 2010 October 2010 September 2010 October 201
Copyright © mitorosso.com 2012. All Rights Reserved.
Designed by LANDSHUT-COMPUTER.


text:
'Pop Top' operation catches 3 cashiers
Aug 18, 2007 at 12:01 AM Jun 29, 2009 at 11:35 AM
DAVENPORT - Some cashiers at Davenport-area retailers are in trouble for allegedly selling alcohol to minors.
On Thursday, Polk County Sheriff's Office deputies used Sheriff's Office Explorers, who are volunteer high school students, to identify stores in the northeast area of the county that sell alcohol to minors.
The effort, which goes by the name "Operation Pop Top," identified four stores that did not sell alcohol to minors and three stores that did, according to sheriff's spokeswoman Carrie Rodgers.
Each of the following people was issued a notice to appear in court on a charge of selling alcohol to minors:
n Arlene Rodriguez, 27, of 43420 U.S. Highway 27 N., No. 131, Davenport. Rodriguez works at the Marathon gas station at 43804 U.S. 27 in Davenport.
n Augustin Oropeza, 57, of 105 Sahali Court, Davenport. Oropeza works at the BP gas station at 43568 U.S. Highway 27 in Davenport.
n Lekisha Smith, 23, of 43420 U.S. 27 N., No. 312, Davenport. Smith works at the 7-Eleven store at 43332 U.S. 27 N. in Davenport.
Clerks at the following stores refused to sell alcohol to the minors:
n Kangaroo gas station at 45489 U.S. 27 N., Davenport.
n Publix at 2424 Sand Mine Road, Davenport.
n 7-Eleven at 44249 U.S. 27 N., Davenport.
n Race Trac gas station at 43250 U.S. 27 N., Davenport.
According to Florida State Statute 542.11(1)(a), selling alcohol to a minor is a second-degree misdemeanor.


text:
package stroom.proxy.repo;

public class ForwardDest {
    private final int id;
    private final String name;

    public ForwardDest(final int id, final String name) {
        this.id = id;
        this.name = name;
    }

    public int getId() {
        return id;
    }

    public String getName() {
        return name;
    }
}

redpajama_set_name: RedPajamaGithub


A dip into the science behind the brain's love of visual data.
Visual data isn't new. We take cues from symbols on directional signage on the roads and in the subway every day. There's a reason for that – we absorb data faster and more accurately when there are visuals. This isn't surprising considering nearly 50% of our brain is involved during visual processing, according to this interactive infographic.
Pie charts and bar graphs are no longer enough, and reading through text only guarantees that a fraction of your message will be remembered. Furthermore, we are so overloaded with information that we ignore more than what we pay attention to because there is simply too much. The amount of information we receive has quintupled in the past 30 years. No wonder our attention spans are fatigued and our filters are pickier than ever!
The rise in popularity of infographics has told us there are still effective ways to get our message across – by transforming information into a story.
What better way to deliver statistics about our visual inclinations than using an infographic? Find out how you can integrate infographics into your marketing strategy.


text:
Vaccination Updates and Medical Forms
COVID-19 Screening Checklists for Parents
CHS Snowmen Displayed
CHS Food Drive Winners!
Student/Athletic Handbook
Harmony Teacher Access
Cloverdale High School » News » What's New » CHS Food Drive Winners!
Congratulations to the following CHS students who were winners of the Food Drive drawing!
Winners of the two flat screen TVs were Ashton Betz and Brady Koosman.
Winner of the $50 was Jesse Mitchell.
Thank you for giving!
205 East Market Street, Cloverdale, IN 46120
Cloverdale Community Schools does not discriminate on the basis of race, color, national origin, sex, age, or disability in admission to its programs, services, or activities, in access to them, in treatment of individuals, or in any aspect of their operations. The lack of English language skills shall not be a barrier to admission or participation in the district's activities and programs. Cloverdale Community Schools also does not discriminate in its hiring or employment practices.