text:
\section{Introduction}
Many statistical learning tasks are posed as penalized maximum likelihood estimation problems, which require solving optimization problems of the form
$$
\text{maximize} \quad \ell(\boldsymbol{\theta}) - \rho g(\boldsymbol{\theta}),
$$
where $\boldsymbol{\theta}$ is a parameter specifying a model, $\ell(\boldsymbol{\theta})$ is a log-likelihood  quantifying lack-of-fit between the model parameterized by $\boldsymbol{\theta}$ and the data, $g(\boldsymbol{\theta})$ is a penalty function that promotes the recovery of parameter estimates that have a desired structure, and $\rho$ is a nonnegative regularization strength parameter that trades off model fit encoded in $\ell(\boldsymbol{\theta})$ with the desired structure encoded in $g(\boldsymbol{\theta})$. Canonical examples of penalty functions for $g(\boldsymbol{\theta})$ include the $\ell_1$-norm which incentivizes recovery of sparse models and the nuclear norm which incentivizes recovery of low-rank models. To date, most work has focused exclusively on point estimates without quantifying the uncertainty in the estimates. Lacking tools for assessing the uncertainty in findings from regularized models, practitioners often resort to classical inference tools designed for fixed models. This practice can lead to seriously inflated type I error and is partly to blame for the reproducibility crisis in science \citep{ioannidis2005most}.

This challenge has motivated the development  of post-selection inference techniques such as simultaneous inference \citep{BerkBrownBujaZhangZhao12PostSelectionInference, BachocPreinerstorferSteinberger2020PostSelectionInference, Kuchibhotla20PostSelectionInference} and selective inference \citep{LeeSunSunTaylor2016PostSelectionInference, choi2017postpca, taylor2018postglm}. A closely related approach calculates confidence intervals for coefficients of high-dimensional linear models through bias-correction \citep{vandeGeerBuhlmannRitovdezeure14DebaisedLasso, ZhangZhang14DebiasedLasso, JavanmardMontanari14DebiasedLasso}.
Most of this literature, however, focuses on variable selection through the $\ell_1$-penalty. Extending these strategies to more complicated penalties and constraints is not straightforward. Moreover, caution is warranted when reporting these confidence intervals because their interpretation (e.g., conditional on the selection event) can be quite different from traditional ones.

An alternative approach is to cast the problem in the Bayesian framework. For example, \citet{TrevorCasella08BayesianLasso} introduced the Bayesian lasso. In this work, the $\ell_1$-penalty was identified with a Laplace prior and a Gibbs sampler was used to sample from the posterior distribution. This early work helped spark the development of Bayesian variable selection methods, specifically alternative sparsity inducing prior distributions such as the spike-and-slab prior \citep{MitchellBeauchamp88BayesianVariableSelection,GeorgeMcCulloch93SpikeSlab}, horseshoe prior \citep{CarvalhoPolsonScott10Horseshoe, PolsonScott10ShrinkagePriors,piironen2017horseshoe}, the orthant normal prior \citep{Hans11ShrinkagePrior}, the correlated Normal-Gamma prior \citep{GriffinBrown12ShrinkagePrior, GriffinBrown13ShrinkagePrior}, the generalized double Pareto prior \citep{Armagan11ShrinkagePrior}, and the Dirichlet-Laplace prior \citep{BhattacharyaPatiPillaiDunson15ShrinkagePrior}. While there have been many innovations in Bayesian techniques for variable selection, more general penalties and constraints beyond sparsity require substantially more problem-specific analysis.

More recently, \citet{pereyra2016proximal} and \citet{durmus2018efficient} proposed the proximal Markov Chain Monte Carlo (ProxMCMC) algorithm for quantifying uncertainty in Bayesian imaging applications where the penalties of interest include the total-variation semi-norm \citep{RudinOsherFatemi1992} and the $\ell_1$-norm. To deal with the non-smoothness of these penalties, they employ the Moreau-Yosida envelope to obtain a smooth approximation to the total-variation semi-norm and $\ell_1$-norm penalties. Samples from the smooth approximate posterior distribution can be generated via the Langevin algorithm.

Their proposed approach opens the door to a potentially general and flexible framework for performing posterior inference for penalized regression models whenever the penalty term is convex and admits a proximal map which can be computed easily -- a situation that holds true for a wide variety of convex penalties. The fly in the ointment, however, is that their approach requires manually setting the regularization strength parameter $\rho$. This limitation prevents the previous formulations of ProxMCMC from being a fully Bayesian framework for generating posterior samples for many existing and more importantly potentially yet-to-be-invented non-smooth penalties.

\textbf{Contributions:} In this paper, we address this limitation and extend ProxMCMC to be fully Bayesian by incorporating penalties and constraints through epigraph priors. Our extended ProxMCMC inference framework is suitable for regularized or constrained statistical learning problems and offers three main advantages.

First, it provides valid and automatic statistical inference even for problems that involve non-smooth and potentially non-convex penalties or constraints. The inference for such problems is traditionally considered difficult.

Second, it is fully Bayesian so that parameter tuning is not required, in contrast to previous ProxMCMC methods \citep{durmus2018efficient} where the regularization strength parameters are either manually fixed or needs to be tuned.

Third, the method is highly modular in the sense that its components -- model, prior, proximal map, and sampling algorithm -- are independent of each other and thus can be modified to accommodate new problems. This feature makes ProxMCMC easily  customizable so that practitioners can adapt it to their unique problems. For example, in analyzing compositional data from microbiome studies, one often needs to fit a lasso model where regression coefficients sum to 0. Although such problems are difficult to tackle in a post-selection inference framework, as we will demonstrate later in Section \ref{sec:constrained-lasso}, it is straightforward to solve using our ProxMCMC method.

Finally, we put our ProxMCMC method on firm foundations by providing guarantees on properness of the approximate posterior and show that the approximate posterior can be made arbitrarily close to a target posterior in total-variation for both convex and non-convex penalties and constraints.

The rest of our paper is organized as follows. Section 2 reviews concepts from convex optimization that underlie the algorithmic building blocks of our ProxMCMC framework. Section 3 illustrates our method using the familiar lasso problem. Section 4 summarizes the key elements from our case study of the lasso to show how our ProxMCMC method can be applied generally. Section 5 presents a variety of illustrative applications.  Sections 6 and 7 provide theoretical guarantees and discussions respectively.

\section{Background}

We review concepts from convex analysis essential for ProxMCMC, specifically Moreau-Yosida envelopes and  proximal maps. For a more thorough review of proximal maps and their applications in statistics and machine learning, we refer readers to \citep{Combettes2005, Combettes2011, PolsonScott2015Proximal}. Recall in convex optimization it is often convenient to work with functions that map into the extended reals, $\mathbb{\bar{R}} = \mathbb{R} \cup \{\infty\}$. The indicator function of a set $C$, denoted $\delta_C({\bm x})$, is the function that is zero for all ${\bm x} \in C$ and is infinity for all ${\bm x} \not\in C$. When the set $C$ is closed and convex, the indicator function $\delta_C({\bm x})$ of $C$ is lower-semicontinuous and convex. A function $g$ is proper if it takes on a finite value for some element in its domain. Let $\Gamma(\mathbb{R}^m)$ denote the set of all proper, lower-semicontinuous, convex functions from $\mathbb{R}^m$ into $\mathbb{\bar{R}}$. Let $\lVert {\bm x} \rVert$ denote the Euclidean norm of a point ${\bm x}$.

\subsection{Moreau-Yosida Envelopes and Proximal Maps}

\begin{definition} Given $g \in \Gamma(\mathbb{R}^m)$ and a positive scaling parameter $\lambda$, the Moreau-Yosida envelope of $g$, denoted by $g^\lambda$, is given by
\begin{eqnarray*}
g^\lambda({\bm x}) & = & \inf_{\greekbold{\omega}} \left\{g(\greekbold{\omega}) + \frac{1}{2\lambda}\|\greekbold{\omega}-{\bm x}\|^2\right\}.
\end{eqnarray*}
\end{definition}
The infimum is always attained at a unique point when $g \in \Gamma(\mathbb{R}^m)$, and the minimizer defines the proximal map of $g$.
\begin{definition}
Given $g \in \Gamma(\mathbb{R}^m)$ and a positive scaling parameter $\lambda$, the proximal map of $g$, denoted $\operatorname{prox}_g^\lambda$, is given by
\begin{eqnarray*}
\operatorname{prox}_g^\lambda({\bm x}) &= & \underset{\greekbold{\omega}}{\arg\min}\; \left\{g(\greekbold{\omega}) + \frac{1}{2\lambda}\|\greekbold{\omega}-{\bm x}\|^2\right\}.
\end{eqnarray*}
\end{definition}
The well known Huber function \citep[Example 6.54]{Beck17FOM}
\begin{eqnarray*}
g^\lambda(x) & = & \begin{cases}
\frac{1}{2\lambda}x^2 & \text{if $\lvert x \rvert \leq \lambda$} \\
\lvert x \rvert - \frac{\lambda}{2} & \text{otherwise}
\end{cases}
\end{eqnarray*}
is the Moreau-Yosida envelope of the absolute value function $g(x) = \lvert x \rvert$. Figure \ref{fig:moreau-envelope} shows $g(x)$ and $g^\lambda(x)$ for three different $\lambda$ values. We can see immediately from this familiar example from robust statistics that the Moreau-Yosida envelope provides a differentiable approximation to a non-smooth function where the approximation improves as $\lambda$ gets smaller.
\begin{figure}[!tbp]
\centering
\includegraphics[width=0.5\textwidth]{figures/moreau-envelope.png}
  \caption{The Moreau-Yosida envelope of the absolute value function $g(x) = \lvert x \rvert$.}
  \label{fig:moreau-envelope}
\end{figure}

In general, the Moreau-Yosida envelope $g^\lambda({\bm x})$ has several important properties. First, $g^\lambda({\bm x})$ is convex when $g({\bm x})$ is convex. Second, if $g({\bm x})$ is convex, then $g^\lambda({\bm x})$ is always differentiable even if $g({\bm x})$ is not, and its gradient can be expressed in terms of $\operatorname{prox}^\lambda_g({\bm x})$, namely
\begin{eqnarray}
\label{eqn:moreau-gradient}
\nabla g^\lambda ({\bm x}) & = & \frac{1}{\lambda}\left[{\bm x}-\operatorname{prox}^\lambda_g({\bm x}) \right].
\end{eqnarray}
Moreover, $\nabla g^\lambda({\bm x})$ is $\lambda^{-1}$-Lipschitz since proximal operators are firmly nonexpansive \citep{Combettes2011}. Finally and most importantly, $g^\lambda({\bm x})$ converges pointwise to $g({\bm x})$ as $\lambda$ tends to zero~\citep{rockafellar2009variational}. In summary, the Moreau-Yosida envelope of a non-smooth function $g({\bm x})$ is a  Lipschitz-differentiable, arbitrarily close approximation to $g({\bm x})$.

The closely related proximal maps play a prominent role in modern statistical learning since many popular non-smooth penalties have unique proximal maps that either have explicit formulas or can be computed efficiently.  For example, the proximal map of $g({\bm x}) = \lVert {\bm x} \rVert_1$ is the celebrated soft-thresholding operator $S_\lambda({\bm x})$ defined by
\begin{equation}
\label{eqn:soft-thresholding}
S_\lambda({\bm x})_i = \begin{cases}
x_i - \lambda  & \text{if $x_i > \lambda$} \\
0 & \text{if $\lvert x_i \rvert \leq \lambda$} \\
x_i + \lambda & \text{if $x_i < -\lambda$.}
\end{cases}
\end{equation}
When the function $g$ is an indicator function $\delta_\mathcal{E}$ of a set $\mathcal{E}$, the proximal map $\operatorname{prox}_{\delta_\mathcal{E}}^\lambda({\bm x})$ is
the Euclidean projection operator onto the set $\mathcal{E}$, namely
\begin{eqnarray*}
\mathcal{P}_{\mathcal{E}}({\bm x}) = \underset{\greekbold{\omega} \in \mathcal{E}}{\arg\min}\; \lVert \greekbold{\omega} - {\bm x} \rVert
\end{eqnarray*}
for all $\lambda > 0$. Let $d_\mathcal{E}({\bm x})$ denote the Euclidean distance of the point ${\bm x}$ to the set $\mathcal{E}$, namely
\begin{eqnarray*}
d_\mathcal{E}({\bm x}) & = & \underset{{\bm y} \in \mathcal{E}}{\inf}\; \lVert {\bm x} - {\bm y} \rVert,
\end{eqnarray*}
then $P_\mathcal{E}({\bm x})$ is the point in $\mathcal{E}$ that is closest in Euclidean distance to the set ${\bm x}$, namely
\begin{eqnarray*}
d_\mathcal{E}({\bm x}) & = & \lVert {\bm x}- \mathcal{P}_{\mathcal{E}}({\bm x})\rVert,
\end{eqnarray*}
and the Moreau-Yosida envelope $g^\lambda({\bm x})$ of $g({\bm x}) = \delta_{\mathcal{E}}({\bm x})$ is
\begin{eqnarray*}
g^\lambda({\bm x}) & = &
\frac{1}{2\lambda}d^2_\mathcal{E}({\bm x}) = \frac{1}{2\lambda}\lVert {\bm x}- \mathcal{P}_{\mathcal{E}}({\bm x})\rVert^2.
\end{eqnarray*}

\subsection{Projections onto Epigraphs}

The key algorithmic primitive in our ProxMCMC framework is the projection onto the epigraph of a penalty function $g({\bm x})$, namely the set
\begin{eqnarray*}
\mathcal{E} = \operatorname{epi}(g) = \{({\bm x}, \alpha) : g({\bm x}) \leq \alpha\}.
\end{eqnarray*}

The Moreau-Yosida envelope of the indicator function $g({\bm x}, \alpha) = \delta_{\mathcal{E}}({\bm x}, \alpha)$ of $\mathcal{E} = \operatorname{epi}(g)$ plays a central role in defining our Bayesian hierarchical model.
The Moreau-Yosida envelope of $\delta_\mathcal{E}({\bm x},\alpha)$ is $\frac{1}{2\lambda}d^2_\mathcal{E}({\bm x},\alpha)$, which is jointly differentiable in ${\bm x}$ and $\alpha$. Subsequently, we can assign $\alpha$ a  prior to incorporate it into posterior inference. Computing with these priors relies on projection onto epigraphs which we describe next.

Projection onto the epigraph of $g({\bm x}, \alpha)$ depends on the proximal map of $g({\bm x}, \alpha)$ \citep[Theorem 6.36]{Beck17FOM}, namely

\begin{eqnarray}\label{eqn:epiprojection}
\mathcal{P}_{\mathcal{E}}({\bm x},\alpha) &=&\begin{cases}
({\bm x},\alpha) & g({\bm x})\le\alpha\\
\left(\operatorname{prox}^{\lambda^*}_g({\bm x}),\alpha+\lambda^* \right) & g({\bm x})>\alpha
\end{cases},
\end{eqnarray}
where
 $\lambda^*$ is any positive root of the auxiliary function
\begin{eqnarray*}
F(\lambda) & = & g\left(\operatorname{prox}^{\lambda}_g({\bm x})\right)-\lambda-\alpha,
\end{eqnarray*}
and can be found using bisection.

\section{An illustrative case study}

To introduce our framework, we first consider a canonical example: the lasso regression \citep{Tibshirani96Lasso}
\begin{eqnarray}
\text{minimize} \quad \frac 12 \|{\bm y} - {\bm X} \greekbold{\beta}\|_2^2 + \rho \|\greekbold{\beta}\|_1, \label{eqn:lasso-penalized-objective}
\end{eqnarray}
where ${\bm y} \in \mathbb{R}^n$ is a vector of continuous responses, ${\bm X} \in \mathbb{R}^{n\times p}$ is a design matrix whose $p$ columns are covariates,  $\greekbold{\beta}\in \mathbb{R}^p$ is the vector of regression coefficients that we seek to estimate, and $\rho$ is a nonnegative regularization strength parameter that trades off model fit with sparsity in our estimate of $\greekbold{\beta}$.
To attack this problem in the ProxMCMC framework, we first write the penalized form \eqref{eqn:lasso-penalized-objective} in an equivalent constrained form
\begin{eqnarray*}
&\text{minimize}& \frac 12 \|{\bm y} - {\bm X} \greekbold{\beta}\|_2^2 \\
&\text{subject to}& \|\greekbold{\beta}\|_1 \le \alpha.
\end{eqnarray*}
There is a one-to-one correspondence between the regularization strength parameter $\rho$ and the constraint parameter $\alpha$. For this reason we will refer to $\alpha$ as the regularization strength parameter as well. We specify the following Bayesian hierarchical model for the constrained formulation of the lasso:
\begin{itemize}
\item Data likelihood: ${\bm Y} \mid \greekbold{\beta}, \sigma^2 \sim N({\bm X} \greekbold{\beta}, \sigma^2 {\bm I})$,
\item A prior $\pi(\sigma^2)$ for the variance: $\sigma^2 \sim IG(r_{\sigma^2}, s_{\sigma^2})$, where $IG(r, s)$ denotes the Inverse-Gamma distribution with scale parameter $r$ and shape parameter $s$ (mean = $\frac{r}{s-1}$ for $s>1$),
\item A prior $\pi(\greekbold{\beta} \mid \alpha)$ for $\greekbold{\beta}$
conditional on $\alpha$, namely
$$
\pi(\greekbold{\beta} \mid \alpha) = \frac{p! }{\alpha^p 2^p} \exp\left [-\delta_{\mathcal{E}}(\greekbold{\beta}, \alpha) \right],
$$
where $\mathcal{E} = \{(\greekbold{\beta}, \alpha) : \lVert \greekbold{\beta} \rVert_1 \leq \alpha \}$
and $\frac{p!}{\alpha^p 2^p}$ is the reciprocal of the volume of $\mathcal{E}$.
Note that $\pi(\greekbold{\beta} \mid \alpha)$ is a flat prior over an $\ell_1$-ball of radius $\alpha$.
\item A prior $\pi(\alpha)$ on $\alpha$ that controls the $\ell_1$-regularization strength: $\alpha \sim IG(r_\alpha, s_\alpha)$.
\end{itemize}
The distribution $\pi(\greekbold{\beta}, \alpha) = \pi(\greekbold{\beta} \mid \alpha) \cdot \pi(\alpha)$ specifies a prior on the epigraph $\mathcal{E} = \{(\greekbold{\beta}, \alpha): \|\greekbold{\beta}\|_1 \le \alpha\} \subset \mathbb{R}^{p+1}$. The posterior log-density takes the following form, up to an irrelevant additive constant,
\begin{equation*}
\begin{split}
\log \pi(\greekbold{\beta}, \sigma^2, \alpha)
= &- \left(\frac{n}{2} + s_{\sigma^2} + 1\right) \log \sigma^2 - \frac{\|{\bm y} - {\bm X} \greekbold{\beta}\|^2 + 2r_{\sigma^2}}{2\sigma^2} \\
& - (s_\alpha+1)\log \alpha - \frac{r_\alpha}{\alpha} - g(\greekbold{\beta}, \alpha),
\end{split}
\end{equation*}
where $g(\greekbold{\beta},\alpha) = \delta_{\mathcal{E}}(\greekbold{\beta}, \alpha)$.

The above posterior is not differentiable, but we can approximate it arbitrarily well with a differentiable posterior. The key idea is to approximate the non-smooth function $g(\greekbold{\beta}, \alpha)$ by its Moreau-Yosida envelope $g^\lambda(\greekbold{\beta}, \alpha)$. The smoothed posterior log-density is
\begin{equation*}
\begin{split}
\log \pi^\lambda(\greekbold{\beta}, \sigma^2, \alpha)
= &- \left(\frac{n}{2} + s_{\sigma^2} + 1\right) \log \sigma^2 - \frac{\|{\bm y} - {\bm X} \greekbold{\beta}\|^2 + 2r_{\sigma^2}}{2\sigma^2} \\
&- (s_\alpha+1)\log \alpha - \frac{r_\alpha}{\alpha} - g^\lambda(\greekbold{\beta}, \alpha),
\end{split}
\end{equation*}
which, due to the smoothness of the Moreau-Yosida envelope $g^\lambda(\greekbold{\beta}, \alpha)$ \citep{rockafellar2009variational}, can be readily sampled using any of a multitude of sampling algorithms for smooth log-densities. In this paper, we use Hamiltonian Monte Carlo (HMC) \citep{Neal11HMC} due to its efficiency and generality. Since HMC works on unconstrained domains, we use the parameterization $(\greekbold{\beta}, \log \sigma^2, \log \alpha)$, so
\begin{equation*}
\begin{split}
\log \pi^\lambda(\greekbold{\beta}, \log \sigma^2, \log \alpha)
= &- \left(\frac{n}{2} + s_{\sigma^2}\right) \log \sigma^2-\frac{\|{\bm y} - {\bm X} \greekbold{\beta}\|^2 + 2r_{\sigma^2}}{2\sigma^2}\\
&- s_\alpha \log \alpha - \frac{r_\alpha}{\alpha} - g^\lambda (\greekbold{\beta},\alpha).
\end{split}
\end{equation*}

We compare ProxMCMC to the Bayesian lasso on the diabetes data set in \citet{efron2004least}. The outcome is a quantitative measure of disease progression over a year, and the covariates are age, sex, body mass index, average blood pressure, and six blood serum measurements. All variables are standardized to have zero mean and unit variance. For the Bayesian lasso, we use the \texttt{blasso} function from the \texttt{R} package \texttt{monomvn} \citep{gramacy2019monomvn} with default parameters. We show the results of the Bayesian lasso with and without using reversible jump MCMC (RJMCMC) to perform model selection. For ProxMCMC, we set $\lambda = 0.001$, $\sigma^2 \sim IG(0.1, 0.1)$, and $\alpha \sim IG(1, 10+2)$. We also calculate the 95\% selective inference confidence intervals \citep{LeeSunSunTaylor2016PostSelectionInference} using the \texttt{R} package \texttt{selectiveInference} \citep{Tibshirani19Pkg}. Since this method requires a model to be selected first, we use lasso with 10-fold cross-validation and choose the largest regularization parameter such that the error is within 1 standard error of the minimum (the \texttt{lambda.1se} option from the \texttt{glmnet} package). Figure \ref{fig:blasso-proxmcmc} displays the interval estimates of the  regression coefficient computed by each method.  We see that for null covariates, the credible intervals of the Bayesian lasso are narrower when model selection by RJMCMC is used. This is because RJMCMC results in many exact zeros (75\% in this example) in the posterior sample, which reduce the variability and thus the width of credible intervals.
When RJMCMC is not used, the credible intervals of the null covariates become wider and are similar to those obtained by the ProxMCMC method. The credible intervals for non-null covariates are similar regardless of which method is used. The selective inference confidence intervals have a different interpretation. The coverage guarantee is in the frequentist sense and is conditional on the model being selected, so they are not directly comparable with credible intervals. Nevertheless, it is interesting to note that two of the four intervals for the selected variables are extremely wide.

\begin{figure}[!tbp]
\centering
\includegraphics[width=0.45\textwidth]{figures/blasso-selinf-proxmcmc-CIs.png}
  \caption{The 95\% credible intervals calculated by Bayesian lasso (bls) and ProxMCMC. bls-RJ-T denotes that RJMCMC is used in computing the posterior samples of the Bayesian lasso, while bls-RJ-F denotes results when RJMCMC is not used . Also shown are the 95\% selective inference confidence intervals (SelInf) for the four variables selected by the lasso using 10-fold cross-validation.}
  \label{fig:blasso-proxmcmc}
\end{figure}

\section{Methodology}
\label{sec:methodology}

Having seen how to apply our ProxMCMC method in the special case of the lasso, we next present the framework in greater generality. Our proposed ProxMCMC method consists of three steps.

\textbf{1. Likelihood and prior.} The first step is to specify the likelihood model for the data $Y$ and priors for the model parameters. This is a standard step in Bayesian modeling. For incorporating a penalty function $g({\bm x})$ with a regularization strength parameter $\alpha$, we specify a prior through an indicator function $\delta_\mathcal{E}({\bm x}, \alpha)$ on the epigraph set
\begin{eqnarray*}
\mathcal{E} = \{({\bm x}, \alpha) : g({\bm x}) \leq \alpha \}.
\end{eqnarray*}
For example, in the lasso example where $\greekbold{\beta}$ was a vector of regression coefficients and $\alpha$ was a regularization parameter, $\mathcal{E} = \{(\greekbold{\beta}, \alpha): \|\greekbold{\beta}\|_1 \le \alpha \}.$

Note that the regularization parameter $\alpha$ must be nonnegative and thus requires a prior with nonnegative support. In our experience, placing an inverse Gamma prior on $\alpha$ works well in practice and will be used throughout this paper.

To gain a sense of how the ProxMCMC epigraph prior differs from existing alternatives, consider the simple case where we put a $\ell_1$-penalty on a single parameter $\beta$, namely
\begin{eqnarray*}
\mathcal{E} & = & \{(\beta, \alpha): |\beta| \le \alpha\}.
\end{eqnarray*}
With an $IG(r,s)$ prior on $\alpha$, the marginal density for $\beta$ is
\begin{equation*}
f_\beta(t) = \int_{|t|}^\infty \frac{1}{2\alpha} \pi(\alpha)d\alpha = \frac{s}{2r}\big[1 - F_{IG(r, s+1)}(|t|)\big],
\end{equation*}
where $F_{IG(r, s+1)}(|t|)$ is the cumulative distribution function of $IG(r, s+1)$ evaluated at $|t|$. Figure \ref{fig:three-priors} contrasts the prior densities of the ProxMCMC epigraph prior, Laplacian prior, and horseshoe prior. We can see that the ProxMCMC epigraph prior density shrinks small $\beta$ while allowing strong signals to remain large.

\begin{figure}[!tbp]
\centering
\includegraphics[width=0.49\textwidth]{figures/priors.png}
  \caption{The ProxMCMC epigraph prior and two other commonly used shrinkage priors.}
  \label{fig:three-priors}
\end{figure}

In the multivariate case, when a vector of parameters $\greekbold{\beta}$ is penalized, e.g., $\mathcal{E} = \{(\greekbold{\beta}, \alpha): \|\greekbold{\beta}\|_1 \le \alpha\}$, the ProxMCMC epigraph prior enforces negative correlation among the components of $\greekbold{\beta}$. This repulsive feature distinguishes it from other Bayesian priors such as the  Laplacian or horseshoe prior, where components of $\greekbold{\beta}$ are independent of each other conditional on the hyperparameter and marginally positively correlated.

Besides incorporating penalties, our framework can also handle cases where we wish to impose constraints.
As before, we will enforce the constraints via an indicator function of the constraint set.

Once the likelihood model and the priors are specified, we can write down the posterior. Let $\boldsymbol{\theta} \in \mathbb{R}^d$ denote all model parameters, which  includes the constrained or regularized parameters $\greekbold{\tau} \in \mathbb{R}^p$ and all other parameters $\greekbold{\eta} \in \mathbb{R}^q$  (so $d = p + q$) including the regularization strength parameter $\alpha$. Further let $\ell(\boldsymbol{\theta})$ be the log-likelihood, $\pi(\greekbold{\eta})$ denote the prior density for $\greekbold{\eta}$, and $g(\greekbold{\tau}) = \delta_\mathcal{E}(\greekbold{\tau})$, where $\mathcal{E}$ denotes either the constraint set or the epigraph set depending on the problem. Note that if $\mathcal{E}$ is an epigraph set, then $g$ is a function of both $\greekbold{\tau}$ and $\alpha$, otherwise it is a function of $\greekbold{\tau}$ alone. For simplicity, we will  write $g(\greekbold{\tau})$ unless it is necessary to be more specific. The posterior density is given by
$$
\pi(\boldsymbol{\theta}\mid Y) = \frac{e^{-U(\boldsymbol{\theta})}}{\int e^{-U({\bm s})}\, d{\bm s}} ,
$$
where $U(\boldsymbol{\theta}) = f(\boldsymbol{\theta})+g(\greekbold{\tau})$ and $f(\boldsymbol{\theta}) = -\ell(\boldsymbol{\theta}) - \log \pi(\greekbold{\eta})$. The posterior $\pi(\boldsymbol{\theta}\mid Y)$ is not differentiable because $g(\greekbold{\tau})$ is not differentiable, but $\pi(\boldsymbol{\theta}\mid Y)$ can be smoothed by substituting $g(\greekbold{\tau})$ with its Moreau-Yosida envelope $g^\lambda(\greekbold{\tau})$ so that both $U^\lambda(\boldsymbol{\theta}) = f(\boldsymbol{\theta}) + g^\lambda(\greekbold{\tau})$ and
$$
\pi^\lambda(\boldsymbol{\theta}\mid Y) = \frac{e^{-U^\lambda(\boldsymbol{\theta})}}{\int e^{-U^\lambda({\bm s})}\, d{\bm s}}
$$
become smooth functions.

\textbf{2. Gradient.} We need to efficiently evaluate the gradient of the smoothed posterior log-density.
This is another standard step in Bayesian modeling, and for commonly used likelihood models and priors, the gradient can be computed numerically by auto-differentiation in software packages such as \texttt{Stan} \citep{STAN} and \texttt{Turing.jl} \citep{Ge18turing}.

As noted earlier, the existence of the gradient of the Moreau-Yosida envelope $g^{\lambda}(\greekbold{\tau})$ depends on the convexity of $g(\greekbold{\tau})$ and thus of the convexity of the regularization term or constraint set $\mathcal{E}$. When $g(\greekbold{\tau})$ is convex, which is the case for many commonly used regularization and constraints,
proximal mappings have been extensively studied in the optimization literature \citep[Chapter 6]{Beck17FOM} and efficient implementations are available from mature libraries such as FOM Matlab toolbox \citep{Beck19FOM}, Python package \texttt{PyProximal}, and Julia package \texttt{ProximalOperators.jl}.

When $g(\greekbold{\tau})$ is non-convex, $g^\lambda(\greekbold{\tau})$ is no longer differentiable. However, as we will show in Section \ref{sec:fixed-rank-matrix-regression}, under certain  regularity conditions, $g^\lambda(\greekbold{\tau})$ will be semidifferentiable and we can calculate a subgradient using the above formula and use it in place of gradient in gradient based samplers.

\textbf{3. Sampling algorithm.} Finally, we invoke a gradient based sampling algorithm such as HMC or the Langevin algorithm to efficiently explore the posterior landscape. Software libraries include \texttt{DynamicHMC.jl}, \texttt{AdvancedHMC.jl}, and \texttt{pyhmc}, to name a few.

\vspace{0.5cm}

\noindent \textbf{Remark:} Before we proceed to examples, we pause to highlight  ProxMCMC's close connection to distance majorization and proximal distance algorithms \citep{ChiZhouLange2014, XuChiLange2017, KeysZhouLange2019, LanderosLange2021, LanderosWuLange2022, Landeros2022}. Proximal distance algorithms are used to solve distance penalty problems.
\begin{eqnarray}
\label{eq:distance_majorization}
\text{minimize} \quad f(\boldsymbol{\theta}) + \frac{\rho}{2} d_\mathcal{E}(\boldsymbol{\theta})^2,
\end{eqnarray}
where $f(\boldsymbol{\theta})$ is typically a negative log-likelihood term quantifying model fit, $\mathcal{E}$ is a target constraint set that we wish our estimate of $\boldsymbol{\theta}$ to be close to, and $\rho$ is a nonnegative tuning parameter that trades off model fit with the amount of constraint violation quantified as the distance to $\mathcal{E}$. A solution to (\ref{eq:distance_majorization}) is a maximum a posteriori estimate under a distance-to-set prior $\pi(\boldsymbol{\theta}) \propto \exp({-\frac{\rho}{2} d_\mathcal{E}(\boldsymbol{\theta})^2})$. Thus, the ProxMCMC method that we introduce in this work provides a fully Bayesian framework for generating posterior samples under a distance-to-epigraph set prior.

\section{Examples}

We present four examples to illustrate the generality of ProxMCMC. Since the potential applications of ProxMCMC are innumerable, our examples are not comprehensive. Nonetheless, we chose these four examples because inference in these problems are either unknown or regarded difficult.

\subsection{Constrained lasso} \label{sec:constrained-lasso}

The constrained lasso problem is formulated as
\begin{eqnarray*}
\text{minimize} && \frac 12 \|{\bm y}-{\bm X}\greekbold{\beta}\|_2^2 + \rho\|\greekbold{\beta}\|_1\\
\text{subject to} && {\bm A}\greekbold{\beta} = {\bm b},
\end{eqnarray*}
where ${\bm A}$ has full row-rank. The constrained lasso is relevant to the analysis of compositional data,
where the rows of ${\bm A}$ represents proportions of a whole and thus must sum to one. The method has been applied in problems involving consumer spending in economics, topic consumption of documents in machine learning, and the analysis of the human microbiome \citep{gaines2018algorithms,JamesPaulsonRusmevichientong20ConstrainedLasso}.

The $\ell_1$-penalization is reparameterized using an indicator function on the epigraph of the $\ell_1$-norm
$$
\mathcal{E}_1 = \{(\greekbold{\beta}, \alpha): \|\greekbold{\beta}\|_1 \le \alpha\}.
$$
The equality constraint is imposed through an indicator function on the hyperplane
$$
\mathcal{E}_2 = \{\greekbold{\beta}: {\bm A}\greekbold{\beta} = {\bm b}\}.
$$

We use an $IG(r_{\sigma^2},s_{\sigma^2})$ prior for $\sigma^2$ and an  $IG(r_\alpha,s_\alpha)$ prior for $\alpha$. Using the $(\greekbold{\beta}, \log \sigma^2, \log \alpha)$ parameterization, the smoothed posterior log-density  up to an irrelevant additive constant is
\begin{equation*}
\begin{split}
\log \pi^\lambda(\greekbold{\beta}, \log \sigma^2, \log \alpha)= &- \left(\frac{n}{2} + s_{\sigma^2}\right) \log \sigma^2-\frac{\|{\bm y} - {\bm X} \greekbold{\beta}\|^2 + 2r_{\sigma^2}}{2\sigma^2} \\
& - s_\alpha \log \alpha - \frac{r_\alpha}{\alpha} - g_1^\lambda (\greekbold{\beta}, \alpha) - g_2^\lambda(\greekbold{\beta}),
\end{split}
\end{equation*}
where $g_1^\lambda(\greekbold{\beta},\alpha)$ and $g_2^\lambda(\greekbold{\beta})$ are the Moreau-Yosida envelopes of the indicator functions $g_1(\greekbold{\beta},\alpha) = \delta_{\mathcal{E}_1}(\greekbold{\beta}, \alpha)$ and $g_2(\greekbold{\beta}) = \delta_{\mathcal{E}_2}(\greekbold{\beta})$, respectively. According to \eqref{eqn:epiprojection}, the proximal map of $g_1(\greekbold{\beta},\alpha)$ is the projection onto the epigraph $\mathcal{E}_1$
\begin{eqnarray*}
\operatorname{prox}_{g_1}^{\lambda}(\greekbold{\beta},\alpha) = \begin{cases}
(\greekbold{\beta}, \alpha) & \text{if $\|\greekbold{\beta}\|_1 \le \alpha$} \\
(S_{\lambda^*}(\greekbold{\beta}), \alpha + \lambda^*) & \text{if $\|\greekbold{\beta}\|_1 > \alpha$}
\end{cases},
\end{eqnarray*}
where $S_\lambda$ is the soft-thresholding operator given in \eqref{eqn:soft-thresholding} and $\lambda^*$ is any positive root of the nonincreasing function $\phi(\lambda) = \|S_{\lambda}(\greekbold{\beta})\|_1 - \lambda - \alpha$ \citep{Beck17FOM}.
The proximal map of $g_2$ is the projection onto the hyperplane given by
\begin{eqnarray*}
\operatorname{prox}_{g_2}(\greekbold{\beta}) = \greekbold{\beta} - {\bm A}^T({\bm A}\Abf^T)^{-1}({\bm A} \greekbold{\beta} - {\bm b}),
\end{eqnarray*}
assuming ${\bm A}$ has full row rank.
The gradient of the posterior log-density is given block-wise by
\begin{eqnarray*}
\frac{\partial \log \pi^\lambda}{\partial \greekbold{\beta}} &=& \sigma^{-2} {\bm X}^T({\bm y} - {\bm X} \greekbold{\beta}) - \lambda^{-1} \left[\greekbold{\beta} - \operatorname{prox}_{g_1}^\lambda(\greekbold{\beta},\alpha)_{\greekbold{\beta}}\right] \\
& &-\lambda^{-1} \left[\greekbold{\beta} - \operatorname{prox}_{g_2}^\lambda(\greekbold{\beta})\right] \\
\frac{\partial \log \pi^\lambda}{\partial \log \sigma^2} &=& - \left(\frac{n}{2} + s_{\sigma^2}\right) + \frac{\|{\bm y} - {\bm X} \greekbold{\beta}\|^2 + 2r_{\sigma^2}}{2\sigma^2} \\
\frac{\partial \log \pi^\lambda}{\partial \log \alpha} &=& - s_\alpha + \frac{r_\alpha}{\alpha} - \lambda^{-1} \alpha [\alpha - \operatorname{prox}_g^\lambda(\greekbold{\beta},\alpha)_{\alpha}].
\end{eqnarray*}

\subsubsection{Simulated microbiome data}
We illustrate our proxMCMC method for the constrained lasso using a simulated microbiome data set. The 16S microbiome sequencing technology measures the number of various organisms called operational taxonomic units (OTUs) in samples. For statistical analysis, counts are normalized into proportions for each sample, resulting in a covariate matrix ${\bm X}$ with each of its rows summing to 1. For identifiability, we need to have a sum-to-zero constraint on the regression coefficients, i.e., $\sum_j \beta_j = 0$. We generate a data set with $n = 300$ samples and $p=20$ OTUs. We set $\beta_1 = 1$, $\beta_2=-1$ and the remaining $\beta_j$, $j = 3, ..., 20$, to 0 so that 10\% of the entries in $\greekbold{\beta}$ are nonzero. The design matrix ${\bm X}$ is generated as follows. First, for each element in ${\bm X}$ an i.i.d.\@ sample from a uniform distribution ($U_{[0,1]}$) is drawn. Second, the rows of ${\bm X}$ are then scaled so that each row of ${\bm X}$ sums to 1. The noise is generated from a normal distribution with mean 0 and $\sigma = 0.1$ so that the sample signal-to-noise ratio $\mathrm{Var}({\bm X}\greekbold{\beta})/\sigma^2$ is approximately $0.2$. We use $IG(0.1, 0.1)$ as a prior for $\sigma^2$ and $IG(1, p+1)$ as a prior for $\alpha$, set $\lambda = 10^{-6}$, and ran HMC for 10,000 iterations. From Figure \ref{fig:constrained-lasso} (a), we can see that the 95\% credible intervals provide good coverage for the first 10 coefficients; those for the other coefficients are similar. Figure \ref{fig:constrained-lasso} (b) shows the histogram of posterior samples of $\sum_j \beta_j$, which is highly concentrated around 0.
\begin{figure}
\centering
  \begin{subfigure}{0.45\textwidth}
    \includegraphics[width=\linewidth]{figures/constrained-lasso-microbiome-CI.png}
    \caption{95\% credible intervals for the first 10 coefficients. Dots mark the truth.}
  \end{subfigure}
  \hspace*{\fill}
  \begin{subfigure}{0.45\textwidth}
    \includegraphics[width=\linewidth]{figures/constrained-lasso-microbiome-histogram.png}
    \caption{Histogram of $\sum_i \beta_i$.}
  \end{subfigure}
\caption{Results from the simulated microbiome data} \label{fig:constrained-lasso}
\end{figure}

\subsection{Graphical lasso}

Given i.i.d. $p$-dimensional observations $\{{\bm x}_1, ..., {\bm x}_n\}$, where ${\bm x}_i \sim N(\mathbf{0}, \greekbold{\Sigma})$, the graphical lasso method infers the underlying conditional dependencies among the covariates by estimating the precision matrix $\greekbold{\Theta} = \greekbold{\Sigma}^{-1}$ through maximizing the regularized log-likelihood
$$
-\frac n2 \operatorname{tr}({\bm S}\greekbold{\Theta}) + \frac n2 \operatorname{logdet}(\greekbold{\Theta}) - \rho\sum_{j \neq k}|\greekbold{\Theta}_{jk}|,
$$
where ${\bm S}$ is the sample covariance and $\rho$ is the regularization strength parameter.
Equivalently, we can maximize
$$
-\frac n2 \operatorname{tr}({\bm S}\greekbold{\Theta}) + \frac n2 \operatorname{logdet}(\greekbold{\Theta}) - g(\greekbold{\Theta},\alpha),
$$
where $g(\greekbold{\Theta},\alpha) = \delta_\mathcal{E}(\greekbold{\Theta},\alpha)$ and $\mathcal{E} = \{(\greekbold{\Theta}, \alpha): \sum_{j \neq k}|\greekbold{\Theta}_{jk}| \le \alpha\}$. The function $g(\greekbold{\Theta},\alpha)$ can be thought as a uniform prior for $\greekbold{\Theta}$ over the $\ell_1$-ball $\{\greekbold{\Theta} : \sum_{j \neq k}|\greekbold{\Theta}_{jk}| \le \alpha\}$. With an  $IG(r_\alpha,s_\alpha)$ prior for $\alpha$, the smoothed posterior log-density of $(\greekbold{\Theta}, \log\alpha)$ is
\begin{equation*}
\begin{split}
\log \pi^\lambda(\greekbold{\Theta}, \log\alpha) = &-\frac n2 \operatorname{tr}({\bm S}\greekbold{\Theta}) + \frac{n}{2}\operatorname{logdet}(\greekbold{\Theta}) \\
& - s_\alpha\log \alpha - \frac{r_\alpha}{\alpha} - g^{\lambda}(\greekbold{\Theta},\alpha).
\end{split}
\end{equation*}
Since HMC works on unconstrained domains and $\greekbold{\Theta}$ needs to be positive definite, we parameterize $\greekbold{\Theta}$ in terms of its lower Cholesky factor ${\bm L}$. Adjusting for the log-Jacobian terms, the smoothed posterior log-density becomes
\begin{equation*}
\begin{split}
\log \pi^\lambda({\bm L}, \log\alpha) = &-\frac n2 \operatorname{tr}({\bm S}{\bm L}\Lbf^T) + \frac{n}{2}\operatorname{logdet}({\bm L}\Lbf^T)\\
& -s_\alpha\log \alpha - \frac{r_\alpha}{\alpha} - g^{\lambda}({\bm L}\Lbf^T,\alpha) \\
&+ p\log(2) + \sum_{j=1}^p(p-j+2){\bm L}_{jj}.
\end{split}
\end{equation*}
The gradients are given by
\begin{eqnarray*}
\nabla_{\operatorname{vech} {\bm L}} \log \pi^\lambda &=& -\big(n\operatorname{vech} ({\bm S}{\bm L})\big)^T + n\big(\operatorname{vech}({\bm L}^{-1})^T \big)^T \\
&& -\frac 2\lambda \bigg(\operatorname{vech}\Big([ \greekbold{\Theta} - \operatorname{prox}_g^\lambda(\greekbold{\Theta},\alpha)_{\greekbold{\Theta}}]{\bm L}\Big)\bigg)^T  \\
&&+\Big(\operatorname{vech}\big(\text{diag}(p+1, p, ..., 2)\big)\Big)^T\\
\frac{\partial \log \pi^\lambda}{\partial \log \alpha} &=& - s_\alpha + \frac{r_\alpha}{\alpha} - \lambda^{-1} \alpha [\alpha - \operatorname{prox}_g^\lambda(\greekbold{\Theta},\alpha)_{\alpha}],
\end{eqnarray*}
where $\operatorname{vech}({\bm A})$ denotes the vector obtained from stacking the columns of the lower triangular part of a square matrix ${\bm A}$.

\subsubsection{Cytometry data}

We compare ProxMCMC with the Bayesian graphical lasso \citep{wang2012bayesian} on the cell-signalling data from \citet{sachs2005causal}, which was used in the original graphical lasso paper \citep{friedman2008sparse}. The data set contains flow cytometry measurements on $p = 11$ proteins and $n = 7466$ cells. We first use the \texttt{R} package \texttt{CVglasso} to compute 5-fold cross-validated graphical lasso estimates for $\greekbold{\Theta}$ and use them as reference in the results below. For the Bayesian graphical lasso we use the \texttt{R} package \texttt{BayesianGLasso} \citep{wang2012bayesian}. We experimented with both the default prior and a few other prior settings, but found little difference among them. Thus we report the results using the default prior (Gamma distribution with shape parameter $1$ and scale parameter $0.1$). For ProxMCMC we use an $IG(1, p+1)$ prior for $\alpha$ and set $\lambda = 0.01$.  We ran 10,000 iterations for both methods. Figure \ref{fig:graphical-lasso} displays the credible intervals. Due to the large number of parameters, we only show the results for the first ten parameters in the plot, but the same pattern is observed in other parameters. We can see that the ProxMCMC credible intervals are consistently narrower and provide good coverage of the graphical lasso estimate, whereas those provided by the Bayesian graphical lasso can be wide and fail to cover the cross-validated estimates.  Among all 66 parameters, all ProxMCMC credible intervals cover the reference values whereas only 24\% of the Bayesian graphical lasso credible intervals do.

\begin{figure}[!tbp]
\centering
\includegraphics[width=0.46\textwidth]{figures/graphical-lasso-cytometry.png}
  \caption{Comparing the 95\% credible intervals of Bayesian graphical lasso versus ProxMCMC on the cytometry data. Black dots are estimates obtained from 5-fold cross-validated graphical lasso.}
  \label{fig:graphical-lasso}
\end{figure}

\subsection{Matrix completion}
Given a matrix ${\bm Y}$ with entries only observed on the set $\Omega=\{(i, j): y_{ij} \text{ is observed}\}$, \citet{Mazumder10SVDReg} propose to complete the matrix by minimizing the convex objective function
$$
\frac 12 \sum_{(i,j) \in \Omega} (y_{ij} - x_{ij})^2
+ \rho \|{\bm X}\|_*,
$$
where $\rho$ is a regularization strength parameter and $\|{\bm X}\|_*$ is the nuclear norm of the completed matrix ${\bm X}$.  The nuclear norm is defined as $\|{\bm X}\|_*= \|\greekbold{\sigma}({\bm X})\|_1 = \sum_i \sigma_i({\bm X})$, where $\sigma_1({\bm X}) \ge \cdots \ge \sigma_{m}({\bm X}) \ge 0$ are the singular values of ${\bm X}$. To put the problem into the ProxMCMC framework, we assume $\operatorname{vec} {\bm Y} \sim N(\operatorname{vec} {\bm X}, \sigma^2 {\bm I})$. Let
$\mathcal{E} = \{({\bm X}, \alpha): \|{\bm X}\|_* \le \alpha \}$ and $g({\bm X}, \alpha) = \delta_\mathcal{E}({\bm X}, \alpha)$. With an $IG(r_{\sigma^2},s_{\sigma^2})$ prior for $\sigma^2$ and an $IG(r_\alpha,s_\alpha)$ prior for $\alpha$, the smoothed posterior log-density using the $\log \sigma^2, \log \alpha$ parameterization is
\begin{equation*}
\begin{split}
\log \pi^\lambda({\bm X}, \log \sigma^2, \log \alpha)
= &- \left(\frac{|\Omega|}{2} + s_{\sigma^2}\right) \log \sigma^2 -\frac{\sum_{(i,j) \in \Omega} (y_{ij}-x_{ij})^2 + 2r_{\sigma^2}}{2\sigma^2} \\
& -s_{\alpha} \log \alpha - \frac{r_\alpha}{\alpha} - g^\lambda ({\bm X},\alpha),
\end{split}
\end{equation*}
The proximal mapping of $g({\bm X}, \alpha)$ is the projection given by
\begin{equation*}
\begin{split}
\operatorname{prox}_g^{\lambda}({\bm X},\alpha) = \begin{cases}
({\bm X}, \alpha) & \text{if $\|{\bm X}\|_* \le \alpha$} \\
({\bm U} \text{diag}(S_{\lambda^*}(\sigma({\bm X}))){\bm V}^T, \alpha + \lambda^*) & \text{if $\|{\bm X}\|_* > \alpha$}
\end{cases},
\end{split}
\end{equation*}
where $S_\lambda$ is the soft-thresholding operator defined in \eqref{eqn:soft-thresholding} and $\lambda^*$ is any positive root of the nonincreasing function $\phi(\lambda) = \|S_{\lambda}(\sigma({\bm X}))\|_1 - \lambda - \alpha$. The gradient of the smoothed posterior log-density is
\begin{eqnarray*}
\frac{\partial \log \pi^\lambda}{\partial {\bm X}} &=& \sigma^{-2} \left[P_{\Omega}({\bm Y}) - P_{\Omega}({\bm X})\right]- \lambda^{-1} [{\bm X} - \operatorname{prox}_g^\lambda({\bm X},\alpha)_{{\bm X}}], \\
\frac{\partial \log \pi^\lambda}{\partial \log \sigma^2} &=& - \left(\frac{|\Omega|}{2} + s_{\sigma^2}\right) + \frac{\sum_{(i,j) \in \Omega} (y_{ij}-x_{ij})^2 + 2r_{\sigma^2}}{2\sigma^2}, \\
\frac{\partial \log \pi^\lambda}{\partial \log \alpha} &=& - s_\alpha + \frac{r_\alpha}{\alpha} - \lambda^{-1} \alpha \left[\alpha - \operatorname{prox}_g^\lambda({\bm X},\alpha)_{\alpha}\right],
\end{eqnarray*}
where $P_{\Omega}({\bm Y})$ is the projection of ${\bm Y}$ onto the set of observed entries $\Omega$, namely,
the $ij$-th entry of $P_{\Omega}({\bm Y})_{ij}$ is $y_{ij}$ for $(i,j) \in \Omega$ and is zero otherwise. Thus, the difference $P_{\Omega}({\bm Y}) - P_{\Omega}({\bm X})$ denotes the matrix of residuals of the observed entries.

\subsubsection{Simulated matrix}
We apply our method to a simulated matrix of size $250 \times 200$. The truth is generated by ${\bm Y} = {\bm Y}_1{\bm Y}_2 + \sigma{\bm E}$ where ${\bm Y}_1$ ($250 \times 3$), ${\bm Y}_2$ ($3 \times 200$), and entries of ${\bm E}$ are generated from the standard normal distribution and $\sigma = 0.1$.  We randomly mask 20\% of the entries (9853 missing) and apply ProxMCMC to calculate 95\% credible intervals for the missing entries. We use an $IG(0.01, 0.01)$ prior for $\sigma^2$ and an $IG(1, 250\times 200+1)$ prior for $\alpha$, and set $\lambda = 0.001$.  Figure \ref{fig:matrix-completion} shows the 95\% credible intervals for the first 20 missing entries and we observe that the credible intervals cover the truth well. In fact, all 9853 missing entries are covered by their credible intervals.

\begin{figure}[!tbp]
\centering
\includegraphics[width=0.46\textwidth]{figures/matrixcompletion-250x200.png}
  \caption{95\% credible intervals and truth (dots) for the first 20 missing entries of the simulated matrix.}
  \label{fig:matrix-completion}
\end{figure}

\subsection{Sparse low rank matrix regression} \label{sec:fixed-rank-matrix-regression}

We consider linear regression with matrix covariates, where the rank of coefficient matrix is subject to regularization. One approach is to penalize the nuclear norm of the coefficient matrix \citep{zhou2014regularized}, the ProxMCMC version of which is very similar to the matrix completion example above because they share the same proximal map. Alternatively, one can constrain the coefficient matrix to have a user-specified rank $k$ \citep{ZhouLiZhu13CPTensor}. Here we explore the second approach to illustrate the potential of ProxMCMC for problems where the regularization or constraints are not convex.

Let $y_i$ be the response for the $i$-th sample. Further let ${\bm Z}_i \in \mathbb{R}^p$ and ${\bm X}_i \in \mathbb{R}^{q\times r}$  be the vector and matrix covariates, respectively. The model is
$$
y_i = {\bm Z}_i^T\greekbold{\gamma} + \langle{\bm B}, {\bm X}_i\rangle + \epsilon_i,
$$
where $\greekbold{\gamma}$ and ${\bm B}$ are the vector and matrix coefficients, $\langle{\bm B}, {\bm X}_i\rangle = \operatorname{tr}({\bm B}^T{\bm X}_i) = \langle\operatorname{vec}{\bm B}, \operatorname{vec}{\bm X}_i\rangle$ is the inner product of the two matrices, and $\epsilon_i \sim N(0, \sigma^2)$. We fix $\mathrm{rank}({\bm B})$ at a user-specified value $k$ through an explicit constraint $\delta_{\mathcal{E}_1}({\bm B})$ where $\mathcal{E}_1 = \{{\bm B}: \mathrm{rank}({\bm B}) = k\}$. To promote sparsity in ${\bm B}$, we also incorporate $\delta_{\mathcal{E}_2}({\bm B},\alpha)$, where $\mathcal{E}_2 = \{({\bm B}, \alpha): \|\operatorname{vec}{\bm B}\|_1 \le \alpha\}$. With a flat prior on $\greekbold{\gamma}$ (i.e., $\pi(\greekbold{\gamma}) \propto 1$), an $IG(r_{\sigma^2},s_{\sigma^2})$ prior for $\sigma^2$, and an  $IG(r_\alpha,s_\alpha)$ prior for $\alpha$, the smoothed posterior log-density is
\begin{equation*}
\begin{split}
\log \pi(\gamma, {\bm B}, \log\sigma^2, \log\alpha) = & - \frac{\sum_{i=1}^n(y_i - {\bm Z}_i^T\gamma - \langle{\bm B}, {\bm X}_i\rangle)^2 + 2r_{\sigma^2}}{2\sigma^2}\\
& -(\frac n2 + s_{\sigma^2})\log\sigma^2 - s_\alpha \log\alpha - \frac{r_\alpha}{\alpha} \\
&- g_1^\lambda({\bm B}) - g_2^\lambda({\bm B}, \alpha),
\end{split}
\end{equation*}
where $g_1^\lambda({\bm B})$ and $g_2^\lambda({\bm B},\alpha)$ are the Moreau-Yosida envelopes of $g_1({\bm B}) = \delta_{\mathcal{E}_1}({\bm B})$ and $g_2({\bm B},\alpha) = \delta_{\mathcal{E}_2}({\bm B},\alpha)$. The proximal map of $g_1({\bm B})$ is the projection onto the set $\mathcal{E}_1$ obtained by thresholding the singular values of ${\bm B}$. However, since $g_1^\lambda({\bm B})$ is non-convex, the gradient formula \eqref{eqn:moreau-gradient} for Moreau-Yosida envelope no longer holds. Instead, we resort to the subsmoothness property of Moreau-Yosida envelopes, for which we need the following definitions \citep{rockafellar2009variational}.

\begin{definition}
(\textbf{Prox-boundedness}) A function $g: \mathbb{R}^n \rightarrow \bar{\mathbb{R}}$ is prox-bounded if there exists $\lambda > 0$ such that its Moreau-Yosida envelope $g^\lambda > -\infty$ for some ${\bm x}\in \mathbb{R}^n$. The supremum of the set of all such $\lambda$ is the threshold $\lambda_g$ of prox-boundedness for $g$.
\end{definition}
In the ProxMCMC framework, we only need the Moreau-Yosida envelope of indicator functions, for which we have $g^\lambda({\bm x}) > -\infty$ for any $\lambda > 0$, so they are always prox-bounded and the threshold $\lambda_g = \infty$.

\begin{definition}
(\textbf{Semidifferentiability}) Let $g: \mathbb{R}^n \rightarrow \bar{\mathbb{R}}$ and $\bar{{\bm x}}$ be a point such that $g(\bar{{\bm x}})$ is finite. If the (possibly infinite) limit $$\lim_{\tau \downarrow 0, {\bm w}' \rightarrow {\bm w}} \frac{g(\bar{{\bm x}} + \tau {\bm w}') - g(\bar{{\bm x}})}{\tau}$$ exists, it is the semiderivative of $g$ at $\bar{{\bm x}}$ for ${\bm w}$, and $g$ is  semidifferentiable at $\bar{{\bm x}}$ for ${\bm w}$. If this holds for every ${\bm w}$, $g$ is semidifferentiable at $\bar{{\bm x}}$.
\end{definition}

By \citet[Example 10.32]{rockafellar2009variational}, if $g({\bm x})$ is lower-semicontinuous, proper, and prox-bounded with threshold $\lambda_g$, then for $\lambda \in (0, \lambda_g)$, the Moreau-Yosida envelope $g^\lambda({\bm x})$ is
semidifferentiable and the subgradient set is
$$
\partial g^\lambda({\bm x}) \subset \lambda^{-1}\left[{\bm x}-\operatorname{prox}_g^\lambda({\bm x})\right].
$$

The function $g_1({\bm B}) = \delta_{\mathcal{E}_1}({\bm B})$ satisfies the above conditions, so we can calculate its subgradient using the above formula and use it in place of the gradient in HMC
\begin{eqnarray*}
\frac{\partial \log \pi}{\partial \greekbold{\gamma}} &=&  \sigma^{-2} \sum_i (y_i - {\bm Z}_i^T\greekbold{\gamma} - \langle{\bm B}, {\bm X}_i\rangle)Z_i, \\
\frac{\partial \log \pi}{\partial {\bm B}} &=&  \sigma^{-2} \sum_i (y_i - {\bm Z}_i^T\greekbold{\gamma} - \langle{\bm B}, {\bm X}_i\rangle)X_i -\lambda^{-1} \left[{\bm B} - \operatorname{prox}_{g_1}^\lambda({\bm B})\right] \\
&& - \lambda^{-1} \left[{\bm B} - \operatorname{prox}_{g_2}^\lambda({\bm B},\alpha)_{{\bm B}}\right], \\
\frac{\partial \log \pi}{\partial \log \sigma^2}&=& - \left(\frac n2 + s_{\sigma^2}\right)+ \frac{\sum_{i=1}^n(y_i - {\bm Z}_i^T\greekbold{\gamma} - \langle{\bm B}, {\bm X}_i\rangle)^2 + 2r_{\sigma^2}}{2\sigma^2}, \\
\frac{\partial \log \pi}{\partial \log \alpha} &=& - s_\alpha + \frac{r_\alpha}{\alpha} - \lambda^{-1} \alpha \left[\alpha - \operatorname{prox}_{g_2}^\lambda({\bm B},\alpha)_{\alpha}\right].
\end{eqnarray*}
Since $g^\lambda_1({\bm B})$ is non-convex, $\operatorname{prox}_{g_1}^\lambda({\bm B})$ is not unique. Our approach is to pick an arbitrary element in the proximal map set, which works well in practice.

\subsubsection{Detecting the cross-shaped signal}
We illustrate the method on a simulated data of cross-shaped signal. The mean responses are $\mu_i = {\bm Z}_i^T\greekbold{\gamma} + \langle {\bm B}, {\bm X}_i \rangle$, where ${\bm Z}_i\in \mathbb{R}^2$ and ${\bm X}_i \in \mathbb{R}^{16 \times 16}$ and their entries are generated from i.i.d. standard normal. We set $\greekbold{\gamma} = (1, 1)^T$ and ${\bm B}$ to have a cross shape (Figure \ref{fig:SLRMR}(a)), where the white cross entries equal 1 and the rest 0. The response $y_i$ equals $\mu_i + \epsilon_i$, where $\epsilon_i$ ($i = 1, ..., n$ and $n = 100$) are also generated from independent standard normal. We use an $IG(0.01, 0.01)$ prior for $\sigma^2$ and an $IG(\sum_i \sigma({\bm B}_0)_i, 2)$ prior for $\alpha$, where $\sigma({\bm B}_0)_i$ is the $i$-th singular value of ${\bm B}_0$ and ${\bm B}_0$ is the initial estimate of ${\bm B}$ obtained by least squares without regularization or constraints. We set the Moreau-Yosida envelope parameter $\lambda = 0.001$. Figure \ref{fig:SLRMR} shows the true signal ${\bm B}$ (panel (a)), the posterior mean from 10,000 HMC samples (panel (b)), and the standard error (panel (c)). Due to space limitations, we only show the 95\% credible intervals of the 8-th column of ${\bm B}$ (Figure \ref{fig:SLRMR-CI}), but all entries of ${\bm B}$ are covered by their 95\%  credible intervals.

\begin{figure}
  \begin{subfigure}{0.3\textwidth}
    \includegraphics[width=\linewidth]{figures/SLRMR-TrueSignal.png}
    \caption{True Signal}
  \end{subfigure}%
  \hfill
  \begin{subfigure}{0.3\textwidth}
    \includegraphics[width=\linewidth]{figures/SLRMR-N100-PostMean.png}
    \caption{Posterior Mean}
  \end{subfigure}%
  \hfill
  \begin{subfigure}{0.3\textwidth}
    \includegraphics[width=\linewidth]{figures/SLRMR-N100-PostSE.png}
    \caption{Standard Error}
  \end{subfigure}
\caption{Proximal MCMC for sparse low rank matrix regression on the cross-shaped data.} \label{fig:SLRMR}
\end{figure}

\begin{figure}
\centering
  \begin{subfigure}{0.5\textwidth}
    \includegraphics[width=\linewidth]{figures/SLRMR-N100-CIs.png}
  \end{subfigure}%
\caption{95\% credible intervals of the eighth column of the cross-shaped signal. X-axis indicates their position in $\operatorname{vec}({\bm B})$. Dots mark the truth.} \label{fig:SLRMR-CI}
\end{figure}

\section{Theoretical properties}

This section presents theoretical results for the ProxMCMC method. The proofs simplify those in \cite{durmus2018efficient} because we focus on the Moreau-Yosida envelope of indicator functions. Our proofs, however, extend to non-convex settings while \citet{durmus2018efficient} assume convexity. As defined in Section \ref{sec:methodology}, $\boldsymbol{\theta} \in \mathbb{R}^d$ represents all model parameters that include both the constrained or regularized parameters $\greekbold{\tau} \in \mathbb{R}^p$ and other parameters $\greekbold{\eta} \in \mathbb{R}^q$. We also use $\ell(\boldsymbol{\theta})$ for the log-likelihood and $\pi(\greekbold{\eta})$ for the prior density of $\greekbold{\eta}$. Our main theoretical results are summarized as follows:

\begin{proposition}
\begin{enumerate}[label = (\arabic*)]
\item For any $\lambda > 0$, the smoothed posterior $\pi^\lambda(\boldsymbol{\theta} \mid Y)$ defines a proper density of a probability measure on $\mathbb{R}^d$, so $$0 < \int_{\mathbb{R}^d}e^{-U^\lambda(\boldsymbol{\theta})}\, d\boldsymbol{\theta} < \infty.$$

\item The approximation $\pi^\lambda(\boldsymbol{\theta} \mid Y)$ converges to $\pi(\boldsymbol{\theta} \mid Y)$ in total-variation as $\lambda \downarrow 0$, i.e., $$\lim_{\lambda \downarrow 0} \|\pi^\lambda(\boldsymbol{\theta} \mid Y) - \pi(\boldsymbol{\theta} \mid Y)\|_{\text{TV}} = 0.$$
\end{enumerate}
\end{proposition}

\begin{proof}
(Posterior properness) The properness of the smoothed posterior $\pi^\lambda(\boldsymbol{\theta} \mid Y)$ follows from the fact that the Moreau-Yosida envelope of an indicator function is always nonnegative. Specifically, when $g = \delta_\mathcal{E}(\greekbold{\tau})$,
$$
g^\lambda(\greekbold{\tau}) = \frac{1}{2\lambda}d_\mathcal{E}(\greekbold{\tau})^2 \geq 0,
$$
where $d_\mathcal{E}(\greekbold{\tau}) = \inf_{{\bm y}\in \mathcal{E}}d(\greekbold{\tau},{\bm y})$ is the distance from $\greekbold{\tau}$ to $\mathcal{E}$, so $-U^\lambda(\boldsymbol{\theta}) = -f(\boldsymbol{\theta})- g^\lambda(\greekbold{\tau}) \leq -f(\boldsymbol{\theta})$, from which we have
$$
e^{-U^\lambda(\boldsymbol{\theta})} \leq e^{-f(\boldsymbol{\theta})} .

$$
Since $f(\boldsymbol{\theta}) = -\ell(\boldsymbol{\theta}) - \log \pi(\greekbold{\eta})$ and both the likelihood and the priors $\pi(\greekbold{\eta})$ are integrable (note that $\greekbold{\eta}$ does not include constrained parameters), we have the desired result.

(Convergence in total-variation)
Let $c = \int e^{-U({\bm s})}\, d{\bm s}$ and $c_\lambda = \int e^{-U^\lambda({\bm s})}\, d{\bm s}$. Since $g^\lambda({\bm x})$ uniformly bounds $g({\bm x})$ from below, i.e., $g^\lambda({\bm x}) \leq g({\bm x})$ for all ${\bm x}$ \citep{rockafellar2009variational}, we have $U^\lambda({\bm x}) \leq U({\bm x})$ and thus $c_\lambda \geq c$. Note that
\begin{equation*}
\begin{split}
\|\pi^\lambda - \pi\|_{\text{TV}}
=& \int |\pi^\lambda({\bm x}) - \pi({\bm x})|\, d{\bm x}\\
=&\int_{\pi^\lambda \geq \pi} \left[\pi^\lambda({\bm x}) - \pi({\bm x})\right]\, d{\bm x} +\int_{\pi^\lambda < \pi} \left[\pi({\bm x})- \pi^\lambda({\bm x})\right]\, d{\bm x}.
\end{split}
\end{equation*}
Let $\mathcal{A}_1 = \left\{{\bm x}: \pi^\lambda \geq \pi\right\}$ and $\mathcal{A}_2 = \left\{{\bm x}: \pi^\lambda < \pi\right\}$,
\begin{equation*}
\begin{split}
&\int_{\mathcal{A}_1} \left [\pi^\lambda({\bm x}) - \pi({\bm x})\right]\, d{\bm x}\\
= &\int_{\mathcal{A}_1} \pi^\lambda({\bm x})\left[1 - \frac{\pi({\bm x})}{\pi^\lambda({\bm x})}\right]\, d{\bm x}\\
= &\int_{\mathcal{A}_1} \pi^\lambda({\bm x})\left[1 -\frac{c_\lambda}{c}e^{g^\lambda({\bm x}) - g({\bm x})}\right]\, d{\bm x}\\
\leq & \int_{\mathcal{A}_1} \left[\pi^\lambda({\bm x}) - e^{g^\lambda({\bm x}) - g({\bm x})}\pi^\lambda({\bm x})\right]\, d{\bm x}\\
\leq &\,\, 1-\frac{c}{c_\lambda},
\end{split}
\end{equation*}
and
\begin{equation*}
\begin{split}
&\int_{\mathcal{A}_2} \left [\pi({\bm x})-\pi^\lambda({\bm x})\right]\, d{\bm x}\\
= &\int_{\mathcal{A}_2} \pi({\bm x}) \left[1 - \frac{\pi^\lambda({\bm x})}{\pi({\bm x})}\right]\, d{\bm x}\\
= &\int_{\mathcal{A}_2} \pi({\bm x})\left[1 -\frac{c}{c_\lambda}e^{g({\bm x}) - g^\lambda({\bm x})}\right]\, d{\bm x}\\
\leq & \int_{\mathcal{A}_2} \pi({\bm x})\left[1 -\frac{c}{c_\lambda}\right]\, d{\bm x}\\
\leq & \,\,1-\frac{c}{c_\lambda}.
\end{split}
\end{equation*}
So $\|\pi^\lambda - \pi\|_{\text{TV}} \leq 2(1-\frac{c}{c_\lambda})$. By \citep{rockafellar2009variational}, when $g({\bm x})$ is proper, lower-semicontinuous, and prox-bounded with threshold $\lambda_g >0$, $g^\lambda({\bm x})$ converges pointwise to $g({\bm x})$ as $\lambda \downarrow 0$. Moreover, since $g^{\lambda}({\bm x})$ is pointwise non-decreasing as $\lambda$ decreases, by the monotone convergence theorem, $\lim_{\lambda \downarrow 0} c_\lambda = c$. Thus
$$
\lim_{\lambda \downarrow 0} \|\pi^\lambda - \pi\|_{\text{TV}} \leq \lim_{\lambda \downarrow 0}  2\left(1-\frac{c}{c_\lambda}\right) = 0.
$$
\end{proof}

\section{Discussion}

Our examples demonstrate that the ProxMCMC method is a highly flexible tool for performing statistical inference on regularized or constrained statistical learning problems. We find that it works well when the regularization or constraints are non-smooth and even non-convex.  In addition, by adopting epigraph priors, our method is fully Bayesian, eliminating the need for tuning the regularization parameter.

For the Moreau-Yosida envelope parameter $\lambda$, a smaller value leads to better satisfaction of the constraints. For example, the histogram of $\sum_j \greekbold{\beta}_j$ from the microbiome example is more concentrated around 0 when $\lambda$ is smaller. Extremely small $\lambda$, however, renders slow mixing of the sampling algorithm. We recommend using smaller $\lambda$ when computational resources allow. Setting $\lambda = 0.001$ works in most applications as the examples show.

Finally, we emphasize that the four examples are meant to whet readers' appetites, not to satiate them. As we mentioned before, the ProxMCMC algorithm is highly modular and can be readily extended to other problems. We hope this paper encourages readers to discover new applications of the ProxMCMC algorithm.

\section*{Funding}

This research was partially funded by the grants from National Science Foundation (DMS-2054253) and National Institutes of Health (R35GM141798, R01HG006139).

\bibliographystyle{apalike}

redpajama_set_name: RedPajamaArXiv


This year I did something I never thought I would do, I looked for love online. All of my friends said I should, a lot of 'what have you got to loses' flew around. To that I say, my dignity, but this was hanging by a thread anyway; so I cleared a corner of my phone and downloaded those darn apps. Apologies to anyone with whom I interacted during this period, it wasn't you, it was me. In that I didn't really want to connect with you or understand you or God forbid go out with you; I wanted to feel better about me.
The last time I dated it was to a Britpop soundtrack; the country was safe in Tony's hands. Times were simpler. If you wanted a date you went to the Firkin and drank until someone looked good enough to snog. Perhaps this is still an accepted practise but with a kid at home, it's not one I can utilise. I figured that there was once a time I turned my nose up at internet shopping and I now spend more time on ASOS than sleeping, so I needed to accept change. Unfortunately I'm not sure I was in the right place and understandably I did not meet the man of my dreams (who I have recently discovered is happily married to Lisa Bonet) but I learnt a lot, about myself and about love on the internet, and so if you're thinking about taking the plunge, go ahead but please learn from my mistakes.
Be honest. In all ways but particularly photographically honest. Actually, one guy I dated said, 'You are much hotter in person, you looked really boring in your photo.' Which left me spending the remainder of the date thinking, so why are you here dude? Still I would prefer that to him turning up, saying nothing and spinning on his heels, which has happened to someone I know. I always say somewhere in my profile that I'm a mother because that's a deal breaker for some people. I don't know why, it's not like I'm gonna bring my kid with me to the crappy pasta chain you've chosen for us to dine in but still, it's always best to be upfront.
When people show you who they are believe them the first time. This Maya Angelou quote is one I dish out to my mates all the time but online I didn't always follow it myself. I'd read on his profile that he was 'not looking for anything serious' (which in case you're wondering means, not looking to spend any cash) but I'd still feel disappointed when the date didn't end with a proposal. I learnt this lesson the hard way when I met a man who told me, 'You're amazing but I could never date anyone with a kid.' And my little brain thought, yes, yes but you don't mean me, or my kid. He did.
Deal in reality. The worst thing about online dating is it starts to feel like a game - Candy Crush for the heart. You feel bored, so you decide to have a little swipe and see if you can score a few more points, I mean matches. That disconnection can stop you remembering that those messages come from real life people, with real life jobs, real hopes and dreams and electricity bills and Auntie Pats. Treat people as you would like to be treated because dating karma is a thing. Also keep an open mind, many people I have dated in life, I would not have connected with online. Not because of their looks (what can I say, I have a thing for handsome men) but because they have some 'flaw' that online would have made them dismissible. I have said no to people on the internet for being too positive, not positive enough, gym lovers, dog owners, emoticon enthusiasts and for liking puns a bit more than is healthy and at the time I thought I was being reasonable. Don't think of the internet as a path to your prince charming but a pub full of strangers, in which you may meet someone who looks good enough to snog. Get drunk if you need to, sometimes it helps.
I honestly wouldn't know what to do now in this new world of dating. Whatever happened to just going to the pub. It seems a bit try before you buy these day!
Great tips. Have realistic expectations is a must, I think a lot of people like to fabricate their profiles!!
As someone who met her previous partner of 10 years online and current partner I can completely understand. Dating world has become a hard place, harder than in person dating.
I definitely think the dating world has got a lot harder to navigate in recent years - these are great tips for those of us who remember a much simpler way of dating!
I met my now husband on the internet, but 15 years ago when dating sites/apps didn't exist!
A lot of our friends have met their other halves online, it seems like the way to date now. People are so busy these days, it's a great way to meet people!
Great dating tips here. Not a fan of the whole dating thing. Just to complicated sometimes. However I know many couples who met online.
There are many sources that you can turn to if you are looking for dating questions advice. The downside with getting dating advice is that you are unsure if you are receiving good advice on dating. Your father may give a few tips to help you, but is he offering useful advice, or is he just excited to have grandchildren?


text:
Monthly Report: July 2014
Greetings Citizens,
July was a big month for Star Citizen! The long battle with rubber-banding tested our resolve at times, but we finally achieved victory! We're ecstatic with the results of all our work: Arena Commander's multiplayer is playing like we originally intended.
We're eager to move on to integrating other portions of the larger game… including those that have been in simultaneous development across our studios and outsource partners! August will see several big events, including Chris' presentation at Gamescom and presentations at PAX East and Dragon*Con. Expect to see some cool parts of the Star Citizen universe in the coming weeks!
Cloud Imperium Santa Monica
Travis Day, Dogfight Producer
At this point, I think it's safe to say that you already know a lot of what Cloud Imperium Games Santa Monica has been focusing on! Our engineers have been working the rubber banding problem since the release of 12.4, and we're so happy that the fix is finally out. Eliminating rubber banding (and in the process so many other issues) involved taking several aspects of Arena Commander (including our very complex physics system) apart, examining all the pieces and then hoping they fit back together correctly. But the early word is they do, and 12.5 is the best multiplayer experience yet!
If you're interested in some very specific details about this process, please check out the "patch update" posts released during each week of 12.5's development. To make a long story short, though, every single member of the Santa Monica team went above and beyond to get this thing out, whether that meant playtesting in their spare time or working until the wee hours rewriting physics code.
Arena Commander isn't the only thing expanding this month, though: the office itself is growing. We're in the process of moving the Community team (and several others) into additional office space next door, which will give us room to grow the team here. One thing that's allowing us to do is move the ship production pipeline to Santa Monica. Kami traveled to Austin last week to work on that transition, which will allow Chris more visibility on where a particular ship is at any time.
We've also been preparing for our Gamescom 2014 demo, which will happen in just over two weeks. Sandi and her team have done a great job of planning the event, where you'll get a chance to meet some of the team behind Arena Commander. We're eager to hang out with the backers who are making this all possible and swap some dogfighting war stories. Look for us in Cologne and feel free to say hi if you spot us around the show.
Looking forward, we're hoping to expand the Arena Commander multiplayer base as soon as possible. Expect news on that shortly. In the meantime, please keep helping us test the game! Your Twitch streams, CS tickets, forum comments and other interactions with the team are what has let us get to this point, and I hope you can feel how your input has allowed us to improve the AC experience. Keep your fighters flying and your feedback coming in!
Cloud Imperium Austin
Eric Peterson, Studio Director
The CIG Austin office has been hard at work on Star Citizen, from getting patch 12.5 out, to preparing for the upcoming Gamescom event. Here is an inside look at what our studio has been up to as we continue onwards toward delivering the BDSSE.
Mocap and Cinematics
We have been working on our pipeline for characters and rigging and have been investigating incorporating some interesting clothing technology from Redacted Studios in (somewhere on planet earth). We got our head scanning rig in and will soon be ready to expand our characters, we have also been investigating head customization methodologies for players. We've been supporting the Redacted Commercial, the Redacted II Commercial, and the bigger badder Redacted III Commercial.
Have spent the month doing a 4 week female animation set mocap session focusing on basic locomotion and FPS/weapon set moves. We have also been supporting the various commercials with motion capture, solving, and editing. In addition our very own Sandi Gardiner stopped in and put on the suit for some motion capture sessions, what a trooper!
We are refining the budget and schedule for the S42 mocap and performance capture shoots and beginning the pre-production process. Where will we capture? Look for an announcement sometime in the future about that.
Release 12.4 support and profiling. Assisted with debugging and improvement of ship movement and matchmaking for release 12.5. Performed CryEngine 3.6.3 integration and submitting it for testing. Physics bug fixes. Basic live server metrics. Implemented and testing automated large universe cluster deployment. Implemented caching patch servers and pre-warming, being deployed with IT. Tool improvements and fixes, including Trackview fixes for commercial work and auto-import of Crytek SDK Perforce server. PU planning, star system layout tool, and PU prototyping. Future improved ship and character movement planning and discussion.. Oh, and released 12.5 – finally.
The CIG Austin design team is hard at work on bringing the persistent universe to life. We're starting small, getting all of the basic functionality in place and expanding as we go. We've put together a more detailed roadmap for the next twelve months of development, and we're polishing off all of the design and tasking needs for that development. Of course, all of it centers around the economy and how it connects the universe.
Meanwhile, we're hard at work supporting other areas of the universe: standardizing ship development, pushing ahead with balance for ships and parts, helping to get ships into the hangar, working out the technology and design of our clothing sets, and – as always – helping the test and balancing of the dogfighting module, as well as animation support for some ship setups. Been a busy month, but we love seeing where this is going, and we hope that you will enjoy it.
The CIG ATX ship team has been VERY busy preparing for Gamescom. Although I can't disclose the actually ships you'll see there, I can tell you they are awesome! Having said that we, on the ship team, are looking well past Gamescom. The month of August and September look to be very exciting with several ships in the pipeline. And we hope you enjoy the M50 – as Chris Smith just did an amazing job of getting that ready.
Well, on ships we worked on the m50, mustang, constellation and it's variants, xian scout, Vanduul ship exploration as well as various components to allow some customization in the near future.
On the characters side of the equation – Redacted commercial characters are being completed, other module characters are in the works and lots of work on the Xian Character as well as Character customizations.
On the persistent universe, or planetside of things, we worked on the Gold horizon space station for Redacted as well as Hangar revamps. In addition, the gold shirt auction has been a big success, thank you all for your continued support.
This month CGBot has been focused on the stars – the Constellations, to be precise! They're preparing the ship and the variants ready to go in hangars. Virtuos has been getting acquainted with our editor and preparing to start working on ships, while also continuing with props for the REDACTED demo, and Rmory is creating some awesome weapon concepts and game assets for the REDACTED as well.
On the contractor front, the focus has been mostly on getting concepts for the REDACTED demo – David Brochard and Rob McKinnon have been working on some cool armor art. In the meantime new contractor Hugues Giboire has been working up a 3D Xi'an character. Our other active contractors have been working on ships – Jim Martin has submitted some new concepts for the Caterpillar, Jan Urschel is working on the Xi'an Transport Derelict, Ryan Church continues to develop the Panther, and George Hull has delivered some awesome new revisions to the Collector concept!"
In addition Citizens, we also had a PU summit in Montreal with our partners at BHVR and are very happy with what is going on up there, not to mention the weather in July is WAY better than in Austin. So, as you can see, we have been working hard on making our dream game, and will continue to march forward on making the BDSSE. The team here appreciates all the support, and patience, as we know that sometimes things go a little bit off kilter on delivery, but love being able to share all of the secrets of game development, even when some things take a bit more time. So thank you all for your continued passion, and love for this game, it keeps us all full of pride and satisfaction knowing that we are all in this together.
Foundry 42
Erin Roberts, Studio Director
Hi again, really busy month down at the farm. So much has been achieved throughout this month at Foundry 42, from helping get 12.5 out, to some great work by the team on our next big feature release (13) and Gamescom. Not forgetting S42 which we have made really great progress on.
Loads of focus has been on GamesCom and getting as much polish into our main branch as possible to show the game at its best. (We have been running two sets of the code for the past 6-8 weeks, Release 12 branch which is what we released 12.5 from on Wednesday night, which is predominantly about getting dog fighting stable and balanced, and main where the new features are being implemented which we make new branches from when we get near to release.) This allows us to get on with new feature implementation for the game, while not destabilizing our release builds with new code and data. So basically what you have seen in 12.5 is only the work that has happened in that branch, there is much, much more that has been going on all this time that you will see soon when we hit Gamescom and release version 13.
I look forward to meeting a bunch of you while in Germany in a few weeks and showing some really cool stuff at Gamescom so I can really thank you all for your support and patience, while we endeavour to make this incredible Universe together.
Blimey! Has it been a month already.
Leader Boards, Lobby systems, Key Bindings have all had design focus and are being worked on for release in 13 or soon after. Also we've got two new awesome game modes for DFM which we plan on showing at Gamescom and releasing with 13.
S42 has made a bunch of progress this month. We have broken down the campaign further with the writers and the tasks are all out there for the "Vertical Slice". We have prototypes for landing and take-off being worked on right now. Also, lots of work is going into the FPS side of the game, with AI behaviour, combat, stealth, perception, pick-ups, objective systems, checkpoints and the conversations system, all being designed and implemented for the game. We have a number of missions early white boxed so we can play through from start to end to make sure the pacing is right.
The ship white-boxing on the Panther Class Escort carrier is continuing really well, and the Idris is getting closer to "Hangar" release. (caveat: it won't fit in the hangar so there will be another cool way to get into it from the hangar).
Again, thanks for the fantastic support. It is a pleasure to be working on such an awesome project with you all.
This month we've been splitting our engineering time up into three parallel teams to make sure we deliver everything we want to show you guys in 12.5, Gamescom and give you all the best experience going forwards with S42.
We've been helping out on the 12.5 build. Apart from the rubber banding we've been trying to track down and fix a really nasty low repro bug which would cause the player's head to shake constantly when they respawn after dying. Every time we thought we had a fix the player would start standing up in the cockpit instead. Every time we thought we'd fixed the standing up bug with head shake would return. And so on. We think we've now got a good handle and fix on both issues now for the latest patch.
We're really excited by what we are going to show you at GamesCom. There's been a lot of work on polishing up the HUD to make it more friendly, the radar to make it less cluttered, cameras, lots of audio improvements, new game modes and loads more.
And finally we're supporting our designers here getting the Squadron 42 vertical slice up and running, including prototyping a new landing mechanic, and long term goals like creating a tool to make setting up the vehicles easier and less error prone, (which will allow us to get ships and vehicles into DFM much faster) a new vehicle state system, AI and, as always, lots of bug fixing!
Environment Art
The environment team has been hard at work this month on creating something special, we are producing a new location like nothing you've seen before in Star Citizen. We have taken the architecture, lighting and colour palette in a different direction, this will complement the existing environments – so we will have Dying Star (Red), Broken Moon (Blue), and the new map (Green). All these elements will also find their way into the PU, so it helps flesh out the Universe you will all be living in when the PU comes online.
We are really excited for you guys to see it, all will be revealed at GamesCon and Release 13.
Vehicle Art
The Ship and Vehicle Team here at Foundry 42 , has been hard at work this month continuing work on the Gladius, the Gladiator and the Retaliator, as well as our current cap ship productions like the Javelin and the Panther.
The Gladius is currently been setup for its shading, audio and animations, before it goes into the technical setup stage, where it can be made ready for the Hangar. We're really close on this one.
The Gladiator is currently in its grey box stage, and the Retaliator is getting closer and closer to its final grey box stage.
The ongoing design work on the Javelin is working nicely alongside the art production for this ship, currently we are working on the Bridge and the Crew mess for this ship, so we can tie these rooms into the storyline for SQ42.
The massive Panther is coming along nicely as well, all the hallways are nearing completion, and we are currently focussing on the Crew Barracks and escape pods to be grey boxed.
Lots of work has also been going on with animation, where new pilot animations for the ship cockpits are going into our main branch for Gamescom, and a bunch of testing on Vanduul movement and also our character and facial systems.
This month has been all about getting the audio to follow clearer goals and design direction. We've further revised a selection of weapon sounds as a test bed for a new approach to weapon sounds. The weapons that we've done so far now include percussive/explosive elements and mechanical layers. The new approach to these sounds gives a more believable and satisfying result. We're also excited to introduce the new ship fly-by sounds as of release 13. If a Vanduul ship tears past you, this moment will now have an exciting and carefully designed fly by sound. Plenty of other improvements have been happening across the board for the sound of Star Citizen so look forward to those over the coming months.
As well as the various improvements we've been making, team audio has been hard at work on gamescom related tasks which we are incredibly excited to show everyone!!
BHVR
Mathieu Beaulieu, Producer
The month of July brings beautiful warm weather to Montreal and lots of planning!
This month Behaviour hosted a Persistent Universe Summit where Chris Roberts, Tony Zurovec and other folks from CIG sat down with [Redacted] and the Montreal team to better and more efficiently plan the approach to world building for the PU.
Summer wouldn't be complete without bugs! Programming has been working on a lot of Release 12.5 bug fixes related to the Visor, Arena Commander UI, as well as with the Asteroid Hangar.
The concept phase for the Origin 890 Jump is complete!
Subscriber flair for July is ready to go, and August and September flair objects are underway.
Level design, art, audio and programming teams have also been hard at work reworking the original three hangars. They are now in the debugging/polish phase and the hangars are looking more and more slick.
A galactic tune-up is also underway! The design team has been injecting Science into the current star systems. They have also been hard at work planning the first internal release of planetside.
The UI team continues to advance with various mobiGlas mock-ups, as well as a redesign of the current pause screen, flair object UI, and the occasional bug squishing. They have also been reading all your posts in the Star Citizen UI/UX thread (even though they can't answer every post (yet)). Keep the great ideas coming!
CGBOT
Austin Goddard, Producer
July has come and gone but not without progress on all fronts from the CGBot team.
We are continuing to chip away at all the tasks needed to bring you the Constellation fleet. One of our main focuses when it comes to Constellation is getting one of the ships dogfight ready. The task hasn't been simple but thanks to our friends at CIG ATX, we are well on our way to wrapping things up in the coming days.
Along with Constellation we are also working on Idris. Detailing out each room individually is a long process but we believe it will be well worth the effort when you get to explore every inch of this ship. Development will continue to take place on Idris through August.
As was mentioned in the last update we finished up two characters at the beginning of this month. We look forward to future character work and the name alone of our next character has CGBot excited. It is our hope that with these characters and ships you will enjoy navigating the galaxy.
FPS Team
[Redacted]
Greetings Star Citizens! July was a really solid month for the FPS Team here at [REDACTED]. Between the epic nerf gun fights that have been randomly breaking out in the office, we've been knocking out features and content like there's no tomorrow and the FPS module is really starting to come together.
The engineers and designers have been implementing all the weapons and gadgets. We've got all the things you would normally expect like pistols, rifles, grenades, etc… but there are also some other unique goodies going in as well. Energy based weaponry, countermeasures like EMPs and some others that I don't want to spoil for you right now! Oh and we now have flashlights… Get hype! You can now see in the dark!
More work went in to getting the pistol ready for EVA and it's very close to being done. The new model is currently being rigged and the VFX guys have been tuning and tweaking the effects.
There was a lot of work done on the Visor HUD for FPS. A large part of the work has been integrating with the HUD system that is currently being used for ships. We've also been working on a mission objective system that hooks in to the Visor HUD.
The animators have been chugging along getting all of the new items and weapons rigged up and ready to go, along with character animations to accompany them. Lots of new motion capture has been going on in Austin this past month as well, some of it being for the FPS module. We've got animations for hit reactions coming in, death sequences and some really cool Zero G stuff which you should be hearing more about in the future.
On the audio side, sound effects are being created and polished for all of the weapons and items. Most of them are a nice mix of mechanical along with a sci-fi energy feel. We know that good sounds make for fun weapons, so we are really trying to push them to sound crisp and powerful.
The Art Team has been creating assets for the space station environment. Some new shaders and tools just came down the pipe, which has allowed them to push the visuals even further and acheive some really nice looking materials that weren't possible before. Yes, that's right. The game is going to look even better than it does currently!
Lastly, we've been helping out with some of the networking and matchmaking issues on Arena Commander. One of our engineers is a matchmaking superstar of sorts. I like to call him the Millionaire Matchmaker… hi [REDACTED], thanks for all the hard work!
That about wraps it up for this month fellow Star Citizens. See you in a month! [REDACTED] out!
P.S. – I really need to get a Nerf gun. My current policy of passive resistance is failing in the ever escalating arms race.
Benoit Beauséjour, Founder
No time to enjoy sunny Montreal summer for the platform team!
New ship pages & viewer
The new ship pages have been taking most of our cycles as we have established the data framework and views we need to present proper ship specifications to allow players to browse the ship catalog. This include a custom management console to handle managing specifications data for each ship variant and chassis as well as API endpoints to drive the Holo Viewer and Tech Overview for all current ships.
There was a huge data management task as we coordinated the capture of over 300 NEW ship screenshots for all available ships from different view angles as well as the acquisition and export of the ship technical data. A few sample are included in this report.
This major refactoring has been in the works for some time but has really come together in the last few weeks as we integrated the new WebGL Holo Viewer into the data model. The new viewer allows you to browse hardpoints, propulsion and POI directly on the ship and see the contents of the ship loadout in 3d space. In addition, we built a full 3D POI Editor that allows us to add POI markers to specific areas of each ship and/or correct the data that we imported. It's very convenient for showing POIs on concept ships that have not reached the CryEngine yet. Work is left to fully integrate this process into the ship pipeline ; but we are hopeful that this will be properly integrated soon!
We also wanted to be able to properly report the state of each ship online. Is this ship in the hangar? Can I fly it? Is this ship currently in concept? This new framework will allow us to modify the game module pages to include the state of each ship in relation with each module.
This set of tools and data models is our first piece of the Galactapedia as all of the ship data is entirely separated from the store and will also be used to drive pieces of that module when we are closer to bring it online. We expect to take on the same work for Planets and Systems in the near future.
To properly present the new ship pages, we have also refactored the entire store to properly reflect the vision of the manufacturer and of the ships. Hopefully this new mode to browse will feel more in-fiction and properly reflect the depth of the Star Citizen universe.
You should see the results of this work online very soon! We're excited!
Organizations Drop Two
Concept work on the features for Organizations Drop Two have continued. We plan to introduce a tool specifically for players with the "Operations" role to allow them to manage Jobs & Divisions within their Organizations. This tool mainly brings the feature of structure to your group and will be a very good way to organize it into sub units. This feature coupled with the features of "Jobs", which will act like a tag system, will make filtering and managing members easier. We have some very cool visualization ideas on this front and we hope to be able to show you some of them soon!
We also made some headway on conceptualizing some recruiting tools for Organizations to help leaders in finding members. We hope to include these in the drop.
All in all a great month!
Moon Collider
Matthew Jack, Founder
July was an important month for Moon Collider, as we significantly increased the size of our team, which will allow us to better meet the AI needs for Star Citizen in the future and support the other teams as they work on more AI-heavy features. We've been really pleased with your response to the Vanduul Swarm gameplay so far, and our new team members will really help us to continue delivering the top notch AI that you are all expecting to see when you play Star Citizen.
We've been a team of three-and-a-half for a while and we've now made the jump to six, with a core of four full-time developers and two supporting part-time. So who are the new faces?
Adam and Diana have both left the warm shores of Perth, Australia, and decided to brave the Edinburgh climate in order to be part of the Kythera team. Both of them have worked previously at Crytek as a programmer and project manager, respectively. Adam will be working full time with us programming and lending his animation expertise to help make our Kythera AI system blend seamlessly with CryEngine's animated characters, while Diana will provide support part time taking over administrative duties so the rest of the team can spend more time getting awesome new features into the game and out to all of you to play.
Jurie is a veteran developer, having worked previously at Rockstar Games. He has been the extra "half" in recent months, doing contract work for us on our debugging toolset. Any system is only as good as it's tools; that's especially true for AI and with Jurie's help we're really pushing what's possible (more on that below!) He will continue to work part-time from France, improving Kythera's debugging and visualization features so that it's easier for us and all of the Star Citizen teams to understand what's going on in the virtual heads of AI. This will help us continue to be confident that the AI is robust, and more importantly, fun to play with and against!
Enough with the roll call, though. What have we been doing this month?
We've been continuing our work to get the dogfighting AI running on the server, and that's shaping up nicely. There is still more work to do to make it more robust, but we're happy with what we've achieved so far.
On the FPS side we've been making more improvements to the AI, with a particular focus on their perception abilities, such as incorporating line-of-sight information to determine whether they can see the player and gathering basic information about all of the characters that are nearby. Their combat abilities are still very basic at the moment, but with the groundwork we're adding we can start developing more sophisticated behaviours.
We made some significant improvements to our toolset this month, with the main feature being an ability to record detailed data about what AI is doing while running the game, and then replay it later with visual debugging information inside the game world. This is an extremely useful feature for designers to be able to go back and examine exactly what the AI did in different situations during a playthrough, so that they can find opportunities to refine their behavior to make them more fun to play against. It's also very handy for tracking down bugs!
A big feature that we're starting to devote serious time to is the persistent universe, and designing how the AI needs to work, particularly planetside. There are a lot of big challenges here and we're all very ambitious with what we're hoping to achieve, so we'll be spending a reasonable amount of time on design work in this area so that we get it right.
Finally, the whole team took some time off from Star Citizen earlier in the month to attend the Game/AI Conference in Vienna. This was a great opportunity to talk with other AI developers and learn about the AI technology being developed for other games. We're passionate about keeping Kythera at the cutting edge of game AI and this conference is one of the most important annual gatherings for game AI developers, so it was really great that we could all attend. It was also an opportunity for our newly expanded team to all be in the same place at the same time, which allowed us to take this excellent team photo at the Prater amusement park in Vienna. And yes, yes we think that clown is pretty scary too.
End Transmission
Monthly Report: February 2014
Posted:2014-03-04 05:00:00
Monthly Report: March 2014
Monthly Report: April 2014
Monthly Report: May 2014
Monthly Report: June 2014
Monthly Report: August 2014
Monthly Report: September 2014
Monthly Report: October 2014
Monthly Report: November
Monthly Studio Report: December 2014
Monthly Studio Report: January 2015
Monthly Studio Report: February 2015
Monthly Studio Report: March 2015
Monthly Studio Report: April 2015
Monthly Studio Report: May 2015
Monthly Studio Report: June 2015
Posted:4 years ago
Monthly Studio Report: July 2015
Monthly Studio Report: August 2015
Monthly Studio Report: September 2015
Monthly Studio Report: October 2015
Monthly Studio Report: November 2015
Posted:1 year ago
Posted:11 months ago
Star Citizen Monthly Report: September 2018
Squadron 42 Monthly Report: September 2018
Posted:9 months ago
Star Citizen Monthly Report: November 2018
Squadron 42 Monthly Report: December 2018
Star Citizen Monthly Report: January 2019
Squadron 42 Monthly Report: January 2019
Star Citizen Monthly Report: February 2019
Squadron 42 Monthly Report: February 2019
Star Citizen Monthly Report: March 2019
Squadron 42 Monthly Report: March 2019
Star Citizen Monthly Report: April 2019
Squadron 42 Monthly Report: April 2019
Star Citizen Monthly Report: May 2019
Squadron 42 Monthly Report: May 2019
Posted:1 month ago
Star Citizen Monthly Report: June 2019
Posted:2 weeks ago
Squadron 42 Monthly Report: June 2019
Posted:1 week ago
View mode: One column Two columns Sort by: Oldest first Newest first Most appreciated first
Loading Additional Feedback
Please select the type best matching your report
Obscene Content
Abusive Conduct

redpajama_set_name: RedPajamaCommonCrawl


Now this is what we were all waiting for! With Vampire Weekend playing in the background as i write this it's the beginning of spring and what a better way to start the warmer seasons than a fresh pop of colour. With crisp whites, bold lines and a touch of floral this collection is sure to be a huge hit this spring.
This collection will be out in stores and online in the next week. You can shop online here!


Ok, well the chaos on the roads that which reduced the town to complete gridlock on Friday night may not be a weekly nightmare.
But it is certainly not unusual.
And anyone who has tried to get around Warrington on a Saturday, or simply for commuting in the morning and evening rush hour, will know this is a town blighted by congestion.
New bridges are being suggested, work is currently taking place on Sankey Way and Knutsford Road to improve the situation on two of the major routes into town.
But what is the long term situation?
Well the motorways aren't going anywhere – so a closure like that at Newton on Friday will always impact on Warrington.
But do cars and lorries need to be directed through the town centre when that happens?
Within five years, the new Mersey Crossing between Runcorn and Widnes will be up and running.
And the worries over how the tolls will affect Warrington remain.
Surely now is the time for the Government to say not just that Warrington people won't have to pay tolls to use those bridges (for you will be charged on the existing Runcorn Bridge too) but that they will be free for all.
Because the impact on Warrington should charges be introduced appears unthinkable.
And what of future development?
The more we build, the more traffic we have to deal with.
So a housing development on the former Wilderspool Stadium site, further development at Centre Park, huge transformation of Bridge Street and the impending opening of cavernous warehouses on Omega.
These are all good for business and the economy.
But what business or economy can thrive if you cannot get from A to B without giving yourself more than an hour to cross town?
But a simple solution like stopping diversions through town and saying no to tolls on the new bridges will help.
ON Thursday night, the Warrington Guardian was named the best weekly newspaper in the north west in the widely recognised O2 media awards.
It is a huge thrill for our team here to have been honoured in such a way.
It is a massive privilege to work for this paper, representing the town and its people.
The Warrington Guardian is in many ways the voice of the town.
We support people who need it, fight battles for those unable to do so themselves and challenge those in power.
We celebrate the town's successes and reflect on its failings.
And we have done so for 161 years – and will continue to do so for years to come.
So while it is nice to win awards, it is even better to work on a paper, website, Twitter feed, Facebook and Instagram page that is at the heart of Warrington.


text:
Microcharon teissieri är en kräftdjursart som först beskrevs av Levi 1950.  Microcharon teissieri ingår i släktet Microcharon och familjen Microparasellidae. Inga underarter finns listade i Catalogue of Life.

Källor

Externa länkar

Gråsuggor och tånglöss
teissieri

redpajama_set_name: RedPajamaWikipedia


Sunday 14 September is the Feast of the Universal Exaltation of the Precious and Lifegiving Cross. The ceremony of the Exaltation and Veneration of the Holy Cross will be held at the end of Matins – about 10.15 am. This means that the Divine Liturgy will begin a little later than usual.
Although it is a Feast Day, the Exaltation of the Cross is a day of fasting: wine and oil may be consumed, but not meat or dairy products. Please remember this if you are bringing refreshments to share after the Liturgy!
This entry was posted in Uncategorized on September 13, 2014 by frkosmas.