text:
Monument Announces Private Placement of Up to $80 Million
December 17, 2012 08:53 ET | Source: Monument Mining Limited
VANCOUVER, BRITISH COLUMBIA--(Marketwire - Dec. 17, 2012) -
NOT FOR DISTRIBUTION TO U.S. NEWSWIRE SERVICES OR FOR DISSEMINATION IN THE UNITED STATES
Monument Mining Limited (TSX VENTURE:MMY)(FRANKFURT:D7Q1) ("Monument" or the "Company") is pleased to announce that it has engaged a syndicate of agents, led by Mackie Research Capital Corporation ("MRCC") and including Salman Partners Inc. and other agents (collectively, the "Agents"), to complete a best-efforts private placement financing (the "Offering") of up to 160,000,000 common shares of the Company (the "Common Shares") at a price of $0.50 per Common Share (the "Issue Price"), for aggregate gross proceeds of up to $80,000,000.
The Company has granted the Agents an option (the "Over-Allotment Option") to increase the size of the Offering by up to an additional 15% in the number of Common Shares. The Over-Allotment Option shall be exercisable by the Agents by giving written notice thereof, or a part thereof, to the Company at any time up to 48 hours prior to the closing of the Offering. The Agents shall be under no obligation whatsoever to exercise the Over-Allotment Option, in whole or in part.
The Company intends to use the net proceeds from the Offering for the initial development of the Mengapur Polymetallic Project, and for general corporate purposes.
Robert F. Baldock, President and CEO of Monument, commented: "Since engaging MRCC as our exclusive financial advisor on November 14, 2012, we have worked closely with them to identify and select the best financing alternative which would allow us to develop our advanced-stage Mengapur Polymetallic Project. We are very pleased to announce this Offering as it marks the next step in MRCC's mandate and positions the Company for future growth."
The Company, working with MRCC, has been engaged in negotiations with certain Malaysian strategic entities who have expressed an interest in subscribing for a significant portion of the Offering. However, there can be no assurance that any such entities will ultimately participate in the Offering.
Monument intends to close the Offering in January 2013 (the "Closing"). The Common Shares issued pursuant to the Offering will be subject to a statutory four month and one day hold period. The Offering is subject to certain conditions including, but not limited to, the receipt of all necessary stock exchange and regulatory approvals, including the approval of the TSX Venture Exchange.
In consideration for their services, the Agents will receive a cash commission of 7.0% of the gross proceeds of the Offering. In addition, the Agents will receive non-transferable share purchase options (the "Agent's Options") to purchase that number of Common Shares which is equal to 7.0% of the total number of Common Shares sold pursuant to the Offering, including any Common Shares sold pursuant to the exercise of the Over-Allotment Option, for a period of 24 months after Closing at the Issue Price. Commissions are subject to regulatory approval.
The proposed $23.4 million private placement to Tulum Corporation as announced on October 17, 2012 is currently under review. As announced on August 7, 2012, the Company's application with respect to a $70 million private placement to Tulum Corporation was withdrawn by the TSX Venture Exchange and will not be proceeding.
This press release shall not constitute an offer of securities for sale in the United States. The securities referred to in this press release have not been, nor will be, registered under the United States Securities Act of 1933, as amended, and may not be offered or sold in the United States absent registration or an exemption from registration.
About Monument
Monument Mining Limited (TSX VENTURE:MMY)(FRANKFURT:D7Q1) is an established Canadian gold producer that owns and operates the Selinsing Gold Mine in Malaysia, with production cash costs among the lowest in the world. Its experienced management team is committed to growth and is advancing several exploration and development projects in Malaysia, including the advanced-stage Mengapur Polymetallic Project. The Company employs 280 people in Malaysia and is committed to the highest standards of environmental management, social responsibility, and health and safety for its employees and neighboring communities.
Robert F. Baldock, President and CEO
Monument Mining Limited
Suite 910- 688 West Hastings Street
Vancouver B.C. Canada V6B 1P1
FOR FURTHER INFORMATION please visit the company web site at www.monumentmining.com.
This news release contains forward-looking statements and forward-looking information about Monument Mining Limited ("Monument"), its business and future plans (together, "forward-looking statements"). Forward-looking statements are statements that are not historical facts and include the timing of the proposed programs and events. The forward-looking statements in this news release are subject to various risks, uncertainties and other factors that could cause actual results or achievements to different materially from those expressed or implied by the forward-looking statements. These risks and certain other factors include, without limitation, the expected outcomes of negotiations litigation and applications for renewal of property rights; government regulation of mining operations; and environmental risks. Generally, forward-looking statements can be identified by the use of forward-looking terminology such as "plans", "expects" or "does not expect", "is expected", "budget", "scheduled", "estimates", "forecasts", "intends", "anticipates" or "does not anticipate", or "believes", or variations of such words and phrases or state that certain actions, events or results "may", "could", "would", "might" or "will be taken", "occur" or "be achieved". Forward-looking statements are subject to known and unknown risks, uncertainties and other factors that may cause the actual results, level of activity, performance or achievements of the Company to be materially different from those expressed or implied by such forward-looking statements, including but not limited to: general business, economic, competitive, geopolitical and social uncertainties; the actual results of current exploration activities; litigation risks; risks with respect mining licenses and exploration licenses; foreign operations risks; other risks inherent in the mining industry and other risks described in the management's discussion and analysis of the Company, which is available under the profile of the Company on SEDAR at www.sedar.com. Although the Company has attempted to identify important factors that could cause actual results to differ materially from those contained in forward-looking statements, there may be other factors that cause results not to be as anticipated, estimated or intended. There can be no assurance that such information will prove to be accurate, as actual results and future events could differ materially from those anticipated in such statements. Accordingly, readers should not place undue reliance on forward-looking statements. The Company does not undertake to update any forward-looking information, except in accordance with applicable securities laws.
Nick Kohlmann
MMY Toronto
nkohlmann@monumentmining.com
Richard Cushing
MMY Vancouver
+1-604-638-1661 x102
rcushing@monumentmining.com
www.monumentmining.com
CHF IR
Anne Robert
anne@chfir.com
Axino AG-Europe
Wolfgang Seybold
+49 711-2535-92 40
wolfgang.seybold@axino.de


This class is an opportunity for those of our students who work really hard to get the help they need to launch an actual music career. We'll teach you that being an artist is more than singing. We'll help you with everything from booking gigs to working your social media, negotiating record deals, planning tours, hiring bands and doing interviews. The class also includes a fully produced track by Sweekman Production.


Sensibility and practicality define the 2019 Ford F-150. Turbocharger technology provides forced air induction, enhancing performance while preserving fuel economy. Ford prioritized fit and finish as evidenced by: a tachometer, adjustable headrests in all seating positions, front fog lights, tilt and telescoping steering wheel, power door mirrors, a trailer hitch, and air conditioning. Passengers are protected by various safety and security features, including: dual front impact airbags, front side impact airbags, traction control, brake assist, a security system, an emergency communication system, and 4 wheel disc brakes with ABS. Various mechanical systems are monitored by electronic stability control, keeping you on your intended path. We have the vehicle you've been searching for at a price you can afford. Stop by our dealership or give us a call for more information.


text:
\section{Introduction}

The introduction of accelerated gradient methods by Nesterov in~\cite{Nest83} has arguably revolutionized the world of large-scale convex and non-smooth optimization. Computationally, they are as simple as plain gradient descent, but come along with a much faster rate of convergence. Therefore, they found numerous applications in modern signal/image processing, and machine learning. It is well-known that these accelerated gradient methods are \enquote{optimal} for the class of smooth and convex (not necessarily strongly-convex) problems with Lipschitz continuous gradient, in the sense that their worst case complexity is proportional to the theoretical lower complexity bound of first-order methods for this class of problems \cite{NY83,Nest04}. The mechanism of accelerated gradient methods can be interpreted and explained in many different ways and hence, the magic of acceleration is still the subject of intensive research efforts. We refer to \cite{Goh17}, for a detailed exposition of several aspect of acceleration.

Accelerated gradient methods can be seen as a modification of the heavy-ball method of Polyak~\cite{Polyak64}. While the heavy-ball method already achieves an optimal convergence rate on smooth and strongly convex functions with Lipschitz continuous gradient, accelerated gradient methods are optimal, among first-order methods, for smooth convex problems with Lipschitz continuous gradient~\cite{Nest04}.

In the original work~\cite{Nest04}, the acceleration is explained by the concept of so called estimation sequences, which generate a sequence of simple convex functions approximating the original function. Accelerated gradient methods also show strong connections to the St\"ormer--Verlet method~\cite{Hairer03} for discretizing second order ordinary differential equations (ODEs). Using this connection, both the heavy-ball method~\cite{Polyak64} and Nesterov's original method~\cite{Nest83} can be seen as particular types of discrete-time approximations to the \enquote{heavy-ball with friction} dynamical system. The main principle here is to attach a \enquote{mass} to the sequence of points generated which accelerates when moving down the landscape of the objective function. More recent works use this relation to investigate the properties of accelerated gradient methods in the framework of ODEs~\cite{Su14,Attouch16}. Finally, accelerated first-order methods can also be explained by accelerated dual schemes~\cite{Nest07} or by accelerated primal-dual algorithms~\cite{CP11}.

For quadratic functions, both the heavy-ball method and accelerated
gradient methods show striking similarities to the conjugate gradient
method~\cite{Stiefel1952}. However, while in the conjugate gradient
method, the step size and extrapolation parameters are chosen locally
using an exact line (or plane) search, accelerated gradient methods
derive their parameters from global properties of the objective such
as the smallest and largest eigenvalues of the corresponding system
matrix. On the one hand, the conjugate gradient method is optimal, among first-order methods, and
comes along with a finite convergence property for quadratic functions.  On the other hand,
its generalization to non-quadratic functions is much harder. At
this point, accelerated gradient methods are advantageous. They
can be applied not only to quadratic problems but to the whole class
of differentiable functions with Lipschitz continuous gradient.

Accelerated gradient methods have been generalized to be applicable to
a class of convex optimization problems that can be written as the sum
of a differentiable function with Lipschitz continuous gradient and a
non-smooth function with easy to compute proximal map. The most
popular instance is the Fast Iterative Shrinkage Thresholding
Algorithm (FISTA)~\cite{BT09b} but also other, more general schemes
have been proposed. See the work of Tseng~\cite{Tseng08} for an
excellent presentation and unification of a whole family of
accelerated proximal gradient methods.

Finally, accelerated gradient methods have also turned out to perform
very well on non-convex optimization problems. However, while their
empirical performance is often comparable to the convex case, their
convergence properties are still hardly understood from a theoretical
point of view~\cite{GL15,LL15,DK16,OCBP14}.

In this paper, we study a modification of FISTA for
minimizing an objective given by the sum of a smooth function $f$ with
Lipschitz continuous gradient and a non-smooth function $g$ with
simple proximal mapping. We still rely on a step size parameter
related to the global Lipschitz parameter $L$, however similar to the
conjugate gradient method, we propose to locally adapt the
extrapolation parameter by means of an exact line search. A simplified
version of the algorithm's update step for minimizing a smooth function $f$ is the following:
\[
  \begin{split}
    y\idx{\beta}\iter\k =&\  x\iter\k + \beta (x\iter\k - x\iter\km) \\
    x\iter\kp\idx{\beta} \in&\ \argmin_{x\in \R^N}\, g(x)
    + \scal{\nabla f(y\idx{\beta}\iter\k)}{x-y\idx{\beta}\iter\k}
    + \frac L2 \vnorm{x - y\idx{\beta}\iter\k}^2 \,,
  \end{split}
\]
where $x_k$ is the main iteration, $y_k^{(\beta)}$ is the extrapolated
point (depending on the choice of $\beta)$ and $\beta \in \R$ is a
free parameter which is optimized in each step in order to provide the
locally largest decrease of the proximal subproblem. It turns
out that if the function $f$ is quadratic, the optimal choice of
$\beta$ makes the scheme equivalent to a proximal SR1 (identity minus
rank one) quasi-Newton method. This equivalence yields a connection
between accelerated schemes and quasi-Newton methods, which holds even
when $g$ is a non-convex function. The convergence that we establish
for our method in a general non-smooth non-convex setting translates
directly to a class of proximal quasi-Newton methods. Unlike the
general class of proximal quasi-Newton methods which usually suffer
from the problem that simple proximal mappings become hard to solve,
efficient solutions to the \enquote{rank-1 proximal mappings} are
known \cite{BF12,KV17,BFO18}.\\

The remainder of the paper is organized as follows: In Section~\ref{sec:rel-work} we provide a review of state-of-the-art and put the contributions of this paper into related work. In Section~\ref{sec:opt-problem}, we present the considered optimization problem, and in Section~\ref{sec:alg}, we detail the proposed algorithm and give different convergence guarantees for both convex and non-convex problems. Numerical results of the proposed algorithm are provided in Section~\ref{sec:numerics}. In the last section we give some conclusions and discuss open problems for future research.

\section{Related work} \label{sec:rel-work}

\paragraph{Proximal Gradient Method.} The basic update step of our method is a so-called proximal gradient step (also known as forward-backward splitting) \cite{LM79,CW05,CP11a,Nest04}. Although, using this basic step yields an efficient algorithm in many cases, it has been observed that the worst case complexity is not optimal \cite{Nest04} for certain classes of convex optimization problems. This observation led to the exploration of so-called accelerated schemes. The basic proximal gradient step can be accelerated by an additional (computationally cheap) extrapolation step \cite{BT09b,Nest83,Tseng08} (see also \cite{Tseng10}), where the iteration-dependent extrapolation parameter obeys a certain rule derived from global properties of the objective function. In contrast to the prescribed rules, in this paper, we explicitly optimize the extrapolation parameter. Surprisingly, for a certain class of problems, the optimized scheme has a closed form expression as a proximal quasi-Newton method, which yields new convergence results.

\paragraph{Classical quasi-Newton Methods.} Quasi-Newton methods are intensively studied in the classic context of non-linear optimization problems. We refer to \cite{NW06,DS96b,Broyden67} for an overview and references. The basic idea of quasi-Newton methods is to successively improve a quadratic approximation to the objective function, i.e., the goal is to approximate second order information (Hessian) using combinations of first order information. Maybe the most widely known and used quasi-Newton method is BFGS \cite{Fletcher87} or its limited memory variant L-BFGS \cite{LN89}, and its extension to bound constraints L-BFGS-B \cite{BLNZ94}. More results that motivate the acceleration governed by such a variable metric approach are \cite{BGLS95,BQ99,PLS08}.

\paragraph{Quasi-Newton Methods for non-smooth problems.} Though, originally designed for smooth optimization problems, the BFGS method shows good performance on non-smooth problems as well. In \cite{YVGN10}, the method was reinterpreted and analysed in the non-smooth convex setting. Theoretical guarantees of the original BFGS method for one or two dimensional non-smooth problems are established in \cite{LO13,LZ15}. Motivated by the original global convergence proof of Powell \cite{Powell76}, Guo and Lewis \cite{GL17} substantially extended the previous theoretical guarantees to a class of non-smooth convex problems. In general, convergence cannot be expected.

\paragraph{Proximal quasi-Newton Methods.} Driven by the success of proximal splitting methods, the concept of quasi-Newton methods was also applied to optimization problems with more structure than just smoothness, similar to the setting of Proximal Gradient Descent. The crucial aspect for the efficiency of such a variable metric Proximal Gradient Method is the evaluation of the proximal mapping, which is often expensive when coordinates are not separated in the objective.  There are specific choices of the metric that allow the proximal mapping to be solved efficiently, e.g. when the metric is a diagonal matrix, which preserves the separability. Becker and Fadili \cite{BF12} derived efficient solutions when the metric is of type \enquote{identity plus rank 1}.  The proximal step was embedded into a zero memory variant of the SR1 quasi-Newton method \cite{NW06}.  Unlike the BFGS-method, which updates the Hessian approximation with a matrix of rank 2, the SR1 method updates the approximation with a rank 1 matrix.  Nevertheless, Nocedal and Wright \cite{NW06} state the observation that the \enquote{[...] SR1 method appears to be competitive with the BFGS method}. A special case of an \enquote{identity minus rank 1} metric was studied in \cite{KV17}, which also leads to an efficiently solvable proximal mapping. Recently, proximal mappings with respect to metrics of type ``identity plus/minus rank $R$'' have been studied in~\cite{BFO18}. In \cite{FG16}, interior point methods are used to solve the proximal mapping efficiently for so-called quadratic support functions.\\

Variable metric versions of the Proximal Gradient Method have been studied without paying special attention to efficiently solving the proximal mapping. The earliest reference is \cite[Section 5]{CR97}. In the broader context of monotone inclusion problems, the general framework for analyzing the convergence of variable metric methods is that of quasi-Fej\'{e}r sequences \cite{CV13}, which was used in \cite{CV14} to find a relation to primal--dual schemes. For a variable metric algorithm with mild differentiability assumption (without the usual gradient Lipschitz continuity), we refer to \cite{Salzo16}.  The convergence of forward--backward splitting with iteration dependent Bregman distances in an infinite dimensional setting, which contains the variable metric proximal step as a special case, was proved in \cite{Nguyen17}.

\paragraph{Non-convex setting.} The variable metric Proximal Gradient Method in \cite{CPR13} follows the majorization--minimization principle \cite{HL04}. The metric is constructed to induce a quadratic majorizer in each iteration. However, the arising proximal mappings are computationally expensive. Extensions to a block-coordinate descent version are presented in \cite{CPR16} and a combination with an inertial method in \cite{Ochs16}. In \cite{BLPP16,BLPPR16}, the choice of the metric enjoys a great flexibility at the cost of an additional line search step in the algorithm. An extension of their framework to a general non-smooth first-order oracle and a flexible choice of Bregman distances was proposed in \cite{OFB17}. A different way to incorporate a variable metric into forward--backward splitting (FBS) was proposed in \cite{STP17}. They reinterpret FBS as a gradient descent method with a variable metric, which allows them to use some of the machinery from smooth optimization. Besides accelerating optimization methods by a variable metric, accelerated (optimal) gradient methods can be used in the non-convex setting \cite{GL15,LL15,DK16} as well, where the goal is to guarantee the accelerated rate of convergence when the same method is applied to convex problems.

\paragraph{Subspace approach.} The proposed method shows similarities to sequential subspace optimization (SESOP) \cite{NZ05,EMZ07} (see also the review article \cite{ZE10}) and the Majorize-Minimize (MM) subspace algorithm \cite{CIM11,CJPT13} (when $g\equiv0$), which have been introduced as effective acceleration strategies. They generate the next iterate by minimizing the objective (if computationally affordable) or a constructed surrogate function over a subspace. Many variants to define these subspace have been considered. In contrast, we optimize for the extrapolation point on a subspace from where a gradient descent step yields the best decrease of a simple quadratic surrogate function. In general, both approaches differ, however majorizers constructed in subspace approaches could be incorporated also into our scheme.

\subsection{Contribution}

\paragraph{Convergence of an optimally extrapolated Proximal Gradient Method (PGM).} We study a novel, optimally extrapolated PGM for non-convex optimization problems. If the function is composed of a continuously differentiable function with Lipschitz continuous gradient and a non-smooth function, we prove that a subsequence converges to a stationary point and that objective values converge under reasonable conditions.

\paragraph{Equivalence to proximal quasi-Newton methods.} In a broad special non-convex setting, we prove our method to be equivalent to a proximal variant of the SR1 quasi-Newton method \cite{NW06}. To be more precise, we prove the equivalence between an optimally rank-$R$ extrapolated Proximal Gradient Method and an identity minus rank-$R$ proximal quasi-Newton method. Our convergence results can be applied directly to these methods, leading to the first convergence result of this SR1 proximal quasi-Newton for non-convex optimization problems.

\paragraph{Relation to \cite{BF12} and \cite{KV17}.} A related proximal quasi-Newton method was considered by Becker and Fadili in \cite{BF12}. However, they adapt a Barzilai--Borwein step length, whereas we relate the step length to the Lipschitz constant. Moreover, the proposed quasi-Newton metric in the evaluation of the proximal mapping is of type \enquote{identity plus a rank 1 matrix}, whereas ours is of type \enquote{identity minus rank 1}, and we allow for non-convex functions in the proximal mapping. Another closely related approach is that of Karimi and Vavasis \cite{KV17}, which applies to problems that are the sum of a convex quadratic and a convex non-smooth function. The considered proximal mapping is also of type \enquote{identity minus rank 1}, however the relation to the SR1 metric \cite[Section 8.2]{NW06} is unclear and the analysis is completely different to ours. The case of an \enquote{identity plus or minus a rank-$R$ matrix}, which generalizes \cite{BF12,KV17}, was recently studied in \cite{BFO18}.

\section{The Optimization Problem} \label{sec:opt-problem}

The setting of this paper is that of a Euclidean vector space $\R^N$ of dimension $N$ equipped with the standard inner product $\scal\cdot\cdot$ and induced norm $\vnorm{x} = \sqrt{\scal xx}$ for $x\in \R^N$. Moreover, we use the notation $\vnorm[\bmat V]{x}^2:= \scal{x}{x}_{\bmat V} := \scal{x}{\bmat V x}$ for a matrix $\bmat V\in \R^{N\times N}$. We denote by $\mathbb S_{+}(N)$ the set of symmetric positive semi-definite matrices in $\R^{N\times N}$ and by $\mathbb S_{++}(N)\subset\mathbb S_{+}(N)$ the subset of positive definite matrices.\\

We consider the optimization problem
\begin{equation} \label{eq:gen-opt}  \tag{P}
  \min_{x\in \R^N}\, f^g(x)\,,\quad f^g(x):= g(x) + f(x)
\end{equation}
with the following assumptions.
\begin{ASS} \label{ass:problem}
Let $\bmat L\in\mathbb S_{++}(N)$.
\begin{enumerate}
  \item\label{ass:problem:i} $\map{g}{\R^N}{\eR}$ is proper lower semi-continuous (lsc).
  \item\label{ass:problem:ii} $\map{f}{\R^N}{\R}$ is continuously differentiable and $\nabla (f\circ \bmat L^{-1/2})$ is $1$-Lipschitz continuous.

  \item\label{ass:problem:iii} $f^g$ is bounded from below.
\end{enumerate}
\end{ASS}
\begin{REM}
  Assumption~\ref{ass:problem}(\ref{ass:problem:ii}) is equivalent to $1$-Lipschitz continuity of the gradient of $f$ in a metric induced by $\bmat L$ (see Lemma~\ref{lem:Lipschitz-grad-equiv}), and implies a Generalized Descent Lemma (see Lemma~\ref{lem:gen-Descent-Lemma}).
\end{REM}

\section{The Adaptive Extrapolation Algorithm aFISTA}

Before we introduce the full algorithm in Section~\ref{sec:alg}, we present the main update step in Section~\ref{sec:update-step}. In a special setting, the update step is shown to be equivalent to a certain proximal quasi-Newton method without extrapolation in Section~\ref{sec:rel-to-quasi-Newton}. This relation reveals many interesting application, for which the main update step of our algorithm can be computed efficiently. Section~\ref{sec:mm} describes an approach that allows more general problems to be tackled with the same strategy. Additional strategies for the efficient computation of the update are discussed in Section~\ref{sec:efficient-update}. Convergence of the main algorithm is analyzed in Section~\ref{sec:convergence-analysis}. All proofs are given in the appendix.

\subsection{The Extrapolated Proximal Gradient Update Step} \label{sec:update-step}

Let $\ak{x}\in\R^N$ be the current point of the iterative scheme and $\set{d_1,\ldots,d_R}$ be vectors in $\R^N$ and
\begin{equation} \label{eq:extrapolate-y}
  y\idx{\bvec\beta}:= \ak{x} + \sum_{i=1}^R \beta_i d_i
\end{equation}
for $\bvec{\beta}:=(\beta_1,\ldots,\beta_R) \in \R^R$. In matrix--vector notation, we have $y\idx{\bvec\beta}= \ak{x} + \bmat D\bvec \beta$ where $\bmat D=(d_1,\ldots,d_R)\in \R^{N\times R}$ contains the vectors $d_1,\ldots,d_R$ as columns. We denote the by
\begin{equation} \label{eq:def-g-linearization}
    \ell_f^g(x; y\idx{\bvec\beta}) := g(x) + f(y\idx{\bvec\beta}) + \scal{\nabla f(y\idx{\bvec\beta})}{x-y\idx{\bvec\beta}},
\end{equation}
the linearized smooth function $f$ plus the non-smooth function
$g$. The extrapolated proximal gradient (EPG) step is given by the
following combined $x$ and $\bvec\beta$ proximal problem:
\begin{equation} \label{eq:aFista-step-gen-ho}   \tag{EPG}
    (\akp{x},\akp{\bvec\beta})\in \argmin_{x\in \R^N} \min_{\bvec\beta\in \R^R}\, \ell_f^g(x; y\idx{\bvec\beta}) + \frac 1{2} \vnorm[\bmatT]{x-y\idx{\bvec\beta}}^2 \,,
\end{equation}
where $\bmatT\in\mathbb S_{++}(N)$. We
also allow for inexact minimizers $(\akk{x}, \akk{\bvec\beta})$ of
\eqref{eq:aFista-step-gen-ho} as long as the following
condition is satisfied:
\begin{equation} \label{eq:aFista-inexact-step-gen-ho}  \tag{iEPG}
    \ell_f^g(\akk{x}; y\idx{\akk{\bvec\beta}}) + \frac 1{2} \vnorm[\bmatT]{\akk{x}-y\idx{\akk{\bvec\beta}}}^2 \leq f^g(\ak{x}).
\end{equation}
Observe that the right hand side of \eqref{eq:aFista-inexact-step-gen-ho} is the value of the objective in \eqref{eq:aFista-step-gen-ho} at $\ak{x}$ with $\beta=0$. Obviously, \eqref{eq:aFista-step-gen-ho} implies \eqref{eq:aFista-inexact-step-gen-ho}, i.e., $\akk{x}=\akp{x}$ and $\akk{\bvec\beta}=\akp{\bvec\beta}$ with $(\akp{x},\akp{\bvec\beta})$ from \eqref{eq:aFista-step-gen-ho} asserts the inexact condition in \eqref{eq:aFista-inexact-step-gen-ho}.

\subsection{The Algorithm} \label{sec:alg}

We embed the update step from Section~\ref{sec:update-step} into our main algorithm. While we state Algorithm~\ref{alg:afista-nc} with the exact update step \eqref{eq:aFista-step-gen-ho}, the convergence analysis in Section~\ref{sec:convergence-analysis} also accounts for the inexact version \eqref{eq:aFista-inexact-step-gen-ho}.
\mybox{
  \begin{ALG}[Adaptive Fista for Non-convex Problems with Backtracking] \ \label{alg:afista-nc}
    \vspace*{-3ex}
    \begin{itemize}
      \item \emph{Optimization Problem}: \eqref{eq:gen-opt} with Assumption~\ref{ass:problem}.
      \item \emph{Initialization}: $x\iter0=\bar x$ for some $\bar x\in \R^N$. Set $a>0$.
      \item \emph{Selection Rule}: Select extrapolation directions $\seq[\k\in\N]{\bmat D\iter\k}$ with $\bmat D\iter\k\in \R^{N\times R}$.
      \item \emph{Update for $\k\geq 0$}: Find $x\iter\kp$ and $\bvec\beta\iter\k$ such that the following holds:
      \begin{equation} \label{eq:update:alg:afista-nc} \tag{EPG}
            (x\iter\kp,\bvec\beta\iter\k) \in \argmin_{x\in\R^N}\, \min_{\bvec\beta\in \R^R}\, \ell_f^g(x; y\idx{\bvec\beta}) + \frac 1{2} \vnorm[\bmatT\piter\k]{x-y\idx{\bvec\beta}}^2

      \end{equation}
      with $y\idx{\bvec\beta} = x\iter\k + \bmat D\iter\k \bvec\beta$, where $\bmatT\piter\k - \bmat L\piter\k - a{\bm\opid} \in\mathbb S_{+}(N)$ and $\bmat L\piter\k$ satisfies
      \begin{equation} \label{eq:update:alg:afista-nc-L-cond}
        f(x\iter\kp) \leq f(y\idx{\bvec\beta\iter\k}) + \scal{\nabla f(y\idx{\bvec\beta\iter\k})}{x\iter\kp - y\idx{\bvec\beta\iter\k}} + \frac 12 \vnorm[\bmat L\piter\k]{x\iter\kp - y\idx{\bvec\beta\iter\k}}^2 \,.
      \end{equation}
    \end{itemize}
  \end{ALG}
}
\begin{REM}
  The extrapolation directions $\bmat D\iter\k$ may, for example, depend on $\seq[j\leq\k]{x\iter{j}}$ or $\k$.
\end{REM}
The underlying scheme of the algorithm is the extrapolated proximal gradient algorithm. It is complemented with a locally varying Lipschitz upper bound that resembles the Descent Lemma. However, we also allow for an anisotropic quadratic upper bound, represented by the metric $\bmatT\piter\k$. This may yield more flexibility and tighter bounds.
\begin{EX}
  Let $\bmatT\piter\k=\bmat L\piter\k = L\cdot{\bm\opid}$ with identity matrix ${\bm\opid}$ where $L>0$ is the Lipschitz constant of $\nabla f$ and $\bmat D\iter\k=x\iter\k-x\iter\km$. Then, \eqref{eq:update:alg:afista-nc-L-cond} is satisfied and, for $\bvec\beta\iter\k=\beta\piter\k\in\R$ satisfying\footnote{Since \eqref{eq:update:alg:afista-nc} minimizes over $x$ and $\bvec\beta$, the optimal $\bvec\beta$ may actually depend on $x$.} \eqref{eq:update:alg:afista-nc}, the update step \eqref{eq:update:alg:afista-nc} can be written as follows:
  \[
    \begin{split}
    y\idx{\beta\piter\k} =&\ x\iter\k + \beta\piter\k(x\iter\k - x\iter\km) \\
    x\iter\kp \in &\  \argmin_{x\in \R^N}\, g(x) + \frac L{2}
                 \vnorm{x-\big(y\idx{\beta\piter\k} - \frac 1L \nabla f(y\idx{\beta\piter\k})\big)}^2 \,,
    \end{split}
  \]
  which is an extrapolated proximal gradient step. In the case $f$ and $g$ are convex functions, and $\beta\piter\k$ is suboptimal, depending on $k$  like $\seq[\k\in\N]{1-3/\k}$, formally, we obtain the accelerated proximal gradient method, which is known as FISTA \cite{BT09b}. It achieves a convergence rate of $O(1/\k^2)$ in term of objective value residual.

As the proposed update step in \eqref{eq:update:alg:afista-nc} takes the same form, but optimizes the update with respect to $\bvec\beta$, each step guarantees a better objective value than an accelerated proximal gradient step. However, this is not enough to guarantee the same rate of convergence, which requires a global picture of the objective.
\end{EX}

\subsection{Relation to Proximal Quasi-Newton Methods} \label{sec:rel-to-quasi-Newton}

For a quadratic function $f$, the optimal parameter $\bvec \beta$ can be computed analytically. In Section~\ref{sec:mm}, we describe a strategy that applies this result in a more general setting. Using the optimal $\bvec\beta$ in the update step yields an interpretation of the update step \eqref{eq:aFista-step-gen-ho} as a proximal gradient step in a modified metric without extrapolation, also known as proximal quasi-Newton method. The modified metric is of type ``identity minus rank $R$'', where $R$ is the column rank of $\bmat D$. Proximal quasi-Newton methods have recently drawn attention, as for $R=1$, the associated proximal mapping can be evaluated efficiently \cite{BF12,KV17}. See~\cite{BFO18}, for the rank-$R$ case.
\begin{THM} \label{thm:aFista-equiv-ho}
  Let Assumption~\ref{ass:problem} hold and consider the problem in \eqref{eq:aFista-step-gen-ho}. Suppose that $f(x) = \frac 12 \scal{x}{\bmat H x}+\scal{b}{x} + c$ is quadratic with Hessian $\bmat H\in \mathbb S_{++}(N)$, $b\in \R^N$, $c\in \R$, the matrix $\bmatT$ is chosen such that $\bmat M:=\bmatT -\bmat H \in \mathbb S_{++}(N)$, and the columns of $\bmat D\in\R^{N\times R}$ are linearly independent.  Then, the inner optimization problem w.r.t. $\bvec \beta$ is solved by
\[
  \bvec\beta^* = (\bmat D^\top \bmat M \bmat D)^{-1} \bmat D^\top \bmat M (x-\ak{x})\,.
\]
The optimization problem in \eqref{eq:aFista-step-gen-ho} is equivalent to the following:
\begin{equation} \label{eq:thm:aFista-equiv-ho}
  \akp{x} \in\argmin_{x\in \R^N}\, g(x) + \frac 12 \vnorm[\bmat Q]{x- \ak{x} + \bmat Q^{-1}\nabla f(\ak{x})}^2 \,,
\end{equation}
where
\begin{equation} \label{eq:thm:aFista-equiv-ho-Qmatrix}
  {\bm Q}:= \bmatT  - \bmat U^\top\bmat U \in \mathbb S_{++}(N) \qquad \text{with}\qquad \bmat U := (\bmat D^\top \bmat M \bmat D)^{-\frac 12} \bmat D^\top \bmat M
\end{equation}
and $\bmat U^\top \bmat U$ is of rank $R$. The inverse metric is
\begin{equation} \label{eq:thm:aFista-equiv-ho-Qmatrix-inv}
  \bmat Q^{-1} = \bmatT ^{-1} + \bmatT ^{-1} \bmat U^\top ({\bm\opid} - \bmat U \bmatT ^{-1} \bmat U^\top)^{-1} \bmat U \bmatT ^{-1} \,.
\end{equation}
\end{THM}
The proof is in Section~\ref{proof:thm:aFista-equiv-ho}.
\begin{REM}
  The update step \eqref{eq:thm:aFista-equiv-ho} can be written equivalently as
  \begin{equation} \label{eq2:thm:aFista-equiv-ho}
    \akp{x} \in\argmin_{x\in\R^N}\, g(x) + f(\ak{x}) + \scal{\nabla f(\ak{x})}{x- \ak{x}} +  \frac 12 \vnorm[\bmat Q]{x- \ak{x}}^2 \,.
  \end{equation}
\end{REM}
\begin{REM} \label{rem:interpret-Q}
  The proof of Theorem~\ref{thm:aFista-equiv-ho} shows that, on $\im(\bmat D)$, which denotes the image of $\bmat D$, the matrix $\bmat Q$ coincides exactly with the Hessian matrix, i.e., $\bmat Q x = \bmat H x$ for $x\in \im(\bmat D)$, or equivalently, $\bmat Q\bmat D \bvec\beta=\bmat H\bmat D\bvec\beta$ for $\bvec\beta\in\R^R$. On the orthogonal complement $\im(\bmat D)^\bot=\ker(\bmat D^\top)$, the matrix $\bmat Q$ coincides with $\bmat T$, i.e. $\bmat Q x = \bmat T x$ for $x$ with $\bmat D^\top x=0$.
\end{REM}
\begin{REM} \label{rem:know-H-on-imQ}
  The formulas for the matrix $\bmat Q$ in \eqref{eq:thm:aFista-equiv-ho-Qmatrix} and $\bmat Q^{-1}$ in \eqref{eq:thm:aFista-equiv-ho-Qmatrix-inv} do not require explicit inversion of $\bmat H$. In fact, $\bmat H$ must be known on $\im(\bmat D)$ only. Using $\bmat Y:=\bmat H \bmat D$, the formula in \eqref{eq:thm:aFista-equiv-ho-Qmatrix} can be written as
  \[
    \bmat Q = \bmat T - (\bmat T \bmat D - \bmat Y) \big[\bmat D^\top (\bmat T\bmat D - \bmat Y) \big]^{-1} (\bmat T\bmat D - \bmat Y)^\top
  \]
  and \eqref{eq:thm:aFista-equiv-ho-Qmatrix-inv} as
  \[
    \bmat Q^{-1} = \bmat T^{-1} + (\bmat D - \bmat T^{-1} \bmat Y)\big[(\bmat D - \bmat T^{-1} \bmat Y)^\top \bmat Y\big]^{-1}(\bmat D - \bmat T^{-1} \bmat Y)^\top \,.
  \]
  The details of the computation are given in Section~\ref{proof:rem:know-H-on-imQ}. These formulas provide an elegant and efficient way to compute that application of $\bmat Q$ and its inverse.
\end{REM}

\paragraph{The rank 1 case.} In practical applications, the case where $\bvec\beta=\beta$ is 1-dimensional is most interesting, since the resulting quasi-Newton method is of type ``identity minus rank 1'',\linebreak for which the update steps can be evaluated efficiently. In fact, it turns out that in this case \eqref{eq:aFista-step-gen-ho} is equivalent to a generalization of the SR1 quasi Newton method with a non-smooth term, i.e., a proximal SR1 quasi Newton method.
\begin{COR} \label{cor:aFista-equiv}
Consider the situation of Theorem~\ref{thm:aFista-equiv-ho} with $R=1$, i.e. $\bmat D = d\in \R^N$. Then, the inner optimization problem w.r.t. $\bvec\beta=\beta\in\R$ is solved by
\[
  \opt\beta = \frac{\scal{d}{x-\ak{x}}_{\bm M}}{\scal{d}{d}_{\bm M}}\,.
\]
The optimization problem in \eqref{eq:aFista-step-gen-ho} is equivalent to the following:
\begin{equation} \label{eq:id-minus-rank1-fbs-step}
  \akp{x} \in\argmin_{x\in \R^N}\, g(x) + \frac 12 \vnorm[\bmat Q]{x- \ak{x} + {\bm Q}^{-1}\nabla f(\ak{x})}^2 \,,
\end{equation}
where
\begin{equation} \label{eq:id-minus-rank1-metric}
  \bmat Q:= \bmatT  - u u^\top \quad \text{with}\quad u := \frac{{\bm M} d}{\vnorm[{\bm M}]{d}}
\quad\text{and}\quad
  \bmat Q^{-1} = \bmatT ^{-1} + \frac{\bmatT ^{-1} u u^\top \bmatT ^{-1}}{1-u^\top \bmatT ^{-1} u} \,.
\end{equation}
\end{COR}
\begin{proof}
  The statement is an obvious consequence of Theorem~\ref{thm:aFista-equiv-ho}.
\end{proof}
\begin{REM} \label{rem:prox-SR1-metric}
  We explain the naming \enquote{proximal SR1 quasi-Newton method}. The standard notation of quasi-Newton methods uses $\overline{\bmat B}\piter\k$ for the current approximation of the Hessian matrix and $\overline{\bmat H}\piter\k$ for the approximation of the inverse Hessian matrix. We use the ``overline'' to distinguish $\overline{\bmat H}\piter\k$ from the second derivative $\bmat H$ of $f$ in Theorem~\ref{thm:aFista-equiv-ho} and Corollary~\ref{cor:aFista-equiv}. The SR1 update formula \cite[Section 8.2]{NW06} is given by
\[
  \begin{split}
  \overline{\bmat B}\piter\kp
  =&\  \overline{\bmat B}\piter\k + \frac{(y\iter\k-\overline{\bmat B}\piter\k d\iter\k)(y\iter\k-\overline{\bmat B}\piter\k d\iter\k)^\top}{\scal{d\iter\k}{y\iter\k-\overline{\bmat B}\piter\k d\iter\k}}\,, \\
  \overline{\bmat H}\piter\kp
  =&\  \overline{\bmat H}\piter\k + \frac{(d\iter\k-\overline{\bmat H}\piter\k y\iter\k)(d\iter\k - \overline{\bmat H}\piter\k y\iter\k)^\top}{\scal{d\iter\k-\overline{\bmat H}\piter\k y\iter\k}{y\iter\k}} \,.
  \end{split}
\]
  Setting $R=1$, $d=x\iter\k-x\iter\km$, $y=\bmat H d=\nabla f(x\iter\k) - \nabla f(x\iter\km)$ in \eqref{eq:id-minus-rank1-metric} yields:
\[
  \bmat Q = \bmatT  + \frac{(y-\bmatT  d)(y-\bmatT d)^\top}{\scal{d}{y-\bmatT  d}}\quad\text{and}\quad
  \bmat Q^{-1} = \bmatT ^{-1} + \frac{(d-\bmatT ^{-1} y)(d - \bmatT ^{-1}y)^\top}{\scal{d-\bmatT ^{-1} y}{y}} \,,
\]
which is that same as Remark~\ref{rem:know-H-on-imQ} specified for $R =1$. The identification $\overline{\bmat B}\piter\k = \bmat T$ and $\overline{\bmat B}\piter\kp=\bmat Q$ shows that aFISTA in \eqref{eq:id-minus-rank1-fbs-step} uses a $0$-memory SR-1 metric, i.e., we re-initialize $\overline{\bmat B}\piter\k$ in each iteration with $\bmat T$.

  Unfortunately, the memory based SR1 update formula does not preserve the positive definiteness, which makes the convergence analysis for line search based algorithms difficult. For more details on this discussion, we refer to \cite[Section 8.2]{NW06}. Our $0$-memory variant guarantees positive definiteness of $\bmat Q$.

  In case we restart the approximation of the Hessian matrix in each iteration with a multiple of the identity matrix (with a Barzilai--Borwein rule), we obtain the recently proposed zero memory version of the SR1 quasi-Newton method \cite{BF12}.
\end{REM}

\subsection{A Majorization--Minimization Strategy} \label{sec:mm}

Using the majorization--minimization strategy \cite{HL04}, we can apply the results from the previous section to more general problems. This idea is also pursued in \cite{CIM11,CJPT13} for smooth (possibly non-convex) functions (i.e. $g\equiv 0$). However, we consider the full non-convex setting (see Assumption~\ref{ass:problem}). At the current iterate $\ak{x}$, we define a quadratic tangent majorizer\footnote{A tangent majorizer $Q_f(x,\ak{x})$ coincides with $f$ at $x=\ak{x}$, has the same slope as $f$ at $\ak{x}$ and $Q_f(x,\ak{x})\geq f(x)$ holds for all $x$.} of $f$, for example,
\begin{equation} \label{eq:mm:our-quad-tangent-majorizer}
  Q^{\bmat L}_f (x, \ak{x}) := f(\ak{x}) + \scal{\nabla f(\ak{x})}{x-\ak{x}} + \frac 12 \vnorm[\bmat L]{x-\ak{x}}^2\,,
\end{equation}
which is chosen according to the quadratic upper bound from the Generalized Descent Lemma (Lemma~\ref{lem:gen-Descent-Lemma}). Let $\akp{x}$ be the solution of one iteration of \eqref{eq:aFista-step-gen-ho} applied to $g(x) + Q^{\bmat L}_f (x, \ak{x})$ with $\bmat T$ such that $\bmat T-\bmat L \in \mathbb S_{++}(N)$. This setting allows us to apply Theorem~\ref{thm:aFista-equiv-ho} with $\bmat Q:= \bmat T - \bmat U^\top \bmat U$ and $\bmat H=\bmat L$ to compute $\akp{x}$. Then, the following relation holds:
\[
  \begin{split}
  f^g(\akp{x})
  \overset{\ii1}{\leq}&\  g(\akp{x}) + Q^{\bmat L}_f(\akp{x}, \ak{x}) \\
  \overset{\ii2}{\leq}&\  g(\akp{x}) + Q^{\bmat Q}_f(\akp{x},\ak{x}) - \frac{1}{2}\vnorm[\bmat Q-\bmat L]{\akp{x}-\ak{x}}^2 \\
  \overset{\ii3}{\leq}&\  g(\ak{x}) + Q^{\bmat Q}_f(\ak{x},\ak{x}) - \frac{1}{2}\vnorm[\bmat Q-\bmat L]{\akp{x}-\ak{x}}^2 \\
  \leq&\  f^g(\ak{x}) - \frac{1}{2}\vnorm[\bmat Q-\bmat L]{\akp{x}-\ak{x}}^2 \\
  \end{split}
\]
where $\ii1$ uses the properties of a tangent majorizer, $\ii2$ uses Theorem~\ref{thm:aFista-equiv-ho}, and $\ii3$ uses \eqref{eq2:thm:aFista-equiv-ho}. Remark~\ref{rem:interpret-Q} shows that
\[
    \vnorm[\bmat Q-\bmat L]{\akp{x}-\ak{x}}^2 = \vnorm[\bmat T - \bmat L]{\akp{x}^0-\ak{x}^0}^2 \geq c \vnorm{\akp{x}^0-\ak{x}^0}^2
\]
holds for some $c>0$, where $\akp{x}^0-\ak{x}^0$ is the projection of $\akp{x}-\ak{x}$ onto $\ker(\bmat D^\top)$, which is a sufficient decrease condition. If a sufficient decrease condition with respect to all coordinates of $\akp{x}-\ak{x}$ is desired, we can perform the update step \eqref{eq:thm:aFista-equiv-ho} with $\bmat Q = \bmat T - \rho \bmat U^\top \bmat U$ for some $\rho\in [0,1)$.

The algorithm is summarized in Algorithm~\ref{alg:mm-afista-nc}.
\mybox{
  \begin{ALG}[MM based Adaptive Fista for Non-convex Problems] \ \label{alg:mm-afista-nc}
    \begin{itemize}
      \item \emph{Optimization Problem}: \eqref{eq:gen-opt} with Assumption~\ref{ass:problem}.
      \item \emph{Initialization}: $x\iter0=\bar x$ for some $\bar x\in \R^N$. Set $\rho\in[0,1)$.
      \item \emph{Selection Rule}: Select extrapolation directions $\seq[\k\in\N]{\bmat D\iter\k}$ with $\bmat D\iter\k\in \R^{N\times R}$.
      \item \emph{Update for $\k\geq 0$}: Define a tangent majorizer $Q^{\bmat L\iter\k}_f(x,x\iter\k)$ around $x\iter\k$, \linebreak set $\bmat Q\iter\k=\bmat T\iter\k - \rho \bmat U\iter\k^\top \bmat U\iter\k$ with $\bmat U\iter\k$ as in Theorem~\ref{thm:aFista-equiv-ho}, and compute $x\iter\kp$ as follows:
      \begin{equation} \label{eq:update:alg:afista-nc-MM}
        x\iter\kp \in  \argmin_{x\in \R^N}\, g(x) + \frac 12 \vnorm[\bmat Q\iter\k]{x- x\iter\k + \bmat Q\iter\k^{-1}\nabla f(x\iter\k)}^2
      \end{equation}
      where $\bmat T\iter\k - \bmat L\iter\k \in \mathbb S_{++}(N)$.
    \end{itemize}
  \end{ALG}
}
The convergence of this algorithm was analyzed in the context of variable metric forward--backward splitting in \cite{CPR13} when $g$ is a convex function. Assuming the Kurdyka--{\L}ojasiewicz\xspace inequality holds, which is basically satisfied by any function appearing in practical applications, they show convergence of the full sequence generated by the algorithm to a stationary point. \cite{CPR16} covers also the case of a non-convex function $g$.
\begin{REM}
  In practice, there are several ways to construct tangent majorizers. This issue has been considered, for example, in \cite{CIM11,CJPT13} in the context of subspace optimization algorithm. Another possibility is given by approximating the Hessian matrix at $x\iter\k$ by a symmetric positive definite matrix, which can be achieved, for example, by adding a multiple of the identity matrix or by projection onto $\eps\cdot{\bm\opid}+\mathbb S_{++}(N)$ for some $\eps>0$, if the Hessian is not positive definite \cite{NW06}.
\end{REM}

\subsection{Efficient Computation of the Update Step} \label{sec:efficient-update}

Here, we discuss the efficient computation of the update step used in Algorithms~\ref{alg:afista-nc} and~\ref{alg:mm-afista-nc}.
\begin{itemize}
  \item If $f$ is a convex quadratic function or Algorithm~\ref{alg:mm-afista-nc} is used, and $\bmat T\iter\k$ is a diagonal matrix, the update step is a proximal step of type ``identity minus rank $R$''. According to \cite{BFO18}, such proximal mappings can be computed efficiently. The computation can be reduced to proximal mappings with respect to the diagonal part of the metric and $R$-dimensional root finding problems, which can be solved efficiently thanks to their small dimension.
  \item In the same setting as above, but with $R=1$, \cite{BF12} provides a list of efficiently computable rank-1 proximal mapping, which includes $g$ being the $\ell_1$-norm, group $\ell_{1,2}$-norms, $\ell_\infty$-norm, simplex, box, or non-negativity constraints, etc. However, note that we also allow for non-convex functions $g$.
  \item Unless we can find analytic formulas in special cases, the update step in \eqref{eq:update:alg:afista-nc} needs to be computed numerically. For example, this can be achieved by alternating minimization. The minimization with respect to $\bvec\beta$ is a $R$-dimensional smooth minimization problem, and the minimization with respect to $x$ is a proximal gradient step around the extrapolated point $y\idx{\bvec\beta}$. The subproblem w.r.t. $\bvec \beta$ is convex (quadratic), if $f$ in \eqref{eq:update:alg:afista-nc} is convex (quadratic). In particular, this is true when the majorization--minimization strategy (Section~\ref{sec:mm}) is used.
  \item Finally, we can also allow to inexactly computed update steps. The main requirement is a sufficient improvement on the objective function in \eqref{eq:update:alg:afista-nc} from $x\iter\k$ to $x\iter\kp$. This allows for an efficient backtracking strategy for $\bvec\beta$, which always succeeds with $\bvec\beta=0$. Therefore, we can try a few samples of $\bvec\beta$ before falling back to $\bvec\beta=0$, e.g. for $R=1$, we can use Nesterov's sequence $\beta\piter\k\to1$ for $\k\to\infty$.
\end{itemize}

\subsection{Convergence Analysis} \label{sec:convergence-analysis}

Let us analyze Algorithms~\ref{alg:afista-nc} and~\ref{alg:mm-afista-nc}. Preliminaries from non-smooth analysis are provided in Section~\ref{appdx:prelim}.

\begin{PROP}[Convergence of Adaptive Fista] \label{prop:conv-nc}
  Let $\seq[\k\in\N]{x\iter\k}$, $\seq[\k\in\N]{\bvec\beta\iter\k}$, $\seq[\k\in\N]{\bmat D\iter\k}$, $\seq[\k\in\N]{\bmat L\iter\k}$ and $\seq[\k\in\N]{\bmat T\iter\k}$ be as in Algorithm~\ref{alg:afista-nc} for solving \eqref{eq:gen-opt} with Assumption~\ref{ass:problem}.  Suppose $\seq[\k\in\N]{\bmat T\piter\k}$ is bounded. Then, the sequence of objective values $\seq[\k\in\N]{f^g(x\iter\k)}$ is non-increasing and every limit point of $\seq[\k\in\N]{x\iter\kp}$ is a stationary point of $f^g$.
\end{PROP}
The proof is in Section~\ref{proof:prop:conv-nc}.
\begin{REM}
  The boundedness assumption for $\seq[\k\in\N]{\bmat T\piter\k}$ can always be satisfied thanks to Assumption~\ref{ass:problem}(\ref{ass:problem:ii}).
\end{REM}
The existence of a convergent subsequence converging to a stationary point can be derived analogously to Proposition~\ref{prop:conv-nc}. As we have mentioned above, the convergence of such variable metric proximal gradient algorithm is known \cite{CPR13}.

In the following statement, we prove convergence for an inexact version of Algorithm~\ref{alg:afista-nc}. Of course, this more general setting requires a few more assumptions.
\begin{PROP}[Convergence of Inexact Adaptive Fista] \label{prop:conv-inc}
  Let $\seq[\k\in\N]{x\iter\k}$, $\seq[\k\in\N]{\bvec\beta\iter\k}$, $\seq[\k\in\N]{\bmat D\iter\k}$, $\seq[\k\in\N]{\bmat L\iter\k}$ and $\seq[\k\in\N]{\bmat T\iter\k}$ be as in Algorithm~\ref{alg:afista-nc} with \eqref{eq:update:alg:afista-nc} replaced by \eqref{eq:aFista-inexact-step-gen-ho}, i.e., find $x\iter\kp$ and $\bvec\beta\piter\k$ that satisfy
  \begin{equation} \label{prop:conv-inc:inexact-update}
    \ell_f^g(x\iter\kp; y\idx{\bvec\beta\piter\k}) + \frac 1{2} \vnorm[\bmatT]{x\iter\kp-y\idx{\bvec\beta\piter\k}}^2 \leq f^g(x\iter\k)\,,
  \end{equation}
  for solving \eqref{eq:gen-opt} with Assumption~\ref{ass:problem}.  Suppose $\seq[\k\in\N]{\bmat T\piter\k}$ is bounded. Then, the sequence of objective values $\seq[\k\in\N]{f^g(x\iter\k)}$ is non-increasing. If, additionally, $g$ is continuous on its domain and $\norm[-]{\partial f^g(x\iter\kp)}\to 0$ for $\k\to\infty$, then every limit point of $\seq[\k\in\N]{x\iter\kp}$ is a stationary point of $f^g$.
\end{PROP}
The proof is in Section~\ref{proof:prop:conv-inc}.

\subsubsection{Discussion of the Convergence Rate}
  In the convex setting, we can easily derive a convergence rate of $O(1/\k)$ w.r.t. the function gaps for Algorithms~\ref{alg:afista-nc} and~\ref{alg:mm-afista-nc}. However, we cannot guarantee an optimal convergence rate of $O(1/\k^2)$ as for FISTA. Unlike FISTA, our algorithm performs a local optimization, and hence does not exploit the global properties of convex functions. Interestingly, the conjugate gradient method also performs a local optimization (in both the step size and the extrapolation parameter) and still guarantees an optimal $O(1/\k^2)$ convergence rate. Hence it will be interesting in future work to study an extension of the proposed algorithm which also performs a locally optimal choice of the step size parameter.

  However, we can derive hybrid accelerated algorithms that make use of the proposed update step \eqref{eq:aFista-step-gen-ho}. Since, the main focus of this paper is the equivalence between the adaptive extrapolation method and quasi-Newton methods, we do not provide further details here. For the sake of completeness, we list the mentioned variants in the appendix in Section~\ref{sec:acc-aFista} and prove their optimal rate of convergence.

\section{Numerical Experiments} \label{sec:numerics}

For optimization problems \eqref{eq:gen-opt} where $f$ is a quadratic function and $g$ is convex, adaptive FISTA (aFISTA) is equivalent to the zero memory SR1 quasi-Newton method with step size $\bmatT\piter\k = \tau\piter\k^{-1} {\bm\opid}$ and $\tau\piter\k < 1/L$, instead of Barzilai--Borwein step sizes. Therefore, its efficiency has already been demonstrated in \cite{BF12}.  In Section~\ref{sec:exp-nn}, we consider a non-convex and non-smooth setting.

\subsection{Sparsity regularized non-linear inverse problem}  \label{sec:exp-nn}

In this experiment, we consider a neural network formulation of a one
dimensional regression problem. We are given $N=80$ noisy samples $(X,\tilde Y)\in
(\R^{1\times N})^2$ of the function $\map{F}{[-3,3]}{\R}$, $x\mapsto x^3
+ \cos(5x)$, arranged as corresponding columns of the matrices $X$ and $\tilde
Y$, i.e.,
\[
  \tilde Y_{1,i} := F(X_{1,i}) + E_{1,i} \,,
\]
where $E\in \R^{1\times N}$ is an additive noise matrix that models Gaussian
noise with standard deviation $\frac 32$ and 20 randomly scaled outliers.
The neural network optimization problem (non-linear inverse problem)
that we consider is formulated as follows:
\begin{gather} \label{eq:min-prob-SimpleNet}
  \min_{\substack{W_0, W_1, W2\\ b_0, b_1, b_2}}\, \sum_{i=1}^N \Big(\vnorm{(W_2\sigma_2(W_1 \sigma_1(W_0 X+B_0 )+B_1)+B_2 - \tilde Y)_{1,i} }^2 + \eps^2 \Big)^{1/2} + \lambda \sum_{j=0}^2 \norm[1]{W_j} \,, \\
  B_j := b_j \bvec 1^\top\,,\quad \bvec 1^\top := (1,\ldots, 1)\,, \quad j=1,2,3\,,
\end{gather}
where $D_0=D_3=1$, $D_1=D_2=10$, $W_j\in \R^{D_j\times D_{j-1}}$, $B_{j+1}\in
\R^{D_j\times N}$, for $j=1,2,3$, and $\map{\sigma_j}{\R^{D_j\times
N}}{\R^{D_j\times N}}$, $A\mapsto (\max(0,(A_{i,l}^2+\eps^2)^{1/2}))_{i,l}$,
for $j=1,2$, with $\eps = 0.1$. We used $\lambda=1$, which led to a sparsity
level of about $87\%$ of the coordinates.

We run Algorithm~\ref{alg:afista-nc}, denoted aFISTA, with inexact update \eqref{eq:aFista-inexact-step-gen-ho} and backtracking strategy for the extrapolation ($\beta\piter\k\in\set{2,1,0}$ and $\bmat D\iter\k=x\iter\k-x\iter\km$), and Algorithm~\ref{alg:mm-afista-nc}, denoted ZeroSR1, with majorizer computed using the (heuristic) SR1 metric (cf. Remark~\ref{rem:prox-SR1-metric}), which leads to the algorithm in \cite{BF12} (with line-search). Besides our methods, we evaluate Forward--Backward Splitting (FBS), iPiano with $\beta=0.95$ \cite{OCBP14}, and monotone FISTA (MFISTA) \cite{LL15}, a variant of FISTA with convergence guarantees in the non-convex setting. We used the same heuristic step size $\alpha=5\cdot 10^{-5}$ for all methods.  Throughout several experiments, the performance of aFISTA was on a par with MFISTA. The convergence for one problem instance is shown in Figure~\ref{fig:Comparison-SimpleNet}. In this experiment, aFISTA is slightly better and finds a lower objective value. Since aFISTA used a small number of backtracking iterations, the performance with respect to time is also competitive with MFISTA. The ZeroSR1 method is competitive in the beginning.
\begin{figure}[t]
  \begin{center}
\ifShowFigures
    \begin{tikzpicture}
      \begin{axis}[%
        width=0.48\linewidth,%
        height=7.5cm,%

        xmode=log, ymode=log,
        xlabel={iteration $\k$},%
        ylabel=$h(x\iter\k) / h(x\iter0)$,%
        legend columns=5,
        legend cell align=left,
        legend style={at={(1.0,1.02)},anchor=south,font=\footnotesize,column sep=10pt}
        ]
      \addplot[very thick,blue!80!white,solid,mark=triangle,mark repeat={50},mark size={2},mark options={solid}] %
          table {figures/SimpleSparseNet_conv_FBS_iter.dat};
          \addlegendentry{FBS};
      \addplot[very thick,red!80!black,solid,mark=star,mark repeat={50},mark size={2},mark options={solid}]
          table {figures/SimpleSparseNet_conv_iPiano_iter.dat};
          \addlegendentry{iPiano};
      \addplot[very thick,yellow!80!green,solid,mark=square,mark repeat={50},mark size={2},mark options={solid}] %
          table {figures/SimpleSparseNet_conv_mFISTA_iter.dat};
          \addlegendentry{MFISTA};

      \addplot[very thick,green!80!black,solid,mark=*,mark repeat={50},mark size={2},mark options={solid}]
          table {figures/SimpleSparseNet_conv_aFISTA_iter.dat};
          \addlegendentry{aFISTA};
      \addplot[very thick,cyan!80!black,solid,mark=o,mark repeat={50},mark size={2},mark options={solid}]
          table {figures/SimpleSparseNet_conv_zeroSR1_PG_iter.dat};
          \addlegendentry{ZeroSR1};
      \end{axis}
      \begin{scope}[xshift=0.47\linewidth]
      \begin{axis}[%
        width=0.48\linewidth,%
        height=7.5cm,%

        xmode=log,

        xlabel={time [sec]}%
        ]
      \addplot[very thick,blue!80!white,solid,mark=triangle,mark repeat={50},mark size={2},mark options={solid}] %
          table {figures/SimpleSparseNet_conv_FBS_time.dat};
      \addplot[very thick,red!80!black,solid,mark=star,mark repeat={50},mark size={2},mark options={solid}]
          table {figures/SimpleSparseNet_conv_iPiano_time.dat};
      \addplot[very thick,yellow!80!green,solid,mark=square,mark repeat={50},mark size={2},mark options={solid}] %
          table {figures/SimpleSparseNet_conv_mFISTA_time.dat};

      \addplot[very thick,green!80!black,solid,mark=*,mark repeat={50},mark size={2},mark options={solid}]
          table {figures/SimpleSparseNet_conv_aFISTA_time.dat};
      \addplot[very thick,cyan!80!black,solid,mark=o,mark repeat={50},mark size={2},mark options={solid}]
          table {figures/SimpleSparseNet_conv_zeroSR1_PG_time.dat};
      \end{axis}
            \end{scope}
    \end{tikzpicture}
\else
  \texttt{Activate Figure Later: Comparison Simple Net}
\fi
  \end{center}
  \caption{\label{fig:Comparison-SimpleNet}Convergence plots for the methods
  described in Section~\ref{sec:exp-nn} for solving
  \eqref{eq:min-prob-SimpleNet}. The vertical axis is the same for both plots.
  In this experiment, our method aFISTA outperforms the other methods with
  respect to the actual computation time and the final objective value.}
\end{figure}

\section{Conclusion}

In this paper, we analyzed a non-convex variant of the well-known FISTA, where
the extrapolation parameter is adaptively optimized in each iteration, which we
call aFISTA. In the special case where the smooth part of the split objective
function is quadratic, aFISTA is equivalent to a certain proximal
quasi-Newton method, which unlike the general class of proximal quasi-Newton
methods, allows for efficient solutions of the proximal mapping. This
equivalence relies on a reformulation of the quadratic function and is not
influenced by the non-smooth part of the objective, which may also be
non-convex. It provides a different view on quasi-Newton methods, which allows
for accelerated variants.

The general convergence of aFISTA is studied for non-convex objective functions that are the sum of a continuously differentiable function with Lipschitz continuous gradient and a proper lower semi-continuous non-smooth function with simple proximal mapping. We prove that a subsequence converges to a stationary point in terms of the limiting subdifferential. In numerical experiments, aFISTA and its variants have been shown to be competitive with the state-of-the-art.

In future work, we will explore the relationship between the adaptive
extrapolation strategy (aFISTA) with $R$ linearly independent directions and
other well known quasi-Newton methods such as BFGS. Of course, this
requires efficient solution algorithms for rank-$R$ proximal mappings. Moreover,
our perspective of the SR1 quasi-Newton method might also lead to new convergence
results when the construction of the metric uses memory of previous Hessian
approximations. As a third direction of future research, the usage of Bregman
distances needs to be investigated. For strongly convex Bregman functions,
convergence should not be difficult to proof, however, it is not clear which
\enquote{rank-$1$} Bregman proximal mappings can be solved efficiently.

redpajama_set_name: RedPajamaArXiv


Motor driven roll per single, double or triple drums possible.
Per motor driven roll a variable speed is possible.
Ventilation unit tailor-made to fit for every type of drum.
Special exhaust outlet to prevent air escaping along the seed instead of through it.


text:
Q: How do I prevent emacs from loading `update_terminal_cwd` with autojump? On MacOSX 10.7, Apple puts into /etc/bashrc a function: update_terminal_cwd that checks to see if Apple Terminal is running AND if we are not in emacs (snippet pasted below). Unfortunately, by including autojump into my bashrc config file, invoking an inferior shell (i.e. M-x shell) results in update_terminal_cwd being invoked. I get this output for any command invoked in the inferior shell:
bash: update_terminal_cwd: command not found
achan[10:29 AM]~ > echo $PROMPT_COMMAND

When I include autojump in my bashrc, echo $PROMPT_COMMAND inside of emacs looks like this:
{ [[ "$AUTOJUMP_HOME" == "$HOME" ]] && (autojump -a "$(pwd -P)"&)>/dev/null 2>>"${AUTOJUMP_DATA_DIR}/.autojump_errors";} 2>/dev/null ; update_terminal_cwd;

If I don't include autojump, I just get a null string for echo $PROMPT_COMMAND.
In the autojump code, it is inserting the first part:
{ [[ "$AUTOJUMP_HOME" == "$HOME" ]] && (autojump -a "$(pwd -P)"&)>/dev/null 2>>"${AUTOJUMP_DATA_DIR}/.autojump_errors";} 2>/dev/null ;

The update_terminal_cwd is being added just by appending $PROMPT_COMMAND to the end of the above string.
Somehow, autojump is being instantiated in an environment where:
[ "$TERM_PROGRAM" == "Apple_Terminal" ] && [ -z "$INSIDE_EMACS" ]
is true. But, because it is being run in emacs, update_terminal_cwd isn't defined; hence the error.
The autojump folks have suggested that I use iTerm2 instead or figure out why $INSIDE_EMACS isn't working.
I suspect it's something like the following:
Invoking M-x shell is somehow tied to the shell environment that already exists when emacs is invoked. This is why changing the .bashrc file while emacs is running doesn't propagate the changes to emacs. Instead, you to have to create a new bash environment and then invoke emacs with an inferior shell before you see those changes.
Is there a way to tell an inferior shell to disregard the default environment and try to start up bash from scratch?
Apple /etc/bashrc
# System-wide .bashrc file for interactive bash(1) shells.
if [ -z "$PS1" ]; then
   return
fi

PS1='\h:\W \u\$ '
# Make bash check its window size after a process completes
shopt -s checkwinsize
# Tell the terminal about the working directory at each prompt.
if [ "$TERM_PROGRAM" == "Apple_Terminal" ] && [ -z "$INSIDE_EMACS" ]; then
    update_terminal_cwd() {
        # Identify the directory using a "file:" scheme URL,
        # including the host name to disambiguate local vs.
        # remote connections. Percent-escape spaces.
    local SEARCH=' '
    local REPLACE='%20'
    local PWD_URL="file://$HOSTNAME${PWD//$SEARCH/$REPLACE}"
    printf '\e]7;%s\a' "$PWD_URL"
    }
    PROMPT_COMMAND="update_terminal_cwd; $PROMPT_COMMAND"
fi

A: I know this topic is old, but I had the same issue and found no solution (and this page is the first google result).
Here is my fix (might not be the best, but works):
if [ `(type update_terminal_cwd | grep -q 'shell function' | wc -l) 2> /dev/null` -eq 0 ]; then
    update_terminal_cwd() {
        return
    }
fi

[[ -s `brew --prefix`/etc/autojump.sh ]] && . `brew --prefix`/etc/autojump.sh

Regards,

A: After M-x shell you can get a shell with an fairly empty environment (started with the same options that emacs uses) by:
exec -c bash --noediting -i

or you can place the above command in ~/.emacs_bash or ~/.emacs.d/init_bash.sh
If that removes too much of the environment, instead of the exec maybe you can just set $PROMPT_COMMAND to a correct value inside the .emacs_bash startup script.

A: I upgraded to 24.3.50.1 and am not seeing this issue. That's a good solution (i.e. upgrade to the latest emacs).

redpajama_set_name: RedPajamaStackExchange


Below are three dangerous areas that become very evident when my clients and I journey deeper into EOS and become more precise and accurate with corporate language and nomenclature*.
1. Death by pronoun. Everyone understand what I am saying here? Using pronouns instead of nouns is just lazy and is a breeding ground for more lazy. (Yes, profanity in my world is a form of pronoun). Pronouns create confusion and complexity. Stop.
Leadership Challenge: Ask your folks to stop using pronouns, start by having them replace he and she or him and her with real names, like John, Jane, Julie, Josh. One tactic we use to remind everyone is when they are talking about children, spouses etc. is to have them use the child's formal name. Example: "My Son" becomes Bobby.
2. Death by hiding behind confusing, non-sense language. Anybody know what I am saying here? You have a teammate who dodges everything by just repeating or spouting blather. His words form nice sentences, but the content is worthless. Barbarians were called barbarians by the Greeks because they spoke Bar-Bar, what the Greeks called a different language.
Leadership Challenge: This comes out especially on remote worker conference calls... Example: During a video call, David, our remote worker, will be called on for his opinion... while David is clearly "multi-tasking" on another screen. Shaken awake David goes into a 48-word recall of a few words he sort of heard that form clear sentences, but, really misses the point, it is non-sense. Don't let this slide, call David out, ask him to clarify his statement using very precise company language.
3. Death by non-engaged employees who refuse to use our language. This is the biggest take away from my clients that I want to share. Listen to how your employees speak, if they are engaged, bought in, they will be very precise with your corporate nomenclature - you can hear it. For example, if a culture refers to fellow workers as "Teammates", that is the nomenclature, the correct word, not "Employees." Those that are not engaged will be loose with their language. It just comes out and they do not even know it.
A recent example was when a COO insisted on calling their Teammates, Employees. The COO did not care, did not agree, or just did not get it... "We call our fellow workers Teammates, not Employees!" said the CEO over and over and over. Finally, with this awareness of language precision being a direct measure of engagement, this bulky iceberg scraping along the company's hull woke up the owners and they made the necessary moves they needed to make to get an engaged person into their COO seat.
Leadership Challenge: Make a list of your folks and then start listening, I mean really listening. Are they repeating your language, are they using the correct words, are they using them in the correct context? Rate each on a scale of 1-10 with a 10 being someone you know is engaged and is precise... If you are rating them below an 8, imagine what your customers and partners are thinking.
* Nomenclature: is a system of names or terms, or the rules for forming these terms in a particular field of arts, sciences, business, work. Quick examples: Port is port, starboard is starboard on a ship, boat, plane... there is no right or left. There is a bow and a stern... there is no front or back. We go forward or we go aft, not front or back. Very precise language leaving nothing to chance.
Download and utilize EOS tools like the People Analyzer™ to make sure you have the right people on the bus.


GRWM- NIGHT OUT W/ FRIENDS (Simple and Easy!)- Makeup, Outfit and Hair!
Hey Guys!! Its Heaven ! I decided to do a simple gram since I'm going out with my friends. Sorry I haven't been posting regularly! EXAMS KILL ME.
anyways, hope you enjoy!! Make sure to like and subscribe ! Thanks!!!!
❤KEEP UP WITH ME !
Please subscribe for more videos every Wednesday!! Love you all!!
❤Hi, I am Heaven and I do beauty, diy, fashion, and lifestlyle types of videos. I am 15 and I just want to spread my love for beauty and creativity around the world! I hope you will subscribe to support me and see more videos!!
thank you!! I'll be sure to check them out !


We knew the bus to Huancayo would be long. Twelve hours to be exact (and that's if everything went smoothly). As it was supposed to be a scenic ride, we even opted to take the cheaper daytime bus.
Just judging by the fairly dilapidated look of the bus, we instantly knew this was going to be a rough and interesting ride.
-Driving on a bridge with seriously two inches of clearance on each side. I am still amazed we made it through without hitting anything.
You could literally touch the bridge from the bus... scary!
-Driving on rough and bumpy dirt roads. We've actually gotten quite used to this. We weren't really even fazed by the fact that this road had a sheer drop off on one side. However, this road did have a slight lean (towards the cliff, of course) that made taking those bumps a bit more interesting.
-Learning that our bathroom break was a small ravine on the side of the road. Kristen bonded with a few nice Peruvian ladies over that one.
-No TV with bad movies to distract us. The TV looked like it was last functional before we were born.
-Staring at the gorgeous mountain vistas. I think by now we must have seen every mountain in Peru.
-The extra long bus "infomercial." Often times people board the bus and do a little presentation. Billy Mays would have been proud of this guy- he "presented" for a half hour! The people were receptive though, and he sold quite a bit. His spoke slowly and clearly so it gave Kristen a chance to practice hearing and understanding Spanish. Despite understanding 85% of his message, we didn't buy any of his vitamin supplements, even though he offered to double the order for free.
Luckily the ride passed relatively quickly and we made it safe and sound to our destination.
Huancayo was a much larger city than Ayacucho, but it still had its charms. The main square was always full of people, and it was fun to watch the action unfold.
We are horribly ashamed to admit this, but on our first evening in Huancayo, we caved at the sight of Burger King. We think it was the fact that it was the first yellow cheese we've seen in three months, and honestly, it was a pretty decent burger and fries. Don't worry though, we ate plenty of other local food to make up for supporting the King.
Since we only had one full day in Huancayo, but we tried to make the most of it.
First, we hiked up to "Torre Torre," a strange-looking group of "towers" made after years of erosion.
We enjoyed hiking around them and seeing what crazy pictures we could take.
Later that afternoon, the highlight of the day was attending our first futbol game in South America! We were accompanied by our hostel owner's son, who was very excited about the game. We were able to chat in basic Spanish, and he even tested out some of his English with us. He was a great sport and answered all of our questions.
The crowd wasn't as big or rowdy as we were expecting. There were, however, quite a few houses around the stadium with huge groups on their roofs.
Not sure how safe that is.
The game was entertaining to watch, but in all honesty, it wasn't the most exciting game ever. Huancayo won 3-0, and the crowd was pretty tranquilo, so there wasn't a need for all the extra police after all.
The next morning we awoke bright and early at 5 am to catch the train to Lima. This train is definitely the longer way to get to Lima (12 hours vs. 8 hours on a bus), but is scenic and the second highest in the world.
We were lucky to be able to catch this train because it only runs about twice per month and we actually took the last train of the season (the train can't run during the wet season).
The train left the station on time, shocking for Peru, and we headed out of town. You could tell that the train does not run to often because all the locals came out of their houses to wave and watch the train roll by. We were lucky not to have anyone sitting across from us, so we were able to spread out and relax.
We quickly left the city and started climbing into the Andes. We followed the river which carved its way through a canyon while climbing the hills. The scenery and views continued to get better and better as we went.
We passed a few different mining operations along the way, which generally aren't so scenic, but we were surprised to see how they transport the coal.
As we climbed, the mountains became more beautiful and it got significantly colder. At one point it barely started to snow and the locals were sticking their heads out the window trying to catch a microscopic flake or two. Guess they don't see the white stuff too often.
When we reached the highest point at 15,681 feet, the trains stopped and we were able to get out for a bit and take some pictures.
We arrived in Lima a bit shaky after being jostled around for a little over 12 hours. This wasn't the smoothest train ride we'd ever taken, but it was one of the more interesting ones.
Off to eat our way through Lima's many delicious restaurants.
I can safely say that this has been Trevor's favorite entry to date…and the Galapagos set the bar pretty high. He loved the pictures of the trains, especially the one on the turn around. My guess is that you thought of him a few times while aboard the train! Love you guys…will send a full email later.


text:
D. Petre Bogdan
Think Agile? Think recycling!
Agile doesn't have a strict definition. People look at Agile and see different things. You can think of it as an idea, a philosophy, a mindset, or as an umbrella term for several practices. Sometimes it's a methodology, a process, or a buzzword.
Agile at its core is a manifesto. A declaration. Four values and twelve principles that put forth a set of ideas to help you think differently about software development, as an alternative to heavyweight methodologies, with rigid and restrictive practices and processes, often bureaucratic, documentation driven, and demotivating with unrealistic demands.
The values and principles are, in my opinion, the most neglected parts of Agile. With such few (16) conventions to follow, what that ends up meaning is, in fact, a neglect of the core of Agile.
Values and principles are what you find important. They are the support for how you will reason about things or how you will behave. To be Agile means to follow the Agile values and principles.
But many companies only adopt the practices from Agile, not also the values and principles. This way they think they can apply Agile on top of what they are already doing, without the need to truly change anything. They apply Agile like it's a coat of paint, without regard for how the surface should be prepared before painting. The paint might be new, shiny, and looking really nice, but will eventually dry out and peel off because it's not supported by the right surface. Likewise, Agile implementations will inevitably fail if they are not supported by the right values and principles.
Let me explain with a simple example: trash.
Photo by Hans Braxmeier on pixabay
What do you do with trash? You throw it out, right?
But trash isn't just something that should end up in the landfill. It contains useful materials that can be recycled, like plastic, glass, paper, etc. So now another question: do you recycle?
Why, or why not?
If you think it's important, you will recycle. If you don't care, you won't.
I personally think it's important, so I recycle. I separate plastic, glass, paper, and metal and place them in separate containers when I take out the trash. Used cooking oil can also be recycled, but there isn't a container for it, you have to take it to (certain, not all) gas stations that collect it. So I do that too.
In my town, it's easy to recycle if you live in a newer apartment complex because you have separate containers on-premises. But even if it's convenient, many don't recycle because they don't care. Garbage or trash is the same for them. As long as it's not in their house, it doesn't matter where it is. So they throw everything in the same pile.
When living in an older building you don't have separate containers on-site, but you can still recycle by using bins on the street, usually located at large road intersections. I did this for a lot of years, taking the trash out for a walk once per week. And when I reached the recycling bins, I would usually discover that they are were full :) So time for another walk to the next intersection, where maybe the containers were emptier. It wasn't at all convenient. And just to mention it, there weren't any fines to force me to do it, and I wasn't losing any money deposit if I didn't do so. It would have been way easier for me to give up and just throw everything in with the garbage every time, or recycle just some of the time when I didn't have to walk all over the city. But I didn't do that. I still recycled every time, because for me it was something important.
Getting back to what occurs inside companies, when you think about your Agile implementation, think recycling and ask yourself:
Are we adopting Agile or just convenient Agile practices?
What are the values and principles we are using to frame the things we do?
Are these values and principles shared by others in the organization?
What is important?
And while you're at it, ask yourself if you recycle enough.
Involved in software development for the past 15+ years, in various roles. Author of "Quicksands of Software Development": https://tinyurl.com/qosdqosd
More from D. Petre Bogdan
Background Job and Queue — Practical Application Use Cases
Tuan Nguyen in The Startup
MY DATA SCIENCE JOURNEY
Precious Diedemise (Preshel)
What is a Void Pointer?
Stella Sofia
Puppet Code by Example: Part 1
John Tucker in codeburst
UTCTF 2020 — BoF
Craig Knott
Flutter Dynamic Localization Tutorial
Free Time Dev
AWS ALB with WAF
Deadlock found when trying to get lock; try restarting transaction
karbachinsky

redpajama_set_name: RedPajamaCommonCrawl