Whether you're a devotee of the sequence or a first-time sampler, you will discover this most modern part of poultry Soup for the Soul either riveting and heartwarming. Jack Canfield and Mark Victor Hansen have woven one other tapestry of undying stories and encouraging knowledge. This treasury is a tribute to lifestyles and humanity, with issues ranging the total emotional and experiential gamut. the character of the tales invitations you to take pleasure in bird Soup in no matter what approach you discover so much comforting—by the spoonful, by way of the bowl, or the entire pot in a single sitting.
No longer happy with the hot T-shirts on sale on the neighborhood mall? probably you'd like a marriage invitation that expresses your personal imaginative and prescient, now not your get together planner's? How approximately a few customized stationery? An improve on your web site? A enterprise card? A poster on your political crusade? A CD package deal in your band?
It really is one of many enduring enigmas of the human adventure: a lot of our such a lot iconic, inventive endeavors—from Nobel Prize–winning discoveries to entrepreneurial innovations and works within the arts—are no longer achievements yet conversions, corrections after failed attempts.
The reward of failure is a riddle. just like the quantity 0, it's going to continually be either a void and the beginning of countless danger. The Rise—a soulful party of the decision and braveness of the human spirit—makes the case that a lot of our best triumphs come from knowing the significance of this mystery.
This beautiful biography of an concept is set the unbelievable foundations of artistic human exercise. the increase starts off with narratives approximately figures earlier and current who variety from writers to marketers; Frederick Douglass, Samuel F. B. Morse, and J. okay. Rowling, for instance, characteristic along choreographer Paul Taylor, Nobel Prize–winning physicists Andre Geim and Konstantin Novoselov, Arctic explorer Ben Saunders, and psychology professor Angela Duckworth.
The upward push explores the inestimable worth of usually missed ideas—the strength of hand over for fortitude, the criticality of play for innovation, the propulsion of the close to win at the highway to mastery, and the significance of grit and inventive perform. From an uncommonly insightful author, the increase is a real masterwork.
God has supplied wellbeing and fitness and therapeutic for the physique of Christ, however it has to be bought by means of religion. hint God's therapeutic circulate during the outdated and New Testaments. research the reason for ailment, therapeutic tools, and eventually comprehend the reality approximately Paul's thorn within the flesh. Marilyn Hickey's targeted presentation of God's notice pertaining to therapeutic will increase your point of religion during this all-important sector of the Christian existence.
A set of an important letters acquired from teenagers responding to the bird Soup for the Teenage Soul sequence, this ebook is a robust mirrored image of the emotions of iteration of readers. a few letters will make readers snigger and others will cause them to cry; yet all of the letters display that teenagers proportion related outlooks, studies, and emotions.
Soon that puppy disappeared too, and the old dog was alone again. I would have taken him home in a minute. Any homeless dog or cat could win my instant friendship—all it took was one hopeful look. But my husband had explained time and again that we simply could not take in stray animals. I knew he was right, but sometimes my heart forgot. With great determination I tried to steel myself against looking into the eyes of any stray, hungry dog or cat. Old Red never looked hungry, though, so I decided it was all right to form a relationship with him.
But there are compensations. You will come to know people ennobled by their suffering, and children whose sheer force of being will teach you lessons you never dreamed about. You'll learn that Death is ever out there, waiting for his shot, sometimes patiently, sometimes not. But you'll also learn that life is here and now, that every moment has meaning, and that even Death cannot take from you the things that others give. You'll rediscover the truth that the love of a child is the truest gift from God, and while the price may be high, the value is higher still.
On June fifteenth the apple of his eye will graduate with honors from California State University at Santa Barbara. He has hired a taxi to drive him there. The taxi driver (who knows him) will take his own private vehicle, so that John will not embarrass his daughter in any way. John will bathe, shave and put on an old suit to go and watch his little girl receive her degree. My heart is happy yet broken for him, as I've mentally thought of what he will be going through when he sees her walk up on stage to receive her degree.


UTHSC students must maintain health insurance coverage as a condition of enrollment. UTHSC's mandatory policy requires students to provide documentation of insurance coverage (waive) or to enroll (elect) in the Student Health Insurance plan each year. The University will email your UTHSC email address with information regarding your status in the enrollment/waiver process. Failure to waive or enroll in the student health insurance will result in you being enrolled in and charged for student health insurance. The waiver period closes two weeks after the start of classes for both the fall and spring terms.
If you have questions about your waiver status, please contact the One Stop at 901.448.7703.
For more information about the insurance plan including a summary of benefits and premium costs, please see the United Healthcare Student Resources brochure.
For questions about the insurance itself including copays, deductibles, covered expenses and more, please call Holland Insurance at 662.895.5528.
Please contact the One Stop at 901.448.7703 if you need any additional information.
If you need to pick up coverage during the school year . . .
Contact our agent, Holland Insurance at 662.895.5528, with the date you need to begin coverage. They will provide you with the prorated rate for coverage as well as instructions on how to enroll. It is best to contact Holland Insurance directly about a month before you need your new coverage to begin. If you are losing coverage because you are turning 26, please be aware that insurers have different policies on the end of coverage after a dependent turns 26. You should contact your current insurer for the information necessary to determine your start date for student health insurance.
UTHSC has contracted with Delta Dental insurance for student dental insurance. Coverage is optional for students and their dependents. For students electing to purchase dental insurance, the students' dental premiums will be assessed in their Banner accounts along with their tuition and fees. The premiums will be assessed bi-annually in the Fall and Spring semesters. Each assessment will equate to six months of dental premium coverage.
As with tuition and fees, the dental insurance premium will be deducted from the student's account prior to any refunds being issued. Dental insurance coverage is only offered for a full term and thus is not prorated.
For more information about the student dental insurance plan, please see the Delta Dental brochure and Summary of Benefits.
Enrollment for the Spring term has closed. The next period of coverage will begin July 1, 2019 with enrollment opening on May 1, 2019.
For more information about the dental insurance, please contact Delta Dental Customer Support at 1-800-223-3104.
We do not offer a vision insurance plan for students. The University does participate in a discount plan called the Vision Enhancement Plan. This plan is a partnership between the Hamilton Eye Institute, the Southern College of Optometry and other community partners in Memphis. For more information about this free discount plan open to all students, please see the Vision Enhancement Plan brochure.


text:
Exact Solutions of a Deep Linear Network
by Liu Ziyin, et al.
This work finds the exact solutions to a deep linear network with weight decay and stochastic neurons, a fundamental model for understanding the landscape of neural networks. Our result implies that weight decay strongly interacts with the model architecture and can create bad minima in a network with more than 1 hidden layer, qualitatively different for a network with only 1 hidden layer. As an application, we also analyze stochastic nets and show that their prediction variance vanishes to zero as the stochasticity, the width, or the depth tends to infinity.
Liu Ziyin
Botao Li
Xiangming Meng
Understanding Global Loss Landscape of One-hidden-layer ReLU Networks, Part 2: Experiments and Analysis
The existence of local minima for one-hidden-layer ReLU networks has bee...
Bo Liu, et al. ∙
Stochastic Neural Networks with Infinite Width are Deterministic
This work theoretically studies stochastic neural networks, a main type ...
Liu Ziyin, et al. ∙
The Global Landscape of Neural Networks: An Overview
One of the major concerns for neural network training is that the non-co...
Ruoyu Sun, et al. ∙
Bayesian neural networks increasingly sparsify their units with depth
We investigate deep Bayesian neural networks with Gaussian priors on the...
Mariia Vladimirova, et al. ∙
A Better Way to Decay: Proximal Gradient Training Algorithms for Neural Nets
Weight decay is one of the most widely used forms of regularization in d...
Liu Yang, et al. ∙
How and what to learn:The modes of machine learning
We proposal a new approach, namely the weight pathway analysis (WPA), to...
Sihan Feng, et al. ∙
Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality
In this note, we study how neural networks with a single hidden layer an...
Stephan Wojtowytsch, et al. ∙
Applications of neural networks have achieved great success in various fields. One central open theoretical question is why neural networks, being nonlinear and containing many saddle points and local minima, can sometimes be optimized easily to a global minimum (Choromanska et al., 2015a) , while becomes almost impossible to train in some other scenarios (Glorot and Bengio, 2010) and may require many tricks to succeed (Gotmare et al., 2018) . One established approach is to study the landscape of deep linear nets (Choromanska et al., 2015b) , which are believed to approximate the landscape of a nonlinear net well. A series of works proved the famous results that for a deep linear net, all local minima are global (Kawaguchi, 2016; Lu and Kawaguchi, 2017; Laurent and Brecht, 2018) , which is regarded to have successfully explained why deep neural networks are so easy to train because it implies that initialization in any attractive basin can reach the global minimum without much effort (Kawaguchi, 2016) .
In this work, we theoretically study a deep linear net with weight decay and stochastic neurons, whose loss function takes the following form in general:
ExEϵ(1),ϵ(2),...,ϵ(D)⎛⎝d,d1,d2,...dD∑i,i1,i2,...,iDUiDϵ(D)iD...ϵ(2)i2W(2)i2i1ϵ(1)i1W(1)i1ixi−y⎞⎠2L0+γu||U||22+D∑i=1γi||W(i)||2FL2 reg., (1)
where U and W(i) are the model parameters, D is the depth of the network,111In this work, we use "depth" to refer to the number of hidden layers. For example, a linear regressor has depth 0. ϵ is the noise in the hidden layer (e.g., due to dropout), di is the width of the i-th layer, and γ is the strength of the weight decay. Previous works have studied special cases of this loss function. For example, Kawaguchi (2016) and Lu and Kawaguchi (2017) study the landscape of L0 when ϵ is a constant (namely, when there is no noise). Mehta et al. (2021) studies L0 with (a more complicated type of) weight decay but without stochasticity and proved that all the stationary points are isolated. Another line of works studies L0 when the noise is caused by dropout (Mianjy and Arora, 2019; Cavazza et al., 2018) . Our setting is more general than the previous works in two respects. First, apart from the mean square error (MSE) loss L0, an L2 regularization term (weight decay) with arbitrary strength is included; second, the noise ϵ
is arbitrary. Thus, our setting is arguably closer to the actual deep learning practice, where the injection of noises to latent layers is common and the use of weight decay is virtually ubiquitous
(Krogh and Hertz, 1992; Loshchilov and Hutter, 2017) . One major theoretical limitation of our work is that we assume the label y to be 1-dimensional, and it can be an important future problem to prove whether an exact solution exists or not when y is high-dimensional.
Our foremost contribution is to give the exact solution for all the global minimum of an arbitrarily deep and wide linear net on the MSE loss L0 plus a weight decay term and with a general type of stochasticity in the hidden layer. In other words, we identify in closed form the global minimum of Eq. (1). We then show that it has nontrivial properties that can explain many phenomena in deep learning. In particular, the implications of our result include (but are not limited to):
Weight decay makes the landscape of neural nets more complicated;
we show that bad minima222Unless otherwise specified, we use the word "bad minimum" to mean a local minimum that is not a global minimum. Namely, one would have to overcome some nontrivial barrier to reach the global minimum. This usage is consistent with the previous literature (Kawaguchi, 2016) . emerge as weight decay is applied, whereas there is no bad minimum when there is no weight decay. This highlights the need to escape bad local minima in deep learning with weight decay.
Deeper nets are harder to optimize than shallower ones;
we show that a 3-layer linear net contains a bad minimum at zero, whereas a 2-layer net does not. This partially explains why deep networks are much harder to optimize than shallower ones in deep learning practice.
Stochastic networks in a few asymptotic limits can become deterministic;
we show that the prediction variance of a stochastic net scales towards 0 on the MSE loss as (a) the width tends to infinity, or (b) the variance of the latent randomness tends to infinity, or (c) the depth tends to infinity.
In summary, our result deepens our understanding of the loss landscape of neural networks and stochastic networks. Organization: In the next section, we discuss the related works. In Section 3, we derive the exact solution for a two-layer net. Section 4 extends the result to an arbitrary depth. In Section 5, we apply our result to study stochastic networks. The last section concludes the work and discusses unresolved open problems. Moreover, the technical, lengthy or less essential proofs are delayed to Section A.
Notation. For a matrix W, we use Wi: to denote the i
-th row vector of
W. ||Z|| denotes the L2 norm if Z is a vector and the Frobenius norm if Z is a matrix. The notation ∗ signals an optimized quantity. Additionally, we the superscript ∗ and subscript ∗ interchangeably, whichever leads to a simpler expression. For example, b2∗ and (b∗)2 denotes the same quantity, while the former is "simpler."
2 Related Works
Linear Nets. In many ways, linear networks have been used to help understand nonlinear networks. For example, even at depth 0, where the linear net is nothing but a linear regressor, linear nets are shown to be relevant for understanding the generalization behavior of modern overparametrized networks (Hastie et al., 2019) . Saxe et al. (2013) studies the training dynamics of a depth-1
network and uses it to understand the dynamics of learning of nonlinear networks. These networks are the same as a linear regression model in terms of expressivity. However, the loss landscape is highly complicated due to the existence of more than one layer, and linear nets are widely believed to approximate the loss landscape of a nonlinear net
(Kawaguchi, 2016; Hardt and Ma, 2016; Laurent and Brecht, 2018) . In particular, the landscape of linear nets has been studied as early as 1989 in Baldi and Hornik (1989) , which proposed the well-known conjecture that all local minima of a deep linear net are global. This conjecture is first proved in Kawaguchi (2016) , and extended to other loss functions and deeper depths in Lu and Kawaguchi (2017) and Laurent and Brecht (2018) .
Stochastic Net Theory. A major extension of the standard neural networks is to make them stochastic, namely, to make the output a random function of the input. In a broad sense, stochastic neural networks include neural networks trained with dropout (Srivastava et al., 2014; Gal and Ghahramani, 2016)
, Bayesian networks
(Mackay, 1992)
, variational autoencoders (VAE)
(Kingma and Welling, 2013)
, and generative adversarial networks
(Goodfellow et al., 2014) . Stochastic networks are thus of both practical and theoretical importance to study. Theoretically, while a unified approach is lacking, some previous works exist to separately study different stochastic techniques in deep learning. A series of recent works approach the VAE loss theoretically (Dai and Wipf, 2019) . Another line of recent works analyzes linear models trained with VAE to study the commonly encountered mode collapse problem of VAE (Lucas et al., 2019; Koehler et al., 2021) . Another series of work that extensively studied the dropout technique with a linear network (Cavazza et al., 2018; Mianjy and Arora, 2019; Arora et al., 2020) showed that dropout effectively controls the rank of the learned solution. Lastly, it is worth noting that in the original works of Kawaguchi (2016) and Choromanska et al. (2015a)
, a linear net with a special type of noise that is equivalent to dropout has been used to model the effect of the ReLU nonlinearity in actual neural networks. Our work considers an arbitrary type of noise whose second moment exists, not just limited to dropout.
3 Two-layer Linear Net
This section finds the exact solutions of a two-layer linear net. The data point is a d-dimensional vector x∈Rd drawn from a data distribution p(x) and the labels are generated through an arbitrary function y=y(x)∈R. Common in the deep learning practice, a weight decay term is also added. For generality, the two different layers have different strengths of weight decay even though they often take the same value in practice.
In particular, we are interested in finding the global minimum of the following objective:
Ld,d1(U,W)=ExEϵ(d1∑jUjϵjd∑iWjixi−y)2+γw||W||2+γu||U||2, (2)
where d1 is the width of the hidden layer and ϵi
are independent random variables that are often used to stochastify a neural network, dropout
(Srivastava et al., 2014) being one well-known example. γw>0 and γu>0 are the weight decay parameters. Here, we consider a general type of independent noise with E[ϵi]=1 and E[ϵiϵj]=δijσ2+1 where δij is the Kronecker's delta, and σ2>0. For shorthand, we use the notation A0:=E[xxT]
, and the largest and the smallest eigenvalues of
A0 are denoted as amax and amin respectively. ai denotes the i-th eigenvalue of A0 viewed in any order. For now, it is sufficient for us to assume that the global minimum of Eq. (2) always exists. We will prove a more general result in Proposition 1, when we deal with multilayer nets.
3.1 Main Result
We first present two lemmas showing that the global minimum can only lie on a rather restrictive subspace of all possible parameter settings due to invariances (symmetries) in the objective.
At the global minimum of Eq. (2), U2j=γwγu∑iW2ji for all j.
Proof Sketch. We use the fact that the first term of Eq. (2) is invariant to a simultaneous rescaling of rows of the weight matrix to find the optimal rescaling, which implies the lemma statement. □
This lemma implies that for all j, |Uj| must be proportional to the norm of its corresponding row vector in W. The following lemma further shows that, at the global minimum, all elements of U must be equal.
At the global minimum, for all i and j, we have
{U2i=U2j;UiWi:=UjWj:. (3)
Proof Sketch. We show that if the condition is not satisfied, then, an "averaging" transformation will strictly decrease the objective. □
The second lemma imposes strong conditions on the solution of the problem, and the essence of this lemma is the reduction of the original problem to a lower dimension.
We are now ready to prove our first main result.
The global minimum U∗ and W∗ of Eq. (2) is U∗=0 and W∗=0 if and only if
||E[xy]||2≤γuγw. (4)
When ||E[xy]||2>γuγw, there exists b>0 such that the global minima are
{U∗=br;W∗=rE[xy]Tb[b2(σ2+d1)A0+γwI]−1, (5)
where r=(±1,...,±1) is an arbitrary vertex of a d1-dimensional hypercube.
Proof. By Lemma 2, at any global minimum, we can write U∗=br for some b∈R. We can also write W∗=rvT for a general vector v∈Rd. Without loss of generality, we assume that b>0 (because the sign of b can be absorbed into r).
The original problem in Eq. (2) is now equivalently reduced following problem because rTr=d1:
minb,vEx⎡⎣(bd1∑jvjxj−y)2+b2d1σ2(∑kvkxk)2⎤⎦+γud1b2+γwd1||v||22. (6)
For any fixed b, the global minimum of v is well known:333Namely, it is the solution of a ridgeless linear regression problem.
v=bE[xy]T[b2(σ2+d1)A0+γwI]−1. (7)
By Lemma 1, at a global minimum, b also satisfies the following condition:
b2=γwγu||v||2, (8)
One solution to this equation is b=0, and we are interested in whether solutions with b≠0 exists. If there is no other solution, then b=0 must be the unique global minimum; otherwise, we need to identify which of the solutions are actual global minima and which are just saddles.444We do not discern between saddles or maxima. When b≠0,
∣∣∣∣∣∣[b2(σ2+d1)A0+γwI]−1E[xy]∣∣∣∣∣∣2=γuγw. (9)
Note that the left-hand side is monotonically decreasing in b2, and is equal to γ−2w||E[xy]||2 when b=0. When b→∞, the left-hand side tends to 0. Because the left-hand side is a continuous and monotonic function of b, a unique solution b∗>0 that satisfies Eq. (9) exists if and only if γ−2w||E[xy]||2>γu/γw, or,
||E[xy]||2>γuγw. (10)
Therefore, at most three candidates for global minima of the loss function exist:
{b=0, v=0if ||E[xy]||2≤γuγw;b=±b∗, v=b[b2(σ2+d1)A0+γwI]−1E[xy],if ||E[xy]||2>γuγw, (11)
where b∗>0.
In the second case, one needs to discern the saddle points from the global minima. Using the expression of v, one finds the expression of the loss function as a function of b
d1(d1+σ2)b4∑iE[x′y]2iai[b2(σ2+d1)ai+γw]2−2b2d1∑iE[x′y]2ib2(σ2+d1)ai+γw+E[y2]+γud1b2+γwd1∑iE[x′y]2ib2[b2(σ2+d1)ai+γw]2, (12)
where x′=Rx such that RA0R−1 is a diagonal matrix. We now show that condition (10) is sufficient to guarantee that 0 is not the global minimum.
At b=0, the first nonvanishing derivative of b is the second-order derivative. The second order derivative at b=0 is
−2d1||E[xy]||2/γw+2γud1, (13)
which is negative if and only if ||E[xy]||2>γuγw. If the second derivative at b=0 is negative, b=0 cannot be a minimum. It then follows that for ||E[xy]||2>γuγw, b=±b∗, v=b[b2(σ2+d1)A0+γwI]−1E[xy],if ||E[xy]||2>γuγw are the two global minimum (because the loss is invariant to the sign flip of b). For the same reason, when ||E[xy]||2<γuγw, b=0 gives the unique global minimum. This finishes the proof. □
Apparently, b=0 is the trivial solution that has not learned any feature due to overregularization. Henceforth, we refer to this solution (and similar solutions for deeper nets) as the "trivial" solution. We now analyze the properties of the nontrivial solution b∗ when it exists.
The condition for the solution to become nontrivial is interesting: ||E[xy]||2≥γuγw. The term ||E[xy]|| can be seen as the effective strength of the signal, and γuγw is the strength of regularization. This precise condition means that the learning of a two-layer can be divided into two qualitatively different regimes: an "overregularized regime," where the global minimum is trivial, and a "feature learning regime", where the global minimum involves actual learning.
3.2 Exact Form of b∗
Note that our main result does not specify the exact value of b∗. This is because b∗ must satisfy the condition in Eq. (9), which is equivalent to a high-order polynomial in b with coefficients being general functions of the eigenvalues of A0, whose solutions are generally not analytical by Galois theory.
However, when A0=σ2xI, the exact form exists. Practically, this can be achieved for any (full-rank) dataset if we disentangle and rescale the data by the whitening transformation: x→σx√A−10x. In this case, we have
b2∗=√γwγu||E[xy]||−γw(σ2+d1)σ2x, (14)
v=±  ⎷√γuγw||E[xy]||−γuσ2x(σ2+d1)E[xy]||E[xy]||. (15)
3.3 Bounding the General Solution
While the solution to b∗ does not admit an analytical form for a general A0, one can find meaningful lower and upper bound to b∗
such that we can perform asymptotic analysis of
b∗. At the global minimum, the following inequality holds:
||[b2(σ2+d1)amaxI+γwI]−1E[xy]||2≤||[
≤||[b2(σ2+d1)aminI+γwI]−1E[xy]||2, (16)
where amin and amax are the smallest and largest eigenvalue of A0, respectively. The middle term is equal to γu/γw by the global minimum condition in (9), and so, assuming amin>0, this inequality is equivalent to the following two inequalities of b∗:
√γwγu||E[xy]||−γw(σ2+d1)amax≤b2∗≤√γwγu||E[xy]||−γw(σ2+d1)amin. (17)
Namely, the general solution b∗ should scale similarly to the homogeneous solution555We use the word "homogeneous" to mean that all the eigenvalues of A0 take the same value. in Eq. (14) if we treat the eigenvalues of A0 as constants.
4 Exact Solution for An Arbitrary-Depth Linear Net
This section extends our result to multiple layers, each with independent stochasticity. We first prove the analytical formula for the global minimum of a general arbitrary-depth model. We then show that the landscape for a deeper network is highly nontrivial.
4.1 General Solution
For this problem, the loss function becomes
ExEϵ(1),ϵ(2),...,ϵ(D)⎛⎝d,d1,d2,...dD∑i,i1,i2,...,iDUiDϵ(D)iD...ϵ(2)i2W(2)i2i1ϵ(1)i1W(1)i1ixi−y⎞⎠2+γu||U||2+D∑i=1γi||W(i)||2, (18)
where all the noises ϵ are independent from each other, and for all i and j, E[ϵ(i)j]=1 and E[(ϵ(i)j)2]=σ2i+1>1. We first show that for general D, the global minimum exists for this objective.
Proposition 1.
For D≥1 and strictly positive γu, γ1,...,γD, the global minimum for Eq.(18) exists.
Note that the positivity of the regularization strength is crucial. If one of the γi is zero, the global minimum may not exist. The following theorem is our second main result.
Any global minimum of Eq. (18) is of the form
⎧⎪ ⎪ ⎪⎨⎪ ⎪ ⎪⎩U=burD;W(i)=birirTi−1;W(1)=r1E[xy]T(bu∏Di=2bi)μ[(bu∏Di=2bi)2s2(σ2+d1)A0+γwI]−1, (19)
where μ=∏Di=2di, s2=∏Di=2di(σ2+di), bu≥0 and bi≥0, and ri=(±1,...,±1) is an arbitrary vertex of a di-dimensional hypercube for all i.
Proof Sketch. We prove by induction on the depth D. The base case is proved in Theorem 1. We then show that for a general depth, the objective involves optimizing subproblems, one of which is a D−1 layer problem that follows by the induction assumption, and the other is a two-layer problem that has been solved in Theorem 1. Putting these two subproblems together, one obtains Eq. (19). □
Similar to a two-layer network, the scaling factor bi for all i is not independent from one another. The following Lemma is a generalization of Lemma 1
to a multilayer setting and shows that there is only one degree of freedom (instead of
D) in the form of solutions in Eq. (19).
At any global minimum of Eq. (18), let b1:=√||Wi:||2/d and bD+1:=bu,
γk+1dk+1b2k+1=γkdk−1b2k. (20)
Proof Sketch. The proof is similar to Lemma 1. □
The lemma implies the product of all the bi can be written in terms of one of the bi:
buD∏i=2bi=c0sgn(buD∏i=2bi)|bD2|:=c0sgn(buD∏i=2bi)bD (21)
where c0=(γ2d2d1)D/2√γu∏Di=2γi∏Di=2di√d1 and b≥0. Applying Lemma 3 to the first layer of Theorem 2 shows that the global minimum must satisfy the following equation, which is equivalent to a high-order polynomial in b that does not have an analytical solution in general:
||E[xy]Tc0bDμ[c20b2Ds2(σ2+d1)A0+γwI]−1||2=d2b2. (22)
At this point, it pays to clearly define the word "solution," especially given that it has a special meaning in this work because it now becomes highly nontrivial to differentiate between the two types of solutions.
Definition 1.
We say that a non-negative real b is a solution if it satisfies Eq. (22). A solution is trivial if b=0 and nontrivial otherwise.
Namely, a global minimum must be a solution, but a solution is not necessarily a global minimum. We have seen that even in the two-layer case, the global minimum can be the trivial one when the strength of the signal is too weak or when the strength of regularization is too strong. It is thus natural to expect 0 to be the global minimum under a similar condition, and one is interested in whether the condition becomes stronger or weaker as the depth of the model is increased. However, it turns out this naive expectation is not true. In fact, when the depth of the model is larger than 2, the condition for the trivial global minimum becomes highly nontrivial.
The following proposition shows why the problem becomes more complicated. In particular, we have seen that in the case of a two-layer net, some elementary argument has helped us show that the trivial solution b=0 is either a saddle or the global minimum. However, the proposition below shows that with D≥2, the landscape becomes more complicated in the sense that the trivial solution is always a local minimum, and it becomes difficult to compare the loss value of the trivial solution with the nontrivial solution because the value of b∗ is unknown in general.
Let D≥2 in Eq. (18). Then, the solution U=0, W(D)=0, …, W(1)=0 is a local minimum with a diagonal positive-definite Hessian.
Proof Sketch. This is a technical proof that directly computes the Hessian. □
Comparing the Hessian of D≥2 and D=1, one notices a qualitative difference: for D≥2, the Hessian is always diagonal (at 0); for D=1, in sharp contrast, the off-diagonal terms are nonzero in general, and it is these off-diagonal terms that can break the positive-definiteness of the Hessian. This offers a different perspective about why there is a qualitative difference between D=1 and D=2.
Lastly, note that, unlike the depth-1 case, one can no longer find a precise condition such that a b≠0 solution exists for a general A0. The reason is that the condition for the existence of the solution is now a high-order polynomial with quite arbitrary intermediate terms. The following proposition gives a sufficient but stronger-than-necessary condition for the existence of a nontrivial solution, when all the σi, intermediate width di and regularization strength γi are the same.666This is equivalent to setting c0=√d0. The result is qualitatively similar but involves additional factors of c0 if σi, di, and γi all take different values. We thus only present the case when σi, di, and γi are the same for notational concision and for emphasizing the most relevant terms. Also, note that this proposition gives a sufficient and necessary condition if A0=σ2xI is proportional to the identity.
Let σ2i=σ2>0, di=d0 and γi=γ>0 for all i. Assuming amin>0, the only solution is trivial if
D+12D||E[xy]||dD−10((D−1)||E[xy]||2Dd0(σ2+d0)Damin)D−1D+1<γ. (23)
Nontrivial solutions exist if
D+12D||E[xy]||dD−10((D−1)||E[xy]||2Dd0(σ2+d0)Damax)D−1D+1≥γ. (24)
Moreover, the nontrivial solutions are both lower and upper-bounded:777For D=1, we define the lower-bound as limη→0+limD→1+1d0[γ+η||E[xy]||]1D−1, which equal to zero if E[xy]≥γ, and ∞ if E[xy]<γ. With this definition, this proposition applies to a two-layer net as well.
1d0[γ||E[xy]||]1D−1≤b∗≤[||E[xy]||d0(σ2+d0)Damax]1D+1. (25)
Proof Sketch. The proof follows from the observation that the l.h.s. of Eq. (22) is a continuous function and must cross the r.h.s. under certain sufficient conditions. □
One should compare the general condition here with the special condition for D=1. One sees that for D≥2, many the other factors (such as the width, the depth and the spectrum of the data covariance A0) come into play to determine the existence of a solution apart from the signal strength E[xy] and the regularization strength γ.
4.2 Which Solution is the Global Minimum?
Again, we set γi=γ>0, σ2i=σ2>0 and di=d0>0 for all i for notational concision. Using this condition and applying Lemma 3 to Theorem 2, the solution now takes the following form, where b≥0,
⎧⎪ ⎪⎨⎪ ⎪⎩U=√d0brD;W(i)=brirTi−1;W(1)=r1E[xy]TdD−120bD[dD0(σ2+d0)Db2DA0+γ]−1. (26)
Note that the signs of bi and bu are arbitrary due to the invariances of the original objective. The following theorem gives a sufficient condition for the global minimum to be nontrivial. It also shows that the landscape of the linear net becomes complicated and can contain more than 1 local minimum when a certain condition is satisfied.
Let σ2i=σ2>0, di=d0 and γi=γ>0 for all i and assuming amin>0. Then, there exists a constant a0>0 such that for any
0<γ<[1Dd20C1/D14dD0(σ2+d)D||E[xy]||2A−10]DD+1, (27)
the global minimum of Eq. (18) is one of the nontrivial solutions.
Proof Sketch. We find an easy-to-solve upper bound the objective, which is simplified to the above condition. □
While there are various ways this bound can be proved, it is general enough for our purpose. In particular, one sees that, for a general depth, the condition for having a nontrivial global minimum depends not only on the E[xy] and γ but also on the model architecture in general. For a more general architecture with different widths etc., the architectural constant c0 from Eq. (22) will also enter the equation.
Figure 1: An incomplete summary of the network landscape implied by the main results of this work when one increases the weight decay strength γ while fixing other terms. We show that the landscape of a depth-1 net can be precisely divided into two regimes, while, for D≥2, there exists at least three regimes. The solid blue line indicates that the division of the regimes is precisely understood. The dashed lines indicate that the conditions we found are not tight and may be improved in the future.
The combination of Theorem 3 and Proposition 2 shows that the landscape of a deep neural network becomes highly nontrivial when there is a weight decay and when the depth of the model is larger than 2. This gives an incomplete but meaningful picture of a network's complicated but interesting landscape beyond two layers (see Figure 1 for a incomplete summary of our results). In particular, even when the nontrivial solution is the global minimum, the trivial solution is still a local minimum that needs to be escaped. Our result suggests the previous understanding that all local minima of a deep linear net are global cannot generalize to many practical settings where deep learning is found to work well. For example, a series of works attribute the existence of bad (non-global) minima to the use of nonlinearities nonlinear nets (Kawaguchi, 2016) or the use of a non-regular (non-differentiable) loss function (Laurent and Brecht, 2018) . Our result, in contrast, shows that the use of a simple weight decay is sufficient to create a bad minimum.888Some previous works do suggest the existence of bad minima when weight decay is present, but no direct proof exists yet. For example, Taghvaei et al. (2017) shows that when the model is approximated by a linear dynamical system, regularization can cause bad local minima. Mehta et al. (2021) shows the existence of bad local minima in deep linear networks with weight decay through numerical simulations. Moreover, the problem with such a minimum is two-fold: (1) (optimization) it is not global and so needs to be "overcome"999This becomes more a problem because we have shown that this bad minimum is located at the origin where the neural networks are commonly initialized at. and (2) (generalization) it is a minimum that has not learned any feature at all because the model constantly outputs zero. To the best of our knowledge, previous to our work, there has not been any proof that a bad minimum can generically exist in a rather arbitrary network without any restriction on the data.101010In the case of non-linear networks without regularization, a few works proved the existence of bad minima. However, the previous results strongly depend on the data and rather independent of architecture. For example, one major assumption is that the data cannot be perfected fitted by a linear model (Yun et al., 2018; Liu, 2021; He et al., 2020) . Some other works explicitly construct data distribution (Safran and Shamir, 2018; Venturi et al., 2019) . Our result, in contrast, is independent of the data. Thus, our result offers direct and solid theoretical justification for the widely believed importance of escaping local minima in the field of deep learning (Kleinberg et al., 2018; Liu et al., 2021; Mori et al., 2022) . In particular, previous works on escaping local minima often hypothesize landscapes that are of unknown relevance to an actual neural network. With our result, this line of research can now be established with respect to landscapes that are actually deep-learning-relevant.
Previous works also argue that having a deeper depth does not create a bad minimum (Lu and Kawaguchi, 2017) . While this remains true, its generality and applicability to practical settings now also seem low. Our result shows that as long as weight decay is used, and as long as D≥2, there is indeed a bad local minimum at 0. In contrast, there is no bad minimum at 0 for a depth-2 network: the point b=0 is either a saddle or the global minimum.111111Of course, in practice, the model trained with SGD can still converge to the trivial solution even if it is a saddle point (Ziyin et al., 2021)
because SGD with a finite learning rate is in general not a good estimator of the local minima.
Having a deeper depth thus alters the qualitative nature of the landscape and our results agree better with the common observation that a deeper network is harder, if not impossible, to optimize.
4.3 Asymptotic Analysis
Now we analyze the solution when D tends to infinity. We first note that the existence condition bound in (24) becomes exponentially harder to satisfy as D becomes large:
||E[xy]||2≥4d20amaxγeDlog[(σ2+d0)/d0]+O(1). (28)
Recall that for a two-layer net, the existence condition is nothing but ||E[xy]||2>γ2, independent of the depth, width, or the stochasticity in the model. For a deeper network, however, every factor comes into play, and the architecture of the model has a strong (and dominant) influence on the condition. In particular, a factor that increases polynomial in the model width and exponentially in the model depth appears.
There are many ways to interpret this condition. For example, it can be seen as an upper bound for the model depth. Alternatively, it is also an upper bound for γ and an lower bound for ||E[xy]||. This condition means that learning becomes difficult and even impossible if we increase the depth of the model while fixing the weight decay or the dataset, also agreeing with the common observation that deep networks are very hard to train.
5 An Application to Stochastic Nets
As discussed, the property of stochastic nets is an important topic to study in deep learning. In this section, we present a simple application of our general solution to analyze the properties of a stochastic net. The following theorem summarizes our technical results.
Let σ2i=σ2>0, di=d0 and γi=γ>0 for all i. Let A0=σ2xI. Then, at any global minimum of Eq. (18), in the limit of large d0,
Var[f(x)]=O(d−10). (29)
In the limit of large σ2,
Var[f(x)]=O(1(σ2)D). (30)
In the limit of large D,
Var[f(x)]=O(e−2Dlog[(σ2+d0)/d0]). (31)
Proof Sketch. The result follows by applying the bound in Proposition 3 to bound the model prediction. □
Interestingly, the scaling of prediction variance in asymptotic σ2 is different for different widths. The third result shows that the prediction variance decreases exponentially fast in D. In particular, this result answers a question recently proposed in Ziyin et al. (2022) : does a stochastic net trained on MSE have prediction variance that scales towards 0? Ziyin et al. (2022) shows that for a generic nonlinear model, the prediction variance at the global minimum scales is O(d−1). Ziyin et al. (2022) also hypothesizes that the prediction variance scales towards 0 as the strength of latent variance σ2 increases to infinity. We improve on their result in the case of a deep linear net by (a) showing that the d−10 is tight in general, independent of the depth or other factors of the model, (b) proving a power-law bound for asymptotic σ2 that has been conjectured, and (c) proving a novel bound showing that the variance also scales towards zero as depth increases, which is a novel result of our work.
In this work, we derived the exact solution of a deep linear net with arbitrary depth and with stochasticity. The global minimum is shown to take exact forms with a hard-to-determine constant. As argued, we expect our work to shed more light on the highly nonlinear landscape of a deep neural network. Compared to the previous works that mostly focus on the qualitative understanding of the linear net, our result offers more precise quantitative understanding of deep linear nets. The quantitative understanding is one major benefit of knowing the exact solution, whose usefulness we have also demonstrated with the application to stochastic nets. In particular, our result directly implies that a line of previous theoretical results is insufficient to explain why neural networks can often be efficiently optimized by the gradient descent method, especially in realistic settings when a weight decay term is present. One application of our main results also provides insight regarding how stochastic networks works, an understudied but important topic.
Restricting to the specific problem setting we studied, there are also many interesting unresolved problems. The following is an incomplete list:
In general, more than one nontrivial solutions of Eq. (22) can exist. When they do, is every solution a local minimum? If not, is every nontrivial solution a stationary point?
Is there any local minimum that is not a solution?
We have seen that a depth-1 network is qualitatively different from a depth-2 network. Are there any qualitative changes happening for even deeper networks?
Additionally, is an infinite-depth network qualitatively different from a finite-depth network?
Answering these questions can significantly deepen our understanding of the landscape of a neural network.
Our mathematical setting is also limited in two ways. First of all, it is unclear whether the derived results are unique to linear systems or also relevant for nonlinear networks. Secondly, we only gave the exact formula for the global minimum when the output neuron has dimension 1, and it is not clear whether an analytical solution exists for higher output dimensions. To generalize beyond these two limitations is also an important future step.
Arora et al. (2020) Arora, R., Bartlett, P., Mianjy, P., and Srebro, N. (2020). Dropout: Explicit forms and capacity control.
Baldi and Hornik (1989) Baldi, P. and Hornik, K. (1989).
Neural networks and principal component analysis: Learning from examples without local minima.
Neural networks, 2(1):53–58.
Cavazza et al. (2018) Cavazza, J., Morerio, P., Haeffele, B., Lane, C., Murino, V., and Vidal, R. (2018). Dropout as a low-rank regularizer for matrix factorization. In
International Conference on Artificial Intelligence and Statistics
, pages 435–444. PMLR.
Choromanska et al. (2015a) Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B., and LeCun, Y. (2015a). The loss surfaces of multilayer networks. In Artificial Intelligence and Statistics, pages 192–204.
Choromanska et al. (2015b) Choromanska, A., LeCun, Y., and Arous, G. B. (2015b). Open problem: The landscape of the loss surfaces of multilayer networks. In Conference on Learning Theory, pages 1756–1760. PMLR.
Dai and Wipf (2019) Dai, B. and Wipf, D. (2019). Diagnosing and enhancing vae models. arXiv preprint arXiv:1903.05789.
Gal and Ghahramani (2016) Gal, Y. and Ghahramani, Z. (2016). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In
international conference on machine learning
, pages 1050–1059. PMLR.
Glorot and Bengio (2010) Glorot, X. and Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings.
Goodfellow et al. (2014) Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.
Gotmare et al. (2018) Gotmare, A., Keskar, N. S., Xiong, C., and Socher, R. (2018). A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation. arXiv preprint arXiv:1810.13243.
Hardt and Ma (2016) Hardt, M. and Ma, T. (2016). Identity matters in deep learning. arXiv preprint arXiv:1611.04231.
Hastie et al. (2019) Hastie, T., Montanari, A., Rosset, S., and Tibshirani, R. J. (2019). Surprises in high-dimensional ridgeless least squares interpolation. arXiv preprint arXiv:1903.08560.
He et al. (2020) He, F., Wang, B., and Tao, D. (2020). Piecewise linear activations substantially shape the loss surfaces of neural networks. arXiv preprint arXiv:2003.12236.
Kawaguchi (2016) Kawaguchi, K. (2016). Deep learning without poor local minima. Advances in Neural Information Processing Systems, 29:586–594.
Kingma and Welling (2013) Kingma, D. P. and Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.
Kleinberg et al. (2018) Kleinberg, B., Li, Y., and Yuan, Y. (2018). An alternative view: When does sgd escape local minima? In International Conference on Machine Learning, pages 2698–2707. PMLR.
Koehler et al. (2021) Koehler, F., Mehta, V., Risteski, A., and Zhou, C. (2021). Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias. arXiv preprint arXiv:2112.06868.
Krogh and Hertz (1992) Krogh, A. and Hertz, J. A. (1992). A simple weight decay can improve generalization. In Advances in neural information processing systems, pages 950–957.
Laurent and Brecht (2018) Laurent, T. and Brecht, J. (2018). Deep linear networks with arbitrary loss: All local minima are global. In International conference on machine learning, pages 2902–2907. PMLR.
Liu (2021) Liu, B. (2021). Spurious local minima are common for deep neural networks with piecewise linear activations. arXiv preprint arXiv:2102.13233.
Liu et al. (2021) Liu, K., Ziyin, L., and Ueda, M. (2021).
Noise and fluctuation of finite learning rate stochastic gradient descent.
Loshchilov and Hutter (2017) Loshchilov, I. and Hutter, F. (2017). Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.
Lu and Kawaguchi (2017) Lu, H. and Kawaguchi, K. (2017). Depth creates no bad local minima. arXiv preprint arXiv:1702.08580.
Lucas et al. (2019) Lucas, J., Tucker, G., Grosse, R., and Norouzi, M. (2019). Don't blame the elbo! a linear vae perspective on posterior collapse.
Mackay (1992) Mackay, D. J. C. (1992). Bayesian methods for adaptive models. PhD thesis, California Institute of Technology.
Mehta et al. (2021) Mehta, D., Chen, T., Tang, T., and Hauenstein, J. (2021). The loss surface of deep linear networks viewed through the algebraic geometry lens. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Mianjy and Arora (2019) Mianjy, P. and Arora, R. (2019). On dropout and nuclear norm regularization. In International Conference on Machine Learning, pages 4575–4584. PMLR.
Mori et al. (2022) Mori, T., Ziyin, L., Liu, K., and Ueda, M. (2022). Power-law escape rate of sgd.
Safran and Shamir (2018) Safran, I. and Shamir, O. (2018). Spurious local minima are common in two-layer relu neural networks. In International conference on machine learning, pages 4433–4441. PMLR.
Saxe et al. (2013) Saxe, A. M., McClelland, J. L., and Ganguli, S. (2013). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120.
Srivastava et al. (2014) Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958.
Taghvaei et al. (2017) Taghvaei, A., Kim, J. W., and Mehta, P. (2017). How regularization affects the critical points in linear networks. Advances in neural information processing systems, 30.
Venturi et al. (2019) Venturi, L., Bandeira, A. S., and Bruna, J. (2019). Spurious valleys in one-hidden-layer neural network optimization landscapes. Journal of Machine Learning Research, 20:133.
Yun et al. (2018) Yun, C., Sra, S., and Jadbabaie, A. (2018). Small nonlinearities in activation functions create bad local minima in neural networks. arXiv preprint arXiv:1802.03487.
Ziyin et al. (2021) Ziyin, L., Li, B., Simon, J. B., and Ueda, M. (2021). Sgd may never escape saddle points.
Ziyin et al. (2022) Ziyin, L., Zhang, H., Meng, X., Lu, Y., Xing, E., and Ueda, M. (2022). Stochastic neural networks with infinite width are deterministic.
Appendix A Proofs
a.1 Proof of Lemma 1
Proof. Note that the first term in the loss function is invariant to the following rescaling for any a>0:
{Ui→aUi;Wij→Wij/a; (32)
meanwhile, the L2 regularization term changes as a changes. Therefore, the global minimum must have a minimized a with respect to any U and W.
One can easily find the solution:
a∗=argmina(γua2U2i+γw∑jW2ija2)=(γw∑jW2ijγuU2i)1/4. (33)
Therefore, at the global minimum, we must have γua2U2i=γw∑jW2ija2, so that
(U∗i)2=(a∗Ui)2=γwγu∑j(W∗ij)2, (34)
which completes the proof. □
Proof. By Lemma 1, we can write Ui as bi and Wi: as biwi where wi is a unit vector, and finding the global minimizer of Eq. (2) is equivalent to finding the minimizer of the following objective,
Ex,ε⎡⎣(∑i,jb2iϵiwijxj−y)2⎤⎦+(γu+γw)||b||22, (35)
The lemma statement is equivalent to bi=bj for all i and j.
We prove by contradiction. Suppose there exist i and j such that bi≠bj, we can choose i to be the index of bi with maximum b2i, and let j be the index of bj with minimum b2j. Now, we can construct a different solution by the following replacement of biwi: and bjwj::
{b2iwi:→c2v;b2jwj:→c2v, (37)
where c is a positive scalar and v is a unit vector such that 2c2v=b2iwi:+b2jwj:. Note that, by the triangular inequality, 2c2≤b2i+b2j. Meanwhile, all the other terms, bk for k≠i and k≠j, are left unchanged. This transformation leaves the first term in the loss function (36) unchanged, and we now show that it decreases the other terms.
The change in the second term is
(b2i∑kwikxk)2+(b2j∑kwjkxk)2→2(c2∑kvkxk)2=12(b2i∑kwikxk+b2j∑kwjkxk)2. (38)
By the inequality a2+b2≥(a+b)2/2, we see that the left hand side the larger than the right hand side.
We now consider the L2 regularization term. The change is
(γu+γw)(b2i+b2j)→2(γu+γw)c2, (39)
and the left hand side is again larger than the right hand side by the inequality mentioned above: 2c2≤b2i+b2j. Therefore, we have constructed a solution whose loss is strictly smaller than that of the global minimum: a contradiction. Thus, the global minimum must satisfy
U2i=U2j (40)
for all i and j.
Likewise, we can show that UiWi:=UjWj: for all i and j. This is because the triangular inequality 2c2≤b2i+b2j is only an equality if UiWi:=UjWj:. If UiWi:≠UjWj:, following the same argument above, we arrive at another contradiction. □
a.3 Proof of Proposition 1
Proof. We first show that there exists a constant r such that the global minimum must be confined within a (closed) r-Ball around the origin. The objective (18) can be upper-bounded by
Eq. (???)≥γu||U||2+D∑i=1γi||W(i)||2≥γmin(||U||2+∑i||W(i)||2), (41)
where γmin:=mini∈{u,1,2,...,D}>0. Now, let w denote be the union of all the parameters (U,W(i)) and viewed as a vector. We see that the above inequality is equivalent to
Eq. (???)≥γmin||w||2. (42)
Now, note that the loss value at the origin is E[y2], which means that for any w, whose norm ||w||2≥E[y2]/γmin, the loss value must be larger than the loss value of the origin. Therefore, let r=E[y2]/γmin, we have proved that the global minimum must lie in a closed r-Ball around the origin.
As the last step, because the objective is a continuous function of w and the r-Ball is a compact set, the minimum of the objective in this r-Ball is achievable. This completes the proof. □
a.4 Proof of Theorem 2
Proof. Note that the trivial solution is also a special case of this solution with b=0. We thus focus on deriving the form of the nontrivial solution.
We prove by induction on D. The base case with depth 1 is proved in Theorem 1. We now assume that the same holds for depth D−1 and prove that it also holds for depth D.
For any fixed W(1), the loss function can be equivalently written as
E~xEϵ(2),...,ϵ(D)⎛⎝d1,d2,...dD∑i1,i2,...,iDUiDϵ(D)iD...ϵ(2)i2W(2)i2i1~xi1−y⎞⎠2+γu||U||2+D∑i=2γi||W(i)||2+const., (43)
where ~x=ϵ(1)i1∑iW(1)i1ixi. Namely, we have reduced the problem to a problem involving only a depth D−1 linear net with a transformed input ~x.
By the induction assumption, the global minimum of this problem takes the form in Eq. (19), which means that the loss function can be written as the following form:
E~xEϵ(2),...,ϵ(D)⎛⎝bubD...b3d1,d2,...dD∑i1,i2,...,iDϵ(D)iD...ϵ(2)i2vi1~xi1−y⎞⎠2+L2 reg., (44)
for an arbitrary optimizable vector vi1. The term ∑d2,...dDi2,...,iDϵ(D)iD...ϵ(

redpajama_set_name: RedPajamaCommonCrawl


Aviators flight log book : Air mauritius flight schedule : Find me a flight.
Flight Log (1966-1976), released in January 1977 RCA 3766 (Grunt 1255). It is a compilation of Jefferson Airplane and Airplane-related tracks, including tracks by Jefferson Starship and Hot Tuna, as well as solo tracks by Paul Kantner, Grace Slick, and Jorma Kaukonen.
The Aviators is a new weekly magazine-style TV series featuring interesting people, the latest aircraft, current technology and fly-in destinations.
An aviator is a person who flies on aircraft as a profession. The first recorded use of the term (aviateur in French) was in 1887 as a variation of 'aviation', from the Latin 'avis' (meaning bird), coined 1863 by G.
Commander Harris B. Millerd, Ret. died peacefully in his sleep July 2 at the Coarsegold Sierra Estates assisted living home in the company of his wife Mary Castner Millerd, his eldest son Harris B. Millerd, Jr. of Kentucky and his eldest daughter Mary Jean LaGarde of Massachusetts.
Commander Millerd was born September 30, 1918 at the Harris Ranch in Mariposa, now known as the Ponderosa Basin, which was homesteaded by his grandfather, James Marshall Harris in 1863. His grandmother Alice May Harris thought it best to for her daughter to stay at the Harris Ranch until the baby was old enough to travel back to the Millerd ranch in Coarsegold which his paternal grandfather George Millerd had began homesteading in 1910. Thus it was that Alice May Millerd holding her six month old son climbed onto a stack of banded lumber at Sugar Pine and road the flume back down to Coarsegold where her husband Hugh Henry Millerd, a flume herder for the Sugar Pine lumber company, plucked his wife and young son from this fast floating lumber train.
In his teens he and his dad returned to the Harris Ranch to help build the Harris sawmill which processed thousands of board feet of ponderosa pine for the war efforts. While at the Harris ranch he met Ed Nicols a neighboring landowner who was also the manager of the Merced airport. He and his younger brother Glenn L. Millerd persuaded the manager to give them flying lessons in exchange for their labor. Harris said they earned $0.75 an hour to scrub the airport runway, clean engine parts, and test motors; the flying lessons cost $8 an hour. When he enlisted in the Navy in 1942 his mechanical and flying knowledge garnered him a rank of Petty Officer 3rd class, AMM (Aviation Machinist Mate). During WWII he was stationed in the South Pacific Islands of Guam and Corregidor. He petitioned his California Congressman to recommend him for a commission for officer training; he was awarded that status, attended flight school and earned his wings in [ ] He flew 21 years as an active aviator and became trainer himself in the Navy flight schools. He retired in 1963 from the Navy as a full Commander. As an officer he served in occupied Japan.
In 1955 he adopted a family of five Texas children ages 2 to 11— all at one time!
The adoption of his "instant family" brought him positive recognition in the Navy base newsletter in Corpus Christi where he was serving at the time. He was fond of noting that instant fatherhood was a true test of endurance and a shock to the pocketbook. Throughout his life, In true military style, he kept a log book of all the birthdays, graduations, anniversaries of his 5 adopted children, 2 step children 21 grandchildren, [ number?] great grandchildren, 3 nieces and 3 nephews and never tired of bragging about how great they all were.
After he retired, he lived in Phoenix, Arizona, but loved to spend summers at the Coarsegold ranch panning for gold and helping his dad restore fencing on the 1020 acre homestead. He was proud to see his parents honored as Grand Marshalls for the Coarsegold rodeo.
When his father died in 1972 he and his wife Mary Castner Millerd moved to the Coarsegold ranch to care for his mother who passed way in 1979. In addition to his eldest son and daughter, he is survived by his brother Glenn L. Millerd of Marysville,California; his sister Merylyn S. Whited of Coarsegold; his son Chet Millerd of Temecula, California; his other two daughters, Alice Ann Henne of North Carolina, Julie Beth Shible of St. Petersburg, Florida; and Mary's two daughter's Judy Kelton of Garden City, Utah and Geraldine Evans of Phoenix, Arizona.
In lieu of a standard memorial service he asked that family and friends who had known him throughout his long and colorful life contact his sister Merylyn Whited (559-683-0762; or at PO Box 849 Coarsegold 93614) with a vignette. She and her husband Zygmund John Zee are compiling a DVD with family history, news articles, old photos and the vignettes collected from family and the community. The DVD will be given to those who submitted stories.
Following their widespread use as primary trainers during World War II, many Stearman N2S biplanes were purchased by private owners for use as crop dusters. In 1990, with President George Herbert Walker Bush in office, various parties initiated a search to locate any crop dusters still in existence that in their previous military service may have been flown by the Commander in Chief during his time as an aviation cadet and naval aviator. Cross-checking the President's log book entries with the records of various federal agencies revealed that five N2S-3 aircraft flown by Bush at Naval Air Station (NAS) Minneapolis, Minnesota still existed in private hands. Among them was the aircraft owned by former World War II and Korean War pilot and retired TWA Captain Jack Parker of Essex, Connecticut. His aircraft logged two flights with the Aviation Cadet George H.W. Bush at the controls in January 1943 and finished the war with 2,860 flight hours on the airframe. After the war, it operated with Terry's Aircraft Sales and Surplus and after purchase by Parker, towed banners and flew at warbird gatherings.
Acquired by the museum from Parker in 1992, the aircraft was painted in standard wartime Navy markings (Parker has operated it in Army Air Forces livery) and is currently in indoor static display.
Notes It was perhaps the most produced biplane in all of aviation history with 10,346 examples rolling off the production line of the Stearman Aircraft Company between 1934 and 1945. And though the company christened the trainer with the name "Kaydet," those who flew and maintained it universally called it the Stearman. Initially designed as an entry into a procurement contest for a new Army Air Corps trainer, the Stearman served extensively in the Navy as well, with the sea service accepting the first of 4,318 examples of the biplane in the late-1930s. The earliest version was called the NS by the Navy, with later versions designated N2S. By the time of World War II, both Army and Navy operated standardized versions of the aircraft with interchangeable parts, unique in an era in which joint operations was not standard. No matter the uniform they wore, those associated with the Stearman respected it for its redness, ease of maintenance, low operational costs, and flight characteristics. Challenging to an inexperienced pilot was its tendency to ground-loop in crosswind landings.
The N2S-3 on display was flown twice by President George H.W. Bush on solo flights during his training at Naval Air Station (NAS) Minneapolis, Minnesota, and was acquired after serving as a crop duster. Another N2S-5 in the museum collection was donated by Apollo Astronaut Eugene Cernan and is on display at Naval Air Station (NAS) Whiting Field, Florida.
Dimensions: Length: 25 ft., 4 in.; Height: 9 ft., 2 in.; Wingspan: 32 ft., 2 in.
Weights: Empty: 1,940 lb.; Gross: 2,717 lb.


ZIMBABWE has a new national hero – or at least a new national treasure. Luke Steyn waved the Zimbabwean flag proudly at the Winter Olympics in Russia's Sochi as one of only three participating countries from Africa – after South Africa had pulled out.
20-year-old Steyn was the first Zimbabwean to ever compete in the Winter Olympics last week, qualifying in the discipline of Men's Giant Slalom in alpine skiing. He reached the finishing line in 1 minute 32.20 seconds during his first run and 1 minute 34.35 seconds during his second run, crossing the finishing line barely 21.26 seconds after the US gold medallist Ted Ligety. Steyn only managed to come in on 57th place (out of 72) while still representing Zimbabwe proudly.
Amid all the controversy surrounding the Olympic events in Sochi, from culling stray dogs to limiting gay and lesbian rights, Steyn's story has been celebrated widely as a great achievement that stood out on the international stage. News teams from around the world have covered the exceptional story of the young sportsman, who has single-handedly written history as his country's first-ever Winter Olympian, spinning it as a narrative marked by defying the odds.
Steyn hopes to inspire fellow Zimbabwean in the future to also look into competing in winter sports by setting an example for others. "The response has been incredible," Steyn said.
Luke Steyn was born in Zimbabwe but moved to Switzerland as an infant, where he learned skiing and became competitive at it. He divides his time living between the UK and Colorado, where he trains almost year-round for future competitive events -whenever he isn't business studying for his degree in business management. Watch out for this young man in the 2018 Winter Olympics to be held in South Korea.


text:
Seleção podcast: Portugal begin defence of European Championship crown as Ronaldo returns amid more heroics
Published: Saturday, 16 March 2019 14:56
We are back!!! In Episode 16 of the Seleção podcast, host Danny Pinto welcomes Eric Krakauer of BeIN Sports back to the show.
Danny and Eric discuss the Seleção's 25-man roster for the opening two matches of EURO 2020 qualifying, as well as Cristiano Ronaldo's epic hat-trick vs. Atletico Madrid in the Champions League.
To listen to Episode 16 click on the button below.
Related: Fernando Santos calls us Dyego Sousa and João Félix
Related: The Ron and only - "This is why Juve signed me"
jon/usa
Great podcast, guys! Some very interesting points were raised throughout.
We've undoubtedly got the deepest talent pool in the history of the Seleccao. You can certainly argue that we've had stronger starting 11's over the years, but I honestly...
We've undoubtedly got the deepest talent pool in the history of the Seleccao. You can certainly argue that we've had stronger starting 11's over the years, but I honestly can't recall a time when we have had so many quality Portuguese footballers plying their trade across Europe.
I agree with you guys regarding Bernardo Silva's best position. He has shown flashes of his undeniable quality on the right-wing (and he has been playing quite well in that well in recent weeks for City), but he's definitely played his best football in the heart of midfield. If we are to truly reach the potential of this exciting new generation of Portuguese youngsters, Fernando Santos is going to have to devise a tactical scheme that gets the very best of it's most talented player.
I personally wouldn't be opposed to seeing us line up in a 3-5-2 formation. That system is quite rare in Portuguese football, and I highly doubt Santos will veer away from using a back four, but I actually think it suits the characteristics of some of our most talented players.
Portugal is famous for producing elite wingers, but at this current moment in time, the majority of our most creative players are actually second-strikers/#10's (Bernardo, Bruno Fernandes, Diogo Jota, Joao Felix) and wing-backs (Cancelo, Guerreiro, Semedo, Ricardo, Dalot, etc), so I think it would make sense to utilize a system that would enable these players to excel in their preferred roles.
Jerrold Hewson
Based on this podcast I have come up with a 3-4-1-2 model
the back line is Dias Fonte Pepe
the mid field is Raphael William Neves Cancelo
attacking mid Bernardo
striking Felix Ronaldo
Andrew/NY
Cancelo, Pepe, Dias, Guerrerio
Bernardo, William, Moutinho
Felix, Jota, CR7


text:
Q: crear algoritmo para conocer N numeros primos sin usar mod % Alguien sabe como mejorar el código, sin usar mod% ni trunc ?
Algoritmo este_si
escribir " ingrese cuantos numeros desea consultar"
leer numeros
acum<-0
para i<-1 Hasta numeros Con Paso  1 hacer
    acum<-0
    para j<-1 hasta i Con Paso 1 hacer
        resultado = i / j
        si resultado - trunc(resultado) == 0 Entonces
            acum = acum + 1
        FinSi
    FinPara

    si acum == 2 Entonces
        escribir i, " Es primo"
    SiNo
        Escribir i, " no es Primo"
    FinSi
FinPara

FinAlgoritmo

A: podrías hacer con divisibilidad,
por ejemplo no existen primos pares aparte del 2.
si la suma de las cifras es múltiplo de tres no es primo pues es divisible por 3.
si termina en 5 o 0 es múltiplo de 5 lo cual no es primo.
revisa las demás reglas de divisibilidad.
un número primo no es mayor que su raíz cuadrada, luego se debe mirar la divisibilidad entre 1 y la raíz del número que se desea saber si es primo.
aunque sin mod es mucho más difícil y largo, podría ser un buen ejercicio de programacion sin funciones.

redpajama_set_name: RedPajamaStackExchange


text:
Most expensive Russian art lots in London auction
October 27, 2013 Allrus Art
Nicholas Roerich, 'Proceedings of the Virgin', 1931. Sold at Bonhams for 7,88 million pounds
Most expensive Russian art lots in London auction, called Russian auction was arranged by major auction houses – Christie's, Sotheby`s, Bonhams and MacDougall's. For the purchase of works was spent £ 50 million, a record amount for the "Russian Trading" since 2008. According to the British press, the main buyers were Russians. The most expensive item – a painting of Nicholas Roerich "Proceedings of the Mother of God", which was sold at Bonhams for 7,88 million pounds (eight times more than it was estimated). Over 4,78 million pounds got "Still Life with Fruit" by Ilya Mashkov. A "Child Musicians" by Alexander Volkov and "Sunset" by Ivan Shishkin were sold for 2,1 million pounds each. It was certainly not a record to be, the more expensive artworks had been sold at auction before – Chagall and Soutine, Kandinsky and Malevich, but, for 2013 these were the most expensive Russian art lots in London.
Nicholas Roerich "Sanctuaries and Citadels", painted in India in 1925
Nicholas Roerich's 'Proceedings of the Virgin' not only presents reflections on salvation theme of Roerich, but also remarkable for its artistic qualities. The artist managed to convey the divine silver glow of the heavenly temple and a halo of Our Lady, which is opposed to the red fire of hell and the dark color that creates a sense of hopelessness. The conviction of the artist, the way to Bright Castle complex, possible through the path of spiritual purification and culture. Another key lot of "Russian Auction" – painting by Nicholas Roerich "Sanctuaries and Citadels", painted in India in 1925. This work is in the collection of the famous collector Louis Horch.
Ilya Mashkov 'Still Life with Fruit', Sold at Christie's auction for 4,77 million pounds
Painted in 1910 still life participated in the first exhibition of "Jack of Diamonds", held the same year. For the last time, "Still Life with Fruit" was presented to the public during the exhibition, organized by the Society of Contemporary Art in the City Museum in Amsterdam in 1913. By about 1920, "Still Life with Fruit" was in the collection of William Beffi, a famous collector of works of the Expressionists. Apparently, Beffi acquired the work by Mashkov in 1913. The estimated cost of the still life was 1-1.5 million pounds.
Yuri Annenkov 'Portrait of Anna Akhmatova', Sold at Sotheby's auction for £ 902,500
Akhmatova posed for the artist, and, in his own memoirs, was very patient. Annenkov drew two of her portraits: one with ink, gouache – the second. "Ink" portrait is more known since it was included in a collection of poems by Anna Akhmatova and later became a "textbook" picture. Portrait, painted in gouache, was first published in 1962 in France.
Alexander Volkov, 'Child Musicians', Sold at Bonhams auction for £ 2,1 mln
Ivan Shishkin, 'sunset', sold at MacDougall's auction for 2,1 million pounds
Mikhail Nesterov 'Wanderers. Beyond the Volga'. Sold at MacDougall's auction for £ 1,89 mln
Natalia Goncharova Femme Cubiste. Sold at Sotheby's for £ 662,5 thousand
Nicolai Fechin. Mrs Fechin and Daughter, 1925, oil on canvas. Sold at Sotheby's for £ 1,48 mln
Russian auction also sold a large collection of Russian arts and crafts. The main lot of this collection was the sculpture of Mikhail Vrubel's "Spring", which estimated 150-200 thousand pounds. The Russian collection also included pieces by Faberge and the selection of the Yusupov's and the Soviet porcelain. Important lots included 12 items created in 1919-1922 by masters of the State Porcelain Factory in Petrograd.


text:
Car NewsInterview
Top Gear's Chris Harris on cars (Interview)
13 Apr 2017 Isaac Bober
We get Top Gear's Chris Harris on the phone to talk about cars, his motoring heroes, and why he wants a Chevrolet Camaro Z28
UK AUTOMOTIVE JOURNALIST Chris Harris can drive… and so it's no wonder he got a call to work on the resurrection of Top Gear after it collapsed following Jeremy Clarkson's departure. There followed an ordinary season that made headlines for all the wrong reasons (blame Chris Evans), and now it's back with just three presenters and, as the season goes on is becoming better and better.
We called up Chris Harris to talk about how he got into cars (his mum competed in Autocross), how Matt LeBlanc is a carhead, and why the Internet is pushing Top Gear to be bigger and better all the time. Oh, and why TG decided to use a Toyota 86 for their star drivers to pilot around the test track.
QUESTION: Hi Chris, thanks for taking the time to talk with us. Let's get right into it… How hard is it to drift a Ferrari 458, or similar, compared to a Toyota 86?
ANSWER: I think it's easier as you've got more power to get the car to lose grip and traction in the first place. It's easier to make a 458 drift, but here's the complicated bit, it's probably easier to sustain a drift in a GT86. The tricky bit with so much power is balancing it once you've got it going. So, once all is said and done they probably even each other out…
QUESTION: In the first episode of the new series, you drove a La Ferrari, is there any point building a car quicker than the La Ferrari?
ANSWER: No, there is absolutely no point.
QUESTION: So, what, exactly, do you find fun about driving?
ANSWER: I think it's a series of physical inputs that you put into control services, and you feel what you're doing almost immediately; you're in control of that thing. No matter how complicated vehicles are at the moment, until they become autonomous, you control it.
Your feet make it go faster or slower and your hands make it change direction. I'm constantly thrilled by that, I love it. I love the sensation of speed. I love the fact that if I put good inputs into a vehicle the car is happy, it feels good and responds well. If I do it badly and in a ham-fisted way, the vehicle responds badly, so it's something to take pride in.
QUESTION: With the increasing automation of driving where do you see driving pleasure coming from in the future?
ANSWER: Clearly we're moving towards a time where it's going to become more recreational, so I think you're going to find people moving to track days and driving experiences, but I think this autonomous future isn't very well laid out at the moment and we don't quite know where it's going.
I view it two ways – we're looking at an urban future, and a more rural, or, let's say, an inter-urban future. In a big city like Sydney, where I've been stuck in traffic jams before and thought my life is just ebbing away from me, the idea of being in an autonomous vehicle at that moment where I can sit in the back and do some work appeals to me, I can see that making sense, I can see it happening although I can still see it being a disaster at times, because I'm not sure the infrastructure was designed to have autonomous cars in it.
And what happens if you live out in the countryside? What if you're a farmer out in the middle of nowhere? What if you live out in Broken Hill? Your autonomous car's not much use out there, is it?
So, I see a future of enjoyable driving still being something to do out of town on open roads, I see you doing it on tracks, at track days and driving resorts. I can see driving resorts springing up like golf courses. I can see that in the future as well.
QUESTION: Okay, something less serious, what's the setup for the Star's Toyota 86 – is it in Sport mode, what brakes, suspension and tyres are used?
ANSWER: Standard brakes and tyres because we want no grip, so it's on the same tyres as a Prius. The brakes are standard because we don't pound the laps out, we tend to do one or two laps then stop and have a chat.
The entire interior has been ripped out of the car and it's got a TRG cage which is a bulkhead they use for some racing in Germany. It's got racing seats, and harnesses.
Mechanically, it's pretty standard. We didn't really do anything to it. I went down there and did a load of the testing on it to make sure it was robust enough and there wasn't anything we needed to do to the car.
QUESTION: Can a sports car with an automatic transmission give the same involvement as a manual?
ANSWER: No.
QUESTION: You seem to just like to go fast… Would you prefer to have been The Stig rather than a presenter? … did TG offer The Stig position to you?
ANSWER: Sometimes when I read a nasty review I think it might be quite nice to wear a white suit and a white helmet.
QUESTION: LeBlanc doesn't seem to be a car nut in the same league as previous presenters. How often are you stopping to explain something technical off screen. And, how good is his driving really?
ANSWER: He is a big, big car nut and that's what people are surprised by, I think. He has specific areas of interest – he loves his Porsches and is very knowledgeable about those. I don't have to stop and explain anything to him.
Mechanically, in some ways, he has a deeper knowledge than me, he loves the nuts and bolts; he can take stuff apart and put it back together again.
The only time I'll have to explain anything to him, the only holes I see, is that his understanding and knowledge of cars is very much American and mine is European. He has a few gaps around European cars, but that's not because he's not a petrol head, that's because he's not from where I live and it's a UK based show. He quite often explains stuff to me about American cars.
His driving is good, I was amazed. When I started hanging out with him a bit and realised how good he is on a motorcycle, in my experience anyone that can drive a motorcycle fast tends to be pretty good in the car straight away. You'd be surprised at how good he is – he doesn't get carried by me at all.
QUESTION: Have you reconciled with Lamborghini since the spat, and how has it affected what you test on TG?
ANSWER: I'm never going to be best friends with them. But then I didn't enter this world to be best friends with car manufacturers. I suppose I'll sit down with them at some point. They have a new boss. I think he's quite a good bloke, Stefano Domenicali from Ferrari is now the boss of Lamborghini. It doesn't affect me at all.
If a car manufacturer doesn't want to have their car featured on the biggest car show in the world because I've said nasty things about them in the past, well that's their prerogative. I suggest that's not the way to go about it. I've yet to have that happen to me. In Venice, we had a Lamborghini in the show a couple of weeks ago, and I drove that car for a few minutes in the film; no-one from Lamborghini jumped out from behind a bush and told me to get out of it.
QUESTION: You were a motoring writer before you got on the telly, so, what got you into automotive journalism?
ANSWER: I'm a car nut. I've always read car magazines. I read them religiously as a kid. I was influenced a lot by some Australian journalists that dominated Australia and the UK in the 1980s and '90s, names like Peter Robinson – he's a legend and a good friend of mine and is someone I've admired and has nurtured me through my career. I got a break doing work experience for Autocar magazine and I'd studied English as a student. The only thing I could do was write and enjoy cars. I was useless at everything else. And eventually I got all the lucky breaks where I needed them. Every time I needed a door to open it kind of opened. I count myself as very fortunate. I love my job and I never tire of it. I wake up every Monday morning early and I can't wait to get in the next car and drive it.
QUESTION: How has the internet changed the automotive media landscape… does it mean shows like TG need to get bigger and flashier each season?
ANSWER: It's having an effect obviously because you have more competition. There is so much noise out there. There is so much video-based car noise. It's difficult to know where it starts and where it ends. What is content, what is noise and what is journalism? You almost need sub-genres to describe what some of these things are.
How does it affect Top Gear? It keeps us on our toes. But I still think there is a space for a high-quality, beautifully-produced, epically-shot TV show about cars and that is the one thing that the internet cannot compete with. The production values that Top Gear gives people you just cannot replicate on the internet. So, that means it's safe
Top Gear is a broader brand. We need to respond. We have a series of shows called Chris Harris on Cars which are lower budget and I think they are a good response to the internet competition and I'm very interested in keeping those going as I think it is content that sits one or two tiers below the top-level TV show, but still gets a big audience and is more aimed at the real enthusiast. You may see a Ferrari FXX K for 5-7 minutes on the TV show but there is a section of the audience that wants 25 minutes just about the differential in that car and we can deliver that kind of content online to them it's not a TV product but it's not an online product either.
QUESTION: Who were your heroes growing up, motoring or otherwise and why?
ANSWER: I was an Ayrton Senna obsessive. And even though he was competition, I was a Nigel Mansell devotee as well. I loved endurance racing as a kid, so Jack Ickx was my hero too. Weirdly, my mother, who is now 83 years old, was an autocross racer and she was the one who got me into cars.
Journalists were also my heroes. In those days that was the only way you got car material, there was no internet, no YouTube; you read car magazines. I suppose the big names like, LJK Setright, people like Peter Robinson, Steve Cropley and Andrew Frankel, they were all big, big names to me when I was younger. Getting to work with them afterwards and meeting them all has been one of the greatest pleasures of my working life. I got to meet my heroes and work with them.
QUESTION: What's more fun: a fast car slow or a slow car fast?
ANSWER: Slow car fast. No doubt about that.
QUESTION: Do you prefer station wagons or SUVs?
ANSWER: Station wagon every time.
QUESTION: Of all the cars you've driven or been driven in, what was the worst? And what was the best? And why?
ANSWER: My favourite car changes daily. If you could give me one car right now, I'd say the McLaren P1, there's something about the McLaren P1 that just ticks all my boxes and I love it. I've driven some pretty shonky cars in my time, I really have. But I think quite often, anyone can laugh at a rubbish Chinese car that's cheaply made, but some of the most disappointing cars I've driven have carried quite prestigious badges. BMW is a fantastic brand, but I remember when they launched the E65 series, I couldn't believe how bad it was, it looked terrible, it didn't ride well, nothing about it worked. But they turned it around and now they have got an amazing range of cars, but back then that was one of the most disappointing cars I think I'd ever driven.
QUESTION: Who's fastest around the TG track and what was their time?
ANSWER: We've never done it. But if I wasn't first I would resign immediately?
QUESTION: American muscle cars or something British… what and why?
ANSWER: That is a difficult question. I'm in a bit of a muscle car phase now. I'm not that into driving gloves and British sports cars, MGs and all that sort of stuff, that's how people perceive British sports cars and that's not really me. So, right now I'd like a Chevy Camaro Z28.
So you know…
Top Gear airs Mondays at 8.30pm on BBC Knowledge and Chris Harris on Cars, begins Monday May 29 at 8.30pm also on BBC Knowledge.
Tags:Chris HarrisTop gear
Top Gear Season 27 Trailer dropped…
BBC announces new hosts for Top Gear in 2019…
Meet the Lego Stig
Top Gear USA back next week… with an all-new cast