text:
# NOT SO EASY NCERT-1

Chemistry Level 2

Glucose is reacted with excess of Acetic Anhydride. Let the product formed be $$X$$.
$$X$$ is then reacted with Hydroxyl Amine ($$\ce{NH2OH}$$) and let the product formed is $$Y$$.
Find the number of $$\ce{C - N}$$ (carbon-nitrogen single bonds) in $$Y$$.

Assume that all conditions are ideal and the reaction proceed to completion.

×

format: plain


text:
# Integer Apollonian Packings

The slider below allows you to choose from a collection of pre-generated Apollonian packings. You can also click on the figure to zoom in by the factor two. The code often craps out after five or six clicks, however.

The initial packings are chosen so that all four initial circles are integers. This includes the exterior circle, whose oriented curvature is negative. As it turns out, this implies that all subsequent circles in the packing have integer curvatures yielding a configuration sometimes called a Bowl of Integers.

We can see why this works using the Descartes Circle Theorem: Given three, mutually tangent circles with curvatures $\kappa_1$, $\kappa_2$, and $\kappa_3$, there are two more circles mutually tangent to the original three. Furthermore, if the curvatures of such a circle is $\kappa_4$, then these curvatures all satisfy $$(\kappa_1+\kappa_2+\kappa_3+\kappa_4)^2 = 2(\kappa_1^2+\kappa_2^2+\kappa_3^2+\kappa_4^2).$$ For example, in the first labelled packing shown above, there are three gray circles labelled 2, 2, and 3. There is a circle with curvature 15 that is tangent to all three and $$(2+2+3+15)^2 = 2(2^2+2^2+3^2+15^2).$$ There's also an exterior circle with curvature 1 that is tangent to all three. If we apply the convention that an exterior circle has negative curvature, then the formula works again, as you can check.

Note that Descarte's theorem relates the curvatures using a quadratic equation. If we solve for $\kappa_4$ in terms of the others, we get $$\kappa_4 = \kappa_1 + \kappa_2 + \kappa_3 \pm \sqrt{\kappa_1 \kappa_2 + \kappa_1 \kappa_3 + \kappa_2 \kappa_3},$$ corresponding to the two solutions that Descartes' theorem refers to. If we call these two solutions $\kappa_4$ and $\kappa_4'$ and add them together, then the square root terms cancel so that we are left with $$\kappa_4 + \kappa_4' = 2(\kappa_1 + \kappa_2 + \kappa_3).$$ In particular, $\kappa_4'$ must be an integer when the others are.


text:
a field-theory motivated approach to computer algebra

## PartialDerivative

Makes an object a partial derivative.
Makes an object a partial derivative, i.e. a derivative which commutes. The object on which it acts has to be a non-sub/superscript child, while all the sub- or superscript child nodes are interpreted to be the variables with respect to which the derivative is taken.
\partial{#}::PartialDerivative. A_{\mu}::Depends(\partial). ex:= \partial_{\nu}{A_{\mu} B_{\rho}};
$$\displaystyle{}\partial_{\nu}(A_{\mu} B_{\rho})$$
product_rule(_);
$$\displaystyle{}\partial_{\nu}(A_{\mu}) B_{\rho}+A_{\mu} \partial_{\nu}(B_{\rho})$$
unwrap(_);
$$\displaystyle{}\partial_{\nu}(A_{\mu}) B_{\rho}$$
Note that derivative objects do not necessarily need to have a sub- or superscript child, they can be abstract derivatives as in
D{#}::PartialDerivative. ex:= D(c d e);
$$\displaystyle{}D(c d e)$$
product_rule(_);
$$\displaystyle{}D(c) d e+c D(d) e+c d D(e)$$
If you want to write a derivative with respect to a coordinate (instead of with respect to an index, as in the first example above), refer to the Coordinate property.


text:
# differential and inverting op amp problems

I am trying to offset a triangle signal of 4 volts peak to peak by around 10.2V up with only a +20 volt supply voltage, I do it using a differential amplifier but the output signal is inverted so I tried to invert it back by using an inverting op-amp but instead of just inverting the signal it reduced the output of the voltage to a few millivolts with a very low peak to peak value and move it back to the origin point. Any idea on why this happens and how to solve it?

Below is my schematic diagram, Vin is my input signal and Vout1 is the offset that I want but only being inverted instead, and Vout2 is what happens after it passes thru the inverting op-amp

• Where is the schematic? – Unknown123 May 17 '19 at 13:21
• I do it using a differential amplifier but the output signal is inverted You're using the first opamp as an inverting amplifier (the signal is applied to the inverting input so that inverting is totally expected. How about using a non-inverting amplifier? – Bimpelrekkie May 17 '19 at 13:39
• This is not a forum. If you found the solution, post it as an answer, then accept the answer. – JRE May 17 '19 at 14:19
• @JRE You should instead yell at the high-rep users who insist on answering the question in the comment field so they can't be accepted. – pipe May 17 '19 at 14:25
• @JRE Really ….the OP made a simple mistake, it didn't warrant a points chaser answer. Not every question posted here requires explanation beyond pointing out an error. But I'm sure someone will post one. – Jack Creasey May 17 '19 at 14:28

As already mentioned by the others, it fails because the inverting amplifier stage will also invert your signal along with the the DC offset you have added.

Here's the correct one. Figure 1. Single supply amplfier (gain adjustable) with DC offset (can be gain adjustable if needed).

The transfer function would be

$$\\displaystyle V_{out2} = V_{out1} \left(-\frac{R_6}{R_5}\right) + V_{ref} \left(1 + \frac{R_6}{R_5} \right) \$$

$$\\displaystyle V_{out1} = V_{in}\left(-\frac{R_2}{R_1}\right) + V_{ref}\left(\frac{R_4}{R_3+R_4}\right)\left( \frac{R_1+R_2}{R_1}\right) \$$

Thus

$$\\displaystyle V_{out2} = V_{in}\left(-\frac{R_2}{R_1}\right) \left(-\frac{R_6}{R_5}\right) + V_{ref} \left[ \left(\frac{R_4}{R_3+R_4}\right)\left( \frac{R_1+R_2}{R_1}\right) \left(-\frac{R_6}{R_5}\right) + \left(1 + \frac{R_6}{R_5} \right) \right]\$$

If $$\ R_1 = R_2 = R_3 = R_4 \$$, then

$$\\displaystyle V_{out2} = V_{in} \left(\frac{R_6}{R_5}\right) + V_{ref} \$$

Also if $$\R_5 = R_6\$$ (which is the circuit above), then.

$$\ V_{out2} = V_{in} + V_{ref} \$$

The differential gain stage must be at unity or the $$\V_{ref}\$$ factor will not be one anymore. For example if the differential gain stage is 10 and inverting gain stage is -2. Then $$\ V_{out2} = 20V_{in} - 17V_{ref} \$$. Which is absurd.

In addition, the DC offset reference must have low output impedance, or it will messed up the differential gain stage.

You could also change the reference gain stage by providing feedback resistor instead of the usual non-inverting buffer. It will be useful in case you have low voltage precision reference, such as TL431, or 5V DAC, etc. as a substitute for the voltage divider above.

Here is the waveform.

Figure 2. The output waveform of the schematic above.

From a long time ago, I always like to illustrate inverting amplifier like a seesaw, it seems like fun. I imagine both a meter as a voltage indicator and a seesaw as the amplifier. Instead of sit-down on my bench side, I decide to push it by my own hand up/down which acting like an input signal according to the meter. The fulcrum/pivot is positioned according to the resistor ratio. If it has has the same ratio, then it will be positioned in the middle. And whenever there is a reference voltage, i jack up/down the fulcrum. Then, the other side of bench will be the output signal. Not that I implicitly claim this idea is original, but it always works for me.

Figure 3. Seesaw as an analogy of Inverting Amplifier Op-Amp configuration.

By the way, what is your gain requirement? If your goal is unity gain, then you're just wasting an op-amp. Use non-inverting summing amplifier configuration instead. Or even better, let's modify existing already made unity gain differential amplifier chip. For example INA105, it isn't available in the LTSpice library, so search the model and also already made symbol if exist in the LTSpice Yahoo! Groups. There's also exist higher input impedance unity differential amplifier such as AD8479 if needed.

Figure 4. Unity Gain Version, Differential Amplifier chip as a Non-Inverting Summing Amplifier.

The transfer function, as you already guessed it, $$\V_{out} = V_{in} \cdot \text{Gain} + V_{ref}\$$.

It's uncommon to see people called it non-inverting summing amplifier configuration because of the gain adjustment is not straightforward as to the common inverting one. People may say it's differential amplifier with reference voltage and V- input grounded. But, in this specific case to call it non-inverting summing amplifier when unity gain differential amplifier is used isn't incorrect either.

I assumed that you don't have negative supply (or can't have it) and also need DC signal (or very low frequency AC signal) to be also amplified. If your frequency is not too low or not too high, let's say audio, then then the common dc-biased ac-coupled non-inverting amplifier configuration can be used, such as in the answer on the other question here.


text:
# Python Indices of numbers greater than K

In this tutorial, we are going to find the indices of the numbers that are greater than the given number K. Let's see the different ways to find them.

A most common way to solve the problem is using the loops. Let's see the steps to solve the problem.

• Initialize the list and K.
• Iterate over the list using its length.
• If you find any number greater than K, then print the current index.

## Example

Live Demo

# initializing the list and K
numbers = [3, 4, 5, 23, 12, 10, 16]
K = 10
# iterating over thAe list
for i in range(len(numbers)):
# checking the number greater than K
if numbers[i] > K:
# printing the number index
print(i, end=' ')

## Output

If you run the above code, then you will get the following result.

3 4 6

Let's solve the problem using enumerate function. It gives you a tuple for each iteration that includes the element's index and element.

## Example

Live Demo

# initializing the list and K
numbers = [3, 4, 5, 23, 12, 10, 16]
K = 10
# finding indexes of the numbers greater than K
result = [index for (index, number) in enumerate(numbers) if number > K]
# printing the indices
print(*result)

## Output

If you run the above code, then you will get the following result.

3 4 6

## Conclusion

If you have any doubts in the tutorial, mention them in the comment section.


text:
# Dark Matter Abundance and Being a Cold Relic

I was reading Longair its stated that,

"An important conclusion was that, if dark matter particles were as common as neutrinos and photons,their rest mass energies could not be much greater than $$10eV$$, or the present density of the Universe would exceed the critical mass density $$Ω_0 = 1$$. Therefore, there would have to be some suppression mechanism to ensure that the massive particles were much less common than the photons and neutrinos at the present day.

There is a natural way in which this can occur if the decoupling of the WIMPs from thermal equilibrium took place after they had become non-relativistic, that is, the decoupling took place after the epoch at which $$kT ∼ m_Xc^22$$, where $$m_X$$ is the mass of the WIMP."

I did not understand this conclusion. What is the relationship between having less DM particle and being cold relic ?

When the dark matter particles were relativistic, their number density goes like $$n_{rel} \sim T^3$$. After becoming non-relativistic, they are exponentially suppressed $$n_{non-rel} \sim T^{3/2} e^{-m/T}$$. (These are found by solving the phase space distributions for species in kinetic equilibrium).


text:
American Institute of Mathematical Sciences

July  2006, 14(3): 597-615. doi: 10.3934/dcds.2006.14.597

Minimum 'energy' approximations of invariant measures for nonsingular transformations

 1 Department of Mathematics and Statistics, University of Victoria, P.O. Box 3045, Victoria, BC, Canada V8W 3P4 2 Department of Mathematics, University of Waikato, Private Bag 3105, Hamilton

Received  October 2004 Revised  May 2005 Published  December 2005

We study variational methods for rigorous approximation of invariant densities for a nonsingular map $T$ on a Borel measure space. The general method takes the form of a convergent sequence of optimization problems on $L^p$, $1 \leq p < \infty$ with a convex objective and finite moment constraints. Provided $T$ admits an invariant density in the appropriate $L^p$ space, weak convergence of the sequence of optimal solutions is observed; norm convergence can be obtained when the objective is a Kadec functional. No regularity or expansiveness assumptions on $T$ need to be made, and the method applies to maps on multidimensional domains. Objectives leading to norm convergence include Entropy, 'Energy' and 'Positively Constrained Energy'.
Explicit solutions for the finite moment problems in the case of the 'Energy' functional are derived using duality - the optimality condition is then a linear algebra problem. Strong duality is obtained even though the dual functional may not be coercive and the set of moment test functions is not assumed to be pseudo-Haar. Finally, some numerical studies are presented for the case of moment test functions derived from a finite partition of the dynamical phase space and the results are compared with Ulam's method.
Citation: Christopher Bose, Rua Murray. Minimum 'energy' approximations of invariant measures for nonsingular transformations. Discrete & Continuous Dynamical Systems, 2006, 14 (3) : 597-615. doi: 10.3934/dcds.2006.14.597
 [1] Manfred Einsiedler, Elon Lindenstrauss. On measures invariant under diagonalizable actions: the Rank-One case and the general Low-Entropy method. Journal of Modern Dynamics, 2008, 2 (1) : 83-128. doi: 10.3934/jmd.2008.2.83 [2] Claudia Lederman, Noemi Wolanski. An optimization problem with volume constraint for an inhomogeneous operator with nonstandard growth. Discrete & Continuous Dynamical Systems, 2021, 41 (6) : 2907-2946. doi: 10.3934/dcds.2020391 [3] Bing Gao, Rui Gao. On fair entropy of the tent family. Discrete & Continuous Dynamical Systems, 2021, 41 (8) : 3797-3816. doi: 10.3934/dcds.2021017 [4] Jicheng Liu, Meiling Zhao. Normal deviation of synchronization of stochastic coupled systems. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021079 [5] Julian Koellermeier, Giovanni Samaey. Projective integration schemes for hyperbolic moment equations. Kinetic & Related Models, 2021, 14 (2) : 353-387. doi: 10.3934/krm.2021008 [6] Elena Bonetti, Pierluigi Colli, Gianni Gilardi. Singular limit of an integrodifferential system related to the entropy balance. Discrete & Continuous Dynamical Systems - B, 2014, 19 (7) : 1935-1953. doi: 10.3934/dcdsb.2014.19.1935 [7] Y. Latushkin, B. Layton. The optimal gap condition for invariant manifolds. Discrete & Continuous Dynamical Systems, 1999, 5 (2) : 233-268. doi: 10.3934/dcds.1999.5.233 [8] Charlene Kalle, Niels Langeveld, Marta Maggioni, Sara Munday. Matching for a family of infinite measure continued fraction transformations. Discrete & Continuous Dynamical Systems, 2020, 40 (11) : 6309-6330. doi: 10.3934/dcds.2020281 [9] Ugo Bessi. Another point of view on Kusuoka's measure. Discrete & Continuous Dynamical Systems, 2021, 41 (7) : 3241-3271. doi: 10.3934/dcds.2020404 [10] Alexander Tolstonogov. BV solutions of a convex sweeping process with a composed perturbation. Evolution Equations & Control Theory, 2021  doi: 10.3934/eect.2021012 [11] Zhang Chen, Xiliang Li, Bixiang Wang. Invariant measures of stochastic delay lattice systems. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3235-3269. doi: 10.3934/dcdsb.2020226 [12] Clara Cufí-Cabré, Ernest Fontich. Differentiable invariant manifolds of nilpotent parabolic points. Discrete & Continuous Dynamical Systems, 2021  doi: 10.3934/dcds.2021053 [13] Ruchika Sehgal, Aparna Mehra. Worst-case analysis of Gini mean difference safety measure. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1613-1637. doi: 10.3934/jimo.2020037 [14] Teddy Pichard. A moment closure based on a projection on the boundary of the realizability domain: 1D case. Kinetic & Related Models, 2020, 13 (6) : 1243-1280. doi: 10.3934/krm.2020045 [15] Joel Fotso Tachago, Giuliano Gargiulo, Hubert Nnang, Elvira Zappale. Multiscale homogenization of integral convex functionals in Orlicz Sobolev setting. Evolution Equations & Control Theory, 2021, 10 (2) : 297-320. doi: 10.3934/eect.2020067 [16] Craig Cowan. Supercritical elliptic problems involving a Cordes like operator. Discrete & Continuous Dynamical Systems, 2021  doi: 10.3934/dcds.2021037 [17] John Villavert. On problems with weighted elliptic operator and general growth nonlinearities. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021023 [18] Paul A. Glendinning, David J. W. Simpson. A constructive approach to robust chaos using invariant manifolds and expanding cones. Discrete & Continuous Dynamical Systems, 2021, 41 (7) : 3367-3387. doi: 10.3934/dcds.2020409 [19] Marcel Braukhoff, Ansgar Jüngel. Entropy-dissipating finite-difference schemes for nonlinear fourth-order parabolic equations. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3335-3355. doi: 10.3934/dcdsb.2020234 [20] Muberra Allahverdi, Harun Aydilek, Asiye Aydilek, Ali Allahverdi. A better dominance relation and heuristics for Two-Machine No-Wait Flowshops with Maximum Lateness Performance Measure. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1973-1991. doi: 10.3934/jimo.2020054

2019 Impact Factor: 1.338


text: A note on $U_n\times U_m$ modular invariants We consider the rings of invariants $R^G$, where $R$ is the symmetric algebra of a tensor product between two vector spaces over the field $F_p$ and $G=U_n\times U_m$. A polynomial algebra is constructed and these invariants provide Chern classes for the modular cohomology of $U_{n+m}$. Keywords:Invariant theory, cohomology of the unipotent groupCategory:13F20


text:
# nLab Hasse quiver

Warning: This page is tentative.

# Idea

A Hasse $n$-quiver is a free n-poset on a Hasse n-graph

A Hasse $1$-quiver is, as usual, referred to simply as a Hasse quiver.

Revised on October 24, 2009 01:35:10 by Toby Bartels (173.51.68.54)


text:
# Multiplication and Division Word Problems

Multiplication and Division Word Problems
• Page 1
1.
Charlie bought 2 ice-creams for $2. What is the cost of each ice-cream?  a.$1 b. $2 c.$1.5 d. $0.5 #### Answer: (a) Correct answer : (1) 2. Justin bought 3 pencils for$6. What is the cost of each pencil?
 a. $2.5 b.$2 c. $1.5 d.$3

3.
Find the number of toys in each shelf, if 28 toys are equally arranged in 4 shelves.
 a. 6 b. 7 c. 9 d. 4

4.
At a factory 3 wheels are used for each tricycle. How many tricycles can be made using 93 wheels?
 a. 30 b. 32 c. 33 d. 31

5.
The baker puts 8 cookies in a tray. How many trays are needed for 64 cookies?
 a. 8 b. 9 c. 72 d. 512

6.
Every day Bob puts $3 in his piggy bank. If he started on Sunday, what would be the total amount in his bank on Friday?  a.$24 b. $16 c.$15 d. \$18

7.
Pam harvested 36 cabbages in 6 rows in her garden. If there were same numbers of cabbages in each row, how many cabbages were there in each row?
 a. 6 b. 42 c. 7 d. 30

8.
Mathew has 4 boxes of crayons. There are 5 crayons in each box. How many crayons does Mathew have?
 a. 20 b. 9 c. 15 d. 25

9.
William bought 6 apples. Each apple was priced at 4¢. How much did he pay for the apples?
 a. 18¢ b. 10¢ c. 24¢ d. 20¢


text:
# Synopsis: Keep it local

A new tool has been developed to measure the local London penetration depth in a superconductor.

The typical distance that an applied magnetic field penetrates a superconductor—called the penetration depth, $\lambda$—is an important measure that can be directly related to the superfluid density. Absolute values of $\lambda$ are notoriously difficult to ascertain. Most experiments therefore measure the difference, $\mathrm{\Delta }$$\lambda$ = $\lambda$$\text{(}T\right)-\lambda \left(\text{0}\right)$, which has the same temperature dependence as the superfluid density at low temperatures. A power-law dependence in $\mathrm{\Delta }$$\lambda$ suggests there are nodes in the superconductivity gap, while an exponential dependence implies a fully open gap.

Recent measurements of $\mathrm{\Delta }$$\lambda$ in iron pnictide superconductors of the $\text{Ba}$-$122$ family show conflicting behavior: when the superconductor is doped with cobalt, there is a power-law dependence, suggesting nodes in the gap, but when doped with potassium, different experiments do not agree on the absence or presence of nodes in the gap. It turns out that to resolve this conflict, it may be necessary to measure the absolute value of $\lambda$.

In a Rapid Communication appearing in Physical Review B, Lan Luan and collaborators from Stanford University in the US have found a new way of measuring both the absolute value of the penetration depth and its spatial homogeneity using magnetic force microscopy (MFM) and SQUID susceptometry. Lan Luan et al. observe that for the $\text{Ba}$-$122$ compound doped with cobalt, the superfluid density has a temperature dependence that is consistent with a fully gapped two-band model, similar to the potassium-doped material. Further, $\lambda$ is found to be spatially homogeneous at the submicron scale, and absolute values of $\lambda$ suggest that phase fluctuations are not as important for iron pnictides as for the underdoped cuprates. The ability of the new tool to obtain the absolute values of the penetration depth and to map its spatial variation down to the submicron scale is likely to be extremely useful. – Sarma Kancharla

More Features »

### Announcements

More Announcements »

## Subject Areas

Superconductivity

## Previous Synopsis

Particles and Fields

Read More »

## Next Synopsis

Statistical Physics

Read More »

## Related Articles

Superconductivity

### Synopsis: Bismuthates Are Surprisingly Conventional

Photoemission experiments challenge the long-held belief that the high-temperature superconductivity of certain bismuth oxides is of the unconventional type. Read More »

Condensed Matter Physics

### Viewpoint: Cuprate Superconductors May Be Conventional After All

Experiments on a copper-based high-temperature superconductor uncover the existence of vortex states—a hallmark of conventional superconductivity. Read More »

Condensed Matter Physics

### Viewpoint: Order on Command

A current of electrons with aligned spins can be used to modify magnetic order and superconductivity in an iron-based superconductor. Read More »


text:
Non-vanishing of twists of L functions for GL(4)

Hello,

This is a question in the spirit of Nonvanishing of central L-values of quadratic twists? and the application I have in mind is to p-adic L-functions a la Ash-Ginzburg.

The question is this:

Suppose $\pi$ is an irreducible unitary automorphic cuspidal representation of $GL(4)$, say, over ${\mathbb Q}$. Under what conditions on $\pi$, do we know the existence of a finite order $\chi$ such that $L(\frac{1}{2}, \pi \otimes \chi) \ne 0$? Here $L$ is the completed $L$-function.

I know that by a result of Luo from 2005 if $\Re (s ) \ne 1/2$, then there is $\chi$ such that $L(s, \pi \otimes \chi) \ne 0$.

Any help would be greatly appreciated.

-
I bet this is an open problem. The issue is that, if $\chi$ has conductor $q$, then the approx. fun. eqn. gives $L(1/2,\pi \otimes \chi)$ as a sum of about $q^2$ terms, and if you average over $\chi$ and $q$ there's an obvious main term, but the error term just barely fails to be manageable (you can succeed on $GL(2)$ and $GL(3)$.) –  David Hansen Apr 12 '11 at 21:28
Hey Ramin, if you take a look at the discussion over at mathoverflow.net/questions/60387 this might apply to the application you're interested in. I don't know much about the $p$-adic $L$-functions for $GL(4)$, but if the Ash–Ginzburg $p$-adic $L$-function is interpolating the values at critical integers (and your central point is critical, like an "even weight" condition for $GL(2)$), then as long as the weight of your automorphic representation is sufficiently regular ($k>2$ for $GL(2)$), the non-vanishing of the $p$-adic $L$-function trivially follows from the non-vanishing of an ... –  Rob Harron Apr 13 '11 at 3:20
(cont'd) $L$-value in the range of convergence of the Euler product. The fact that the $p$-adic $L$-function is a $p$-adic measure will then imply that it has only finitely many zeros and hence there's some twist of the central value which is non-zero. Now, the $p$-adic $L$-functions in Ash–Ginzburg are just for trivial coefficients, i.e. a very small weight, so I imagine this doesn't apply. However, Rob Pollack's answer to the aforementioned mathoverflow question outlines an idea for using higher weight automorphic representations to prove the result for small weights. –  Rob Harron Apr 13 '11 at 3:21
(cont'd) However, this would involve constructing two-variable $p$-adic $L$-functions for $GL(4)$, and knowing something about their $\mu$-invariants, so that would be what they call "difficult". –  Rob Harron Apr 13 '11 at 3:21
Hi Rob, This is very helpful! Thank you. –  Ramin Apr 13 '11 at 14:59


text:
American Institute of Mathematical Sciences

• Previous Article
The impact of toxins on competition dynamics of three species in a polluted aquatic environment
• DCDS-B Home
• This Issue
• Next Article
Asymptotic profiles of the endemic equilibrium of a reaction-diffusion-advection SIS epidemic model with saturated incidence rate
June  2021, 26(6): 3023-3041. doi: 10.3934/dcdsb.2020218

The Keller-Segel system with logistic growth and signal-dependent motility

 1 Department of Mathematics, South China University of Technology, Guangzhou, 510640, China 2 Department of Applied Mathematics, Hong Kong Polytechnic University, Hung Hom, Hong Kong

*Corresponding author: Zhi-An Wang

Received  February 2020 Revised  May 2020 Published  July 2020

Fund Project: The research of H.Y. Jin was supported by the NSF of China (No. 11871226), Guangdong Basic and Applied Basic Research Foundation (No. 2020A1515010140), Guangzhou Science and Technology Program No.202002030363 and the Fundamental Research Funds for the Central Universities. The research of Z.A. Wang was supported by the Hong Kong RGC GRF grant 15303019 (Project ID P0030816)

The paper is concerned with the following chemotaxis system with nonlinear motility functions
 $$$\label{0-1}\begin{cases}u_t = \nabla \cdot (\gamma(v)\nabla u- u\chi(v)\nabla v)+\mu u(1-u), &x\in \Omega, ~~t>0, \\ 0 = \Delta v+ u-v, & x\in \Omega, ~~t>0, \\u(x, 0) = u_0(x), & x\in \Omega, \end{cases}~~~~(\ast)$$$
subject to homogeneous Neumann boundary conditions in a bounded domain
 $\Omega\subset \mathbb{R}^2$
with smooth boundary, where the motility functions
 $\gamma(v)$
and
 $\chi(v)$
satisfy the following conditions
 $(\gamma, \chi)\in [C^2[0, \infty)]^2$
with
 $\gamma(v)>0$
and
 $\frac{|\chi(v)|^2}{\gamma(v)}$
is bounded for all
 $v\geq 0$
.
By employing the method of energy estimates, we establish the existence of globally bounded solutions of ($\ast$) with
 $\mu>0$
for any
 $u_0 \in W^{1, \infty}(\Omega)$
with
 $u_0 \geq (\not\equiv) 0$
. Then based on a Lyapunov function, we show that all solutions
 $(u, v)$
of ($\ast$) will exponentially converge to the unique constant steady state
 $(1, 1)$
provided
 $\mu>\frac{K_0}{16}$
with
 $K_0 = \max\limits_{0\leq v \leq \infty}\frac{|\chi(v)|^2}{\gamma(v)}$
.
Citation: Hai-Yang Jin, Zhi-An Wang. The Keller-Segel system with logistic growth and signal-dependent motility. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3023-3041. doi: 10.3934/dcdsb.2020218
References:

show all references

References:
 [1] Harumi Hattori, Aesha Lagha. Global existence and decay rates of the solutions for a chemotaxis system with Lotka-Volterra type model for chemoattractant and repellent. Discrete & Continuous Dynamical Systems, 2021  doi: 10.3934/dcds.2021071 [2] Xu Pan, Liangchen Wang. Boundedness and asymptotic stability in a quasilinear two-species chemotaxis system with nonlinear signal production. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021064 [3] Manoel J. Dos Santos, Baowei Feng, Dilberto S. Almeida Júnior, Mauro L. Santos. Global and exponential attractors for a nonlinear porous elastic system with delay term. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2805-2828. doi: 10.3934/dcdsb.2020206 [4] Lu Xu, Chunlai Mu, Qiao Xin. Global boundedness of solutions to the two-dimensional forager-exploiter model with logistic source. Discrete & Continuous Dynamical Systems, 2021, 41 (7) : 3031-3043. doi: 10.3934/dcds.2020396 [5] Ying Yang. Global classical solutions to two-dimensional chemotaxis-shallow water system. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2625-2643. doi: 10.3934/dcdsb.2020198 [6] Mehmet Duran Toksari, Emel Kizilkaya Aydogan, Berrin Atalay, Saziye Sari. Some scheduling problems with sum of logarithm processing times based learning effect and exponential past sequence dependent delivery times. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021044 [7] Quan Hai, Shutang Liu. Mean-square delay-distribution-dependent exponential synchronization of chaotic neural networks with mixed random time-varying delays and restricted disturbances. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3097-3118. doi: 10.3934/dcdsb.2020221 [8] Lin Yang, Yejuan Wang, Tomás Caraballo. Regularity of global attractors and exponential attractors for $2$D quasi-geostrophic equations with fractional dissipation. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021093 [9] Irena PawŃow, Wojciech M. Zajączkowski. Global regular solutions to three-dimensional thermo-visco-elasticity with nonlinear temperature-dependent specific heat. Communications on Pure & Applied Analysis, 2017, 16 (4) : 1331-1372. doi: 10.3934/cpaa.2017065 [10] Chenjie Fan, Zehua Zhao. Decay estimates for nonlinear Schrödinger equations. Discrete & Continuous Dynamical Systems, 2021, 41 (8) : 3973-3984. doi: 10.3934/dcds.2021024 [11] Tomoyuki Tanaka, Kyouhei Wakasa. On the critical decay for the wave equation with a cubic convolution in 3D. Discrete & Continuous Dynamical Systems, 2021  doi: 10.3934/dcds.2021048 [12] Mario Pulvirenti, Sergio Simonella. On the cardinality of collisional clusters for hard spheres at low density. Discrete & Continuous Dynamical Systems, 2021, 41 (8) : 3903-3914. doi: 10.3934/dcds.2021021 [13] Alina Chertock, Alexander Kurganov, Mária Lukáčová-Medvi${\rm{\check{d}}}$ová, Șeyma Nur Özcan. An asymptotic preserving scheme for kinetic chemotaxis models in two space dimensions. Kinetic & Related Models, 2019, 12 (1) : 195-216. doi: 10.3934/krm.2019009 [14] Guillermo Reyes, Juan-Luis Vázquez. Long time behavior for the inhomogeneous PME in a medium with slowly decaying density. Communications on Pure & Applied Analysis, 2009, 8 (2) : 493-508. doi: 10.3934/cpaa.2009.8.493 [15] Emily McMillon, Allison Beemer, Christine A. Kelley. Extremal absorbing sets in low-density parity-check codes. Advances in Mathematics of Communications, 2021  doi: 10.3934/amc.2021003 [16] Emanuela R. S. Coelho, Valéria N. Domingos Cavalcanti, Vinicius A. Peralta. Exponential stability for a transmission problem of a nonlinear viscoelastic wave equation. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021055 [17] Hong Yi, Chunlai Mu, Guangyu Xu, Pan Dai. A blow-up result for the chemotaxis system with nonlinear signal production and logistic source. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2537-2559. doi: 10.3934/dcdsb.2020194 [18] Charles Fulton, David Pearson, Steven Pruess. Characterization of the spectral density function for a one-sided tridiagonal Jacobi matrix operator. Conference Publications, 2013, 2013 (special) : 247-257. doi: 10.3934/proc.2013.2013.247 [19] Davide La Torre, Simone Marsiglio, Franklin Mendivil, Fabio Privileggi. Public debt dynamics under ambiguity by means of iterated function systems on density functions. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021070 [20] Wenbin Li, Jianliang Qian. Simultaneously recovering both domain and varying density in inverse gravimetry by efficient level-set methods. Inverse Problems & Imaging, 2021, 15 (3) : 387-413. doi: 10.3934/ipi.2020073

2019 Impact Factor: 1.27


text:
How do you graph the inequality 6(y+1)<-x?

May 12, 2018

graph{6(y+1)<-x [-10, 10, -5, 5]}

Explanation:

Solve for $y$.

$6 \left(y + 1\right) < - x$

$y + 1 < \frac{- x}{6}$

$y < \frac{- x}{6} - 1$

Graph a dotted line along $y = \frac{- x}{6} - 1$.
graph{y=-x/6-1 [-7, 7, -3.5, 3.5]}

Fill in according to the sign of inequality. Since it is a "less than" sign, any space below the line is shaded in.


text:
Issue No. 09 - September (2008 vol. 7)
ISSN: 1536-1233
pp: 1071-1083
ABSTRACT
The problem of searching for mobile users in cellular networks is addressed in this study. Previous studies addressing this issue have focused on the problem of searching for a single user. As it is shown in this study, the problem of maximizing the expected rate of successful searches under delay and bandwidth constraints is NP-hard. {\bf When the potential locations of different users overlap, the derivation of an optimal concurrent search for many independent users from a set of optimal single user searches is NP-hard}. In reality, a cellular network has to serve many competing search requests sharing a limited bandwidth. Since the problem of maximizing the expected rate of successful searches under delay and bandwidth constraints is NP-hard, this study proposes a heuristic algorithm, that is optimal for most probable cases, and its worst case running time complexity is $O(n(\log{n} + C\log{C}))$, where $n$ is the number of mobile users that must be found, and $C$ is the number of their potential locations. {\bf The approximation ratio of the proposed search algorithm is less than 2}.
INDEX TERMS
location management, Paging, mobile, multiple, search, cellular networks.
CITATION
Zohar Naor, "Searching for Multiple Mobile Users", IEEE Transactions on Mobile Computing, vol. 7, no. , pp. 1071-1083, September 2008, doi:10.1109/TMC.2008.25


text:
# Blog

## Deep learning and applied math for financial forecasting [IV] – Dimensionality Reduction [Python]

In previous posts we proposed basic procedures necessary for performing one-step-ahead and multy-step-ahead prediction of S&P500 index based on prices of its constituents. In this post we will focus on accuracy and performance increase using dimension reduction...

## How to build custom Deep learning neural network from scratch in Python [II] – Testing

In previous blog post we proposed how custom Deep learning neural network can be built from scratch using Python. In this post, we will present some verification and testing results.   Let us consider a very simple example of learning the shape of a sinusoidal...

## How to build custom Deep learning neural network from scratch in Python [I] – Basics

Feed-Forward Networks This post illustrates steps required to build and train feed-forward neural networks from scratch in Python. Implementing the Network in Code   We will consider the following $$L$$-layer network structure \[ X^{(l)}_i=...

## Deep learning and applied math for financial forecasting [III] – Deep learning with TensorFlow [Python]

This post illustrates basic procedure necessary for performing one-step-ahead prediction of S&P500 index based on prices of its constituents using feed-forward neural networks implemented in TensorFlow using Keras API. TensorFlow is open source library for dataflow...

## Deep learning and applied math for financial forecasting [II] – simple linear regression, n-steps-ahead [Python]

Let us now attempt to predict future values of S&P500 based on the current values of the constituents. The results obtained for the prediction horizont of n=10 steps ahead will be elaborated, but the same procedure cn be applied for arbitrary n . The results will, off...

## Deep learning and applied math for financial forecasting [I] – simple linear regression [Python]

Introduction Financial forecasting, especially stock price forecasting is one of the key components of the everyday market activities. Small increase of accuracy can lead to significant profit so such algorithms and models attracts significant visibility in academic...

$${}$$


text:
# BibTeX-powered publications list for Pelican with pelican-bibtex

Posted by & filed under Uncategorized.

## Hook

Wouldn’t you like to manage your academic publications list easily within the context of your static website? Without resorting to external services, or to software like bibtex2html, which is very nice but will then require restyling to fit your templates?

Look no more, with the help of pelican-bibtex you can now manage your papers from within Pelican!

## Backstory

At Fabian‘s advice, I started playing around with Pelican, a static website/blog generator for Python. I like it better than the other generators I used before, so I chose it the next time I had to set up a website. I still didn’t make the courage to migrate my current website and blog to it, but I promise I will.

Pelican has a public plugins repository, but they have a license constraint for all contributions. My plugin isn’t complicated, but I had to “reverse engineer” undocumented parts of the pybtex API. I think that maybe that code that I used to render citations programatically can be useful to others, so I don’t want to release it under a restrictive license. For this reason, I publish pelican-bibtex in my personal GitHub account.

You can see it in action in the source code for the website I am working on at the moment, the home page of my research group. Example output generated using pelican-bibtex can be seen here.

## Possible extensions

I have not dug in too deeply but I believe this plugin can be extended, with not much difficulty, to support referencing in Pelican blogs, and render BibTeX references at the end of every post. This idea was suggested by Avaris on #pelican, and I find it very cool. Since I don’t need this feature at the moment, it’s not a priority, but it’s something that I would like to see at some point.

# Really the most common english idioms?

Posted by & filed under corpus linguistics, nlp.

A while back I ran into this blog post and it made me wonder. I’m not a native speaker but the idiomatic phrases that they note as common don’t strike me as such. I don’t think I have ever encountered them very often in real dialogue.

The blog post lists the 10 most common idioms in English. Idioms, also known less ambiguously as fixed expressions, are units of language that span at least two words. Their meaning, relatively to the individual meaning of the parts of the phrase, are figurative. Despite this, fixed expressions don’t classify as creative language, or exploitations. By definition most speakers will unequivocally be familiar with them.

For example, they cite piece of cake as the most common idiomatic expression. This refers to using the phrase to mean that something is easy, that it isn’t challenging. An example of literal use, however, would be when ordering a piece of cake for desert in a restaurant.

Everyone knows that language is a perpetually changing thing, so to begin with it’s even slightly misleading to discuss of the commonness of a phrase, without giving more context. The blog post doesn’t justify the ranking with any numbers anyway, so let’s take them one by one and find out how common they really are!

## Corpus Linguistics

The approach we are taking here is known as corpus linguistics. The best way to argue that a certain phrase is common, that something is used with a specific meaning or that some constructions are normal is, under corpus linguistics, not to make up examples that seem reasonable, but to look at representative collections of text (corpora) and trying to find the examples there. The conclusions you get this way are backed by real-world language use.

An argument often brought against generative linguistics is that it focuses on the (hard) border between grammatical and not grammatical, and the border is usually defined by made-up examples. This is inappropriate for studying how the norms are exploited in real language use, for example. I refer the interested to the work of Patrick Hanks [1, 2].

Corpus linguistics is sensitive to the corpus used. For this example let’s use two British English corpora: the British National Corpus and the Oxford English Corpus. Measuring by number of words, the latter is around 20 times bigger. The strong point of the BNC is the attention given to the mixing proportions of various domains. The OEC, on the other hand, is larger and more recent. I have a feeling (but I cannot strongly affirm) that the differences in the following results arise from the inclusion in the OEC of blogs dating from the mid-2000s.

## Cognitive salience vs. social salience

One of the key ideas that motivate corpus approaches is the mismatch between these. The cognitive salience of something is the ease with which we can recall it. An example often used in language is the fixed expression kicking the bucket. It is one of the standard examples of fixed expressions that people give very often when asked. It is supposed to mean dying.

However, big surprise: the BNC has only 18 instances of this phrase, out of which only 3 are idiomatic, the rest being either literal or metalinguistic. This is a nice example of the salience contrast, but we mustn’t hurry to conclusions. The OEC has 193 examples (still few, relative to its size) but a lot more of them are idiomatic uses. To save the time I didn’t look at all the examples, but took a random sample of size 18, to compare the relative frequencies to BNC. Here, 15 out of 18 instances are idiomatic and none are meta. Quite a difference!

This goes to show the importance of context when we draw conclusions about language use. Now let’s tackle the list with a similar analysis.

## The idioms

1. Piece of cake

In BNC, this phrase occurs 51 times. 29 of these occurrences, however, the meaning is literal. In OEC we find 601 occurrences. In a random sample of size 51 we find 12 literal uses.

2. Costing an arm and a leg

For flexibility we search for the phrase an arm and a leg. In BNC it can be found 29 times: one literal, four with the verb to pay, and 16 with to cost. In OEC it appears 228 times. We take, again, a sample of size 29 and find no literal uses, 16 with to cost, four with to pay, three with to charge and a few different uses. The figurative meaning is the same in all cases: a lot of money.

3. Break a leg

BNC: 16, 13 of which are literal. OEC: 70 hits, 10/16 literal.

4. Hitting the books

BNC: 1 occurrence of hit the record books, which has a different meaning. The idiom is never used. OEC: 135, one of which literal.

5. Letting the cat out of the bag

We just looked for cooccurrences of cat in the context of the phrase out of the bag .
BNC: 19, out of which 3 metalinguistic/literal. OEC: 298, and out of a sample of 19, all were idiomatic.

6. Hitting the nail on the head

BNC: 12 instances, all idiomatic. OEC: 484, and out of a sample of 12 all were idiomatic.

7. When pigs fly

We looked for the lemma fly before the word pigs therefore catching multiple variations.
BNC: 17 hits, OEC: 240.

8. Judging a book by its cover

We looked for the fixed phrase book by its cover, because the leading verb might vary.
In the BNC, 11 instances (1 of them with tell instead of judge). In OEC, 195 instances. Sampling 11, all were idiomatic.

9. Biting off more than one can chew

BNC: 16 occurences, one of which with “to take” instead of “to bite”. OEC: 231, all idiomatic after sampling 16.

10. Scratching one’s back

BNC: 23, out of which only 5 idiomatic. OEC: 756, 5/23 idiomatic.

## Recalculating the rank

We now have enough data to reorder the expressions and compare. The result will be more approximate for the OEC because of our use of small subsamples to estimate the frequencies, but hopefully it will still be interesting. The way we are estimating the counts for the OEC is as follows: take, for instance, break a leg. It was found 70 times, and out of a sample of 16, 10 were literal. The expected number of idiomatic uses is therefore:

$$n = \left ( 1 – \frac{10}{16} \right ) \cdot 70 = 26.25$$

Repeating this computation and skipping a ton of steps leads to the following rankings:

In the British National Corpus:

1. Costing an arm and a leg
2. Piece of cake
3. When pigs fly
4. Letting the cat out of the bag
5. Biting off more than one can chew
6. Hitting the nail on the head
7. Judging a book by its cover
8. Scratching one’s back
9. Break a leg
10. Hitting the books
In the Oxford English Corpus:

1. Hitting the nail on the head
2. Piece of cake
3. Letting the cat out of the bag
4. When pigs fly
5. Biting off more than one can chew
6. Costing an arm and a leg
7. Judging a book by its cover
8. Scratching one’s back
9. Hitting the books
10. Break a leg

We can see that apart from the apparent switching of hitting the nail on the head with costing an arm and a leg, the rankings are not too different. We can quantify this by using the Rank Distance, a metric introduced by Liviu P. Dinu [3, 4]. Here, all our 3 rankings are over the same domain: we are not looking for the most frequent idioms in the corpora, this would be very hard. We are just reordering the proposed rank according to the occurrences in BNC and OEC. In this simple case, Rank Distance reduces to $$\ell_1$$ distance over rank position vectors. The weighted Rank Distance, bounded on $$[0, 1]$$ is in this case given by a scaling factor of $$0.5k^2$$ where k is the length of the rankings (10 in our case).

The computed distance between the original ranking and the BNC reordering is 0.52. Between the original and the OEC reordering, it is 0.68. Our two reorderings are much closer: the distance is 0.28. This is mostly because that the permutations between the two reorderings affect the top position, and are therefore weighted more.

It’s also interesting to look at the ratio of the counts. Interestingly, they approximately differ by a constant factor not far from the relative size difference of the two corpora, as would be expected.

We have to throw away hitting the books because its BNC zero count leads to divisions by zero. After this step, the average of the relative counts of the idioms is 19.5, with a standard deviation of 10.1, while OED is supposed to have around 20 times more words than the BNC.

## Conclusions

Well, it seems people don’t say break a leg and let’s hit the books as often as the original author claims. The popularity of most of the cited idioms seems supported by the data, but we have no easy way to find other idioms that might turn out to be much more frequent. Corpus linguistics is a reliable way to measure the social salience of language patterns It should always be used to verify and back empty claims of the form X is correct, Y is frequent or Nobody says Z.

[1] Patrick Hanks, How people use words to make meanings.
[2] Patrick Hanks, Lexical Analysis: Norms and Exploitations. The MIT Press (January 25, 2013)
[3] Liviu P. Dinu, Florin Manea. An efficient approach for the rank aggregation problem. In: Theoretical Computer Science, Volume 359 Issue 1, 14 August 2006. Pages 455 – 461.
[4] Liviu P. Dinu, On the Classification and Aggregation of Hierarchies with Different Constitutive Elements. Fundam. Inform. 55(1): 39-50 (2003)

# Scikit-learn-speed: An overview on the final day

Posted by & filed under benchmarking, python, scikit-learn.

This summer, I was granted the project called scikit-learn-speed, consisting of developing a benchmarking platform for scikit-learn and using it to find potential speedups, and in the end, make the library go faster wherever I can.

On the official closing day of this work, I’d like to take a moment and recall the accomplishments and failures of this project, and all the lessons to be learned.

## The scikit-learn-speed benchmark platform

Scikit-learn-speed is a continuous benchmark suite for the scikit-learn library. It has the following features:

• vbench-powered integration with Git
• Easily triggered build and report generation: just type make
• Easily readable and writeable template for benchmarks:
    {
'obj': 'LogisticRegression',
'init_params': {'C': 1e5},
'statements': ('fit', 'predict')
}, ...

• Many attributes recorded: time (w/ estimated standard deviation), memory usage, cProfiler output, line_profiler output, tracebacks
• Multi-step benchmarks: i.e. fit followed by predict

What were the lessons I learned here?

### Make your work reusable: the trade-off between good design and get-it-working-now

For the task of rolling out a continuous benchmarking platform, we decided pretty early in the project to adopt Wes McKinney’s vbench. If my goal would’ve been to maintain vbench and extend it into a multi-purpose, reusable benchmarking framework, the work would’ve been structured differently. It also would have been very open-ended and difficult to quantify.

The way things have been, I came up with features that we need in scikit-learn-speed, and tried to implement them in vbench without refactoring too much, but still by trying to make them as reusable as possible.

The result? I got all the features for scikit-learn-speed, but the implementation is not yet clean enough to be merged into vbench. This is fine for a project with a tight deadline such as this one: after it’s done, I will just spend another weekend on cleaning the work up and making sure it’s appreciated upstream. This will be easier because of the constraint to keep compatibility with scikit-learn-speed.

### Never work quietly (unless you’re a ninja)

I know some students who prefer that the professor doesn’t even know they exist until the final, when they would score an A, and (supposedly) leave the professor amazed. In real life, plenty of people would be interested in what you are doing, as long as they know about it. The PSF goes a long way to help this, with the “blog weekly” rule. In the end, however, it’s all up to you to make sure that everybody who should know finds out about your work. It will spare the world the duplicated work, the abandoned projects, but most importantly, those people could point you to things you have missed. Try to mingle in real-life as well, attend conferences, meetups, coding sprints.

I was able to slightly “join forces” with a couple of people who contacted me about my new vbench features (Hi Jon and Joel!), I have shaped my design slightly towards their requirements as well, and hopefully the result will be a more general vbench.

## The speedups

Once scikit-learn-speed was up and running, I couldn’t believe how useful it is to be able to scroll, catch slow code and jump straight at the profiler output with one click. I jumped on the following speed-ups:

• Multiple outputs in linear models. (PR)

Some of them proved trickier than expected, so I didn’t implement it for all the module yet, but it is ready for some estimators.

• Less callable functions passed around in FastICA (merged)
• Speed up euclidean_distances by rewriting in Cython. (PR)

This meant making more operations support an out argument, for passing preallocated memory. This touches many
different objects in the codebase: clustering, manifold learning, nearest neighbour methods.

• Insight into inverse and pseudoinverse computation, new pinvh function for inverting symmetric/hermitian matrices. (PR)

This speeds up the covariance module (especially MinCovDet), ARDRegression and the mixture models. It also lead to an upstream contribution to Scipy

• OrthogonalMatchingPursuit forward stepwise path for cross-validation (PR)

This is only halfway finished, but it will lead to faster and easier optimization of the OMP sparsity parameter.

Lessons? These will be pretty obvious.

### Write tests, tests, tests!

This is a no-brainer, but it still didn’t stick. In that one case out of 10 that I didn’t explicitly test, a bug was obviously hiding. When you want to add a new feature, it’s best to start by writing a failing test, and then making it pass. Sure, you will miss tricky bugs, but you will never have embarrassing, obvious bugs in your code

### Optimization doesn’t have to be ugly

Developers often shun optimization. It’s true, you should profile first, and you shouldn’t focus on speeding up stuff that is dominated by other computations that are orders of magnitude slower. However, there is an elephant in the room: the assumption that making code faster invariably makes it less clear, and takes a lot of effort.

The following code is a part of scipy’s pinv2 function as it currently is written:

    cutoff = cond*np.maximum.reduce(s)
psigma = np.zeros((m, n), t)
for i in range(len(s)):
if s[i] > cutoff:
psigma[i,i] = 1.0/np.conjugate(s[i])
return np.transpose(np.conjugate(np.dot(np.dot(u,psigma),vh)))

psigma is a diagonal matrix, and some time and memory can be saved with simple vectorization. However, this part of the code dominated by an above call to svd. The profiler output would say that we shouldn’t bother, but is it really a bother? Look at Jake’s new version:

    above_cutoff = (s > cond * np.max(s))
psigma_diag = np.zeros_like(s)
psigma_diag[above_cutoff] = 1.0 / s[above_cutoff]

return np.transpose(np.conjugate(np.dot(u * psigma_diag, vh)))

It’s shorter, more elegant, easier to read, and nevertheless faster. I would say it is worth it.

### Small speed-ups can propagate

Sure, it’s great if you can compute an inverse two times faster, say in 0.5s instead of 1s. But if some algorithm calls this function in a loop that might iterate 100, 300, or 1000 times, this small speed-up seems much more important, doesn’t it?

What I’m trying to say with this is that in a well-engineered system, a performance improvement to a relatively small component (such as the function that computes a pseudoinverse) can lead to multiple spread out improvements. Be careful of the double edge of this sword, a bug introduced in a small part can cause multiple failures downstream. But you are fully covered by your test suite, aren’t you?

Overall it has been a fruitful project that may have not resulted in a large number of speed-ups, but a few considerable ones nonetheless. And I venture the claim that the scikit-learn-speed tool will prove useful over time, and that the efforts deployed during this project have stretched beyond the boundary of the scikit-learn.

# Inverses and pseudoinverses. Numerical issues, speed, symmetry.

Posted by & filed under benchmarking, python.

The matrix inverse is a cornerstone of linear algebra, taught, along with its applications, since high school. The inverse of a matrix $$A$$, if it exists, is the matrix $$A^{-1}$$ such that $$AA^{-1} = A^{-1}A = I_n$$. Based on the requirement that the left and right multiplications should be equal, it follows that it only makes sense to speak of inverting square matrices. But just the square shape is not enough: for a matrix $$A$$ to have an inverse, $$A$$ must be full rank.

The inverse provides an elegant (on paper) method of finding solutions to systems of $$n$$ equations with $$n$$ unknowns, which correspond to solving $$Ax = b$$ for $$x$$. If we’re lucky and $$A^{-1}$$ exists, then we can find $$x = A^{-1}b$$. For this to work, it must be the case that:

• We have exactly as many unknowns as equations
• No equation is redundant, i.e. can be expressed as a linear combination of the others

In this setting, there is a unique solution for $$x$$.

## The Moore-Penrose pseudoinverse

What if we have more equations than unknowns? It is most likely the case that we cannot satisfy all the equations perfectly, so let’s settle for a solution that best fits the constraints, in the sense of minimising the sum of squared errors. We solve $$\operatorname{arg\,min}_x ||b – Ax||$$.

And how about the other extreme, where we have a lot of unknowns, but just a few equations constraining them. We will probably have an infinity of solutions, how can we choose one? A popular choice is to take the one of least $$\ell_2$$ norm: $$\operatorname{arg\,min}_x ||x|| \operatorname{s.t.} Ax = b$$. Is there a way to generalize the idea of a matrix inverse for this setting?

The pseudoinverse of an arbitrary-shaped matrix $$A$$, written $$A^{+}$$, has the same shape as $$A^{T}$$ and solves our problem: the answer to both optimization methods above is given by $$x = A^{+}y$$.

The theoretical definition of the pseudoinverse is given by the following conditions. The intuitive way to read them is as properties of $$AA^+$$ or $$A^+A$$:

• $$AA^+A = A$$
• $$A^+AA^+ = A^+$$
• $$(AA^+)^T = AA^+$$
• $$(A^+A)^T = A^+A$$

These conditions do not however give us a way to get our hands on a pseudoinverse, so we need something else.

## How to compute the pseudoinverse on paper

The first time I ran into the pseudoinverse, I didn’t even know its definition, only the expression of the closed-form solution of such a problem, and given as:

$$A^+ = (A^T A)^{-1}A^T$$

What can we see from this expression:

• It gives us a way to compute the pseudoinverse, and hence to solve the problem
• If $$A$$ is actually invertible, it means $$A^T$$ is invertible, so we have $$A^+ = A^{-1}(A^T)^{-1}A^T = A^{-1}$$
• Something bad happens if $$A^TA$$ is not invertible.

The pseudoinverse is still defined, and unique, when $$A^TA$$ is not invertible, but we cannot use the expression above to compute it.

## Numerical issues

Before going on, we should clarify and demystify some of the urban legends about numerical computation of least squares problems. You might have heard the following unwritten rules:

1. Never compute $$A^{-1}$$, solve the system directly
2. If you really need $$A^{-1}$$, use pinv and not inv

The first of these rules is based on some misguided beliefs, but is still good advice. If your goal is a one-shot answer to a system, there’s no use in explicitly computing a possibly large inverse, when all you need is $$x$$. But this paper shows that computing the inverse is not necessarily a bad thing. The key to this is conditional accuracy, and as long as the inv function used has good conditional bounds, you will get as good results as with a least squares solver.

The second rule comes from numerical stability, and will definitely bite you if misunderstood. If $$A$$ is a square matrix with a row full of zeros, it’s clearly not invertible, so an algorithm attempting to compute the inverse will fail and you will be able to catch that failure. But what if the row is not exactly zero, but the sum of several other rows, and a slight loss of precision is propagated at every step?

## Numerical rank vs. actual rank

The rank of a matrix $$A$$ is defined as the number of linearly independent rows (or equivalently, columns) in $$A$$. In other words, the number of non-redundant equations in the system. We’ve seen before that if the rank is less than the total number of rows, the system cannot have a unique solution anymore, so the matrix $$A$$ is not invertible.

The rank of a matrix is a computationally tricky problem. On paper, with small matrices, you would look at minors of decreasing size, until you find the first non-zero one. This is unfeasible to implement on a computer, so numerical analysis has a different approach. Enter the singular value decomposition!

The SVD of a matrix $$A$$ is $$A = USV^{T}$$, where $$S$$ is diagonal and $$U, V$$ are orthogonal. The elements on the diagonal of $$S$$ are called the singular values of $$A$$. It can be seen that to get a row full of zeros when multiplying three such matrices, a singular value needs to be exactly zero.

The ugly thing that could happen is that one (or usually more) singular values are not exactly zero, but very low values, due to propagated imprecision. Why is this a problem? By looking at the SVD and noting its properties, it becomes clear that $$A^{-1} = VS^{-1}U^{T}$$ and since $$S$$ is diagonal, its inverse is formed by taking the inverse of all the elements on the diagonal. But if a singular value is very small but not quite zero, its inverse is very large and it will blow up the whole computation of the inverse. The right thing to do here is either to tell the user that $$A$$ is numerically rank deficient, or to return a pseudoinverse instead. A pseudoinverse would mean: give up on trying to get $$AA^+$$ to be the identity matrix, simply aim for a diagonal matrix with approximately ones and zeroes. In other words, when singular values are very low, set them to 0.

How do you set the threshold? This is actually a delicate issue, being discussed on the numeric Python mailing list.

## Scipy implementations

Scipy exposes inv, pinv and pinv2. inv secretly invokes LAPACK, that ancient but crazy robust code that’s been used since the 70s, to first compute a pivoted LU decomposition that is then used to compute the inverse. pinv also uses LAPACK, but for computing the least-squares solution to the system $$AX = I$$. pinv2 computes the SVD and transposes everything like shown above. Both pinv and pinv2 expose cond and rcond arguments to handle the treatment of very small singular values, but (attention!) they behave differently!

The different implementations also lead to different speed. Let’s look at inverting a random square matrix:

In [1]: import numpy as np

In [2]: from scipy import linalg

In [3]: a = np.random.randn(1000, 1000)

In [4]: timeit linalg.inv(a)
10 loops, best of 3: 132 ms per loop

In [5]: timeit linalg.pinv(a)
1 loops, best of 3: 18.8 s per loop

In [6]: timeit linalg.pinv2(a)
1 loops, best of 3: 1.58 s per loop

Woah, huge difference! But do all three methods return the “right” result?

In [7]: linalg.inv(a)[:3, :3]
Out[7]:
array([[ 0.03636918,  0.01641725,  0.00736503],
[-0.04575771,  0.03578062,  0.02937733],
[ 0.00542367,  0.01246306,  0.0122156 ]])

In [8]: linalg.pinv(a)[:3, :3]
Out[8]:
array([[ 0.03636918,  0.01641725,  0.00736503],
[-0.04575771,  0.03578062,  0.02937733],
[ 0.00542367,  0.01246306,  0.0122156 ]])

In [9]: linalg.pinv2(a)[:3, :3]
Out[9]:
array([[ 0.03636918,  0.01641725,  0.00736503],
[-0.04575771,  0.03578062,  0.02937733],
[ 0.00542367,  0.01246306,  0.0122156 ]])

In [10]: np.testing.assert_array_almost_equal(linalg.inv(a), linalg.pinv(a))

In [11]: np.testing.assert_array_almost_equal(linalg.inv(a), linalg.pinv2(a))

Looks good! This is because we got lucky, though, and a was invertible to start with. Let’s look at its spectrum:

In [12]: _, s, _ = linalg.svd(a)

In [13]: np.min(s), np.max(s)
Out[13]: (0.029850235603382822, 62.949785645178906)

This is a lovely range for the singular values of a matrix, not too small, not too large. But what if we built the matrix in a way that would always pose problems? Specifically, let’s look at the case of covariance matrices:

In [14]: a = np.random.randn(1000, 50)

In [15]: a = np.dot(a, a.T)

In [16]: _, s, _ = linalg.svd(a)

In [17]: s[-9:]
Out[17]:
array([  7.40548924e-14,   6.48102455e-14,   5.75803505e-14,
5.44263048e-14,   4.51528730e-14,   3.55317976e-14,
2.46939141e-14,   1.54186776e-14,   5.08135874e-15])

a has at least 9 tiny singular values. Actually it’s easy to see why there are 950 of them:

In [18]: np.sum(s < 1e-10)
Out[18]: 950

How do our functions behave in this case? Instead of just looking at a corner, let’s use our gift of sight:

The small eigenvalues are large enough that inv thinks the matrix is full rank. pinv does better but it still fails, you can see a group of high-amplitude noisy columns. pinv2 is faster and it also gives us a useful result in this case.

Wait, does this mean that pinv2 is simply better, and pinv is useless?

Not quite. Remember, we are now trying to actually invert matrices, and degrade gracefully in case of rank deficiency. But what if we need the pseudoinverse to solve an actual non-square, wide or tall system?

In [19]: a = np.random.randn(1000, 50)

In [20]: timeit linalg.pinv(a)
10 loops, best of 3: 104 ms per loop

In [21]: timeit linalg.pinv(a.T)
100 loops, best of 3: 7.08 ms per loop

In [22]: timeit linalg.pinv2(a)
10 loops, best of 3: 114 ms per loop

In [23]: timeit linalg.pinv2(a.T)
10 loops, best of 3: 126 ms per loop

Huge victory for pinv in the wide case! Hurray! With all this insight, we can draw a line and see what we learned.

• If you are 100% sure that your matrix is invertible, use inv for a huge speed gain. The implementation of inv from Scipy is based on LAPACK’s *getrf + *getri, known to have good bounds.
• If you are trying to solve a tall or wide system, use pinv.
• If your matrix is square but might be rank deficient, use pinv2 for speed and numerical gain.

## Improving the symmetric case

But wait a second, can’t we do better? $$AA^T$$ is symmetric, can’t we make use of that to speed up the computation even more? Clearly, if $$A$$ is symmetric, in its SVD $$A = USV^T$$, we must have $$U = V$$. But this is exactly the eigendecomposition of a symmetric matrix $$A$$. The eigendecomposition can be computed cheaper than the SVD using Scipy eigh, that uses LAPACK’s *evr. As part of my GSoC this year, with help from Jake VanderPlas, we made a pull request to Scipy containing a pinvh function that is equivalent to pinv2 but faster for symmetric matrices.

In [24]: timeit linalg.pinv2(a)
1 loops, best of 3: 1.54 s per loop

In [25]: timeit linalg.pinvh(a)
1 loops, best of 3: 621 ms per loop

In [26]: np.testing.assert_array_almost_equal(linalg.pinv2(a), linalg.pinvh(a))

# The scikit-learn-speed ship has set sail! Faster than ever, with multi-step benchmarks!

Posted by & filed under benchmarking, python, scikit-learn.

I am pleased to announce that last night at 2:03 AM, the first fully automated run of the scikit-learn-speed test suite has run on our Jenkins instance! You can admire it at its temporary home for now. As soon as we verify that everything is good, we will move this to the official scikit-learn page.

I would like to take this opportunity to tell you about our latest changeset. We made running the benchmark suite tons simpler by adding a friendly Makefile. You can read more about its usage in the guide. But by far, our coolest new toy is:

## Multi-step benchmarks

A standard vbench benchmark has three units of code, represented as strings: code, setup and cleanup. With the original timeit-based benchmarks, this means that for every run, the setup would be executed once. Then, the main loop runs repeat times, and within each iteration, the code is run ncalls times. Then cleanup happens, the best time is returned, and everybody is happy.

In scikit-learn, most of our interesting objects go through a state change called fitting. This metaphor is right at home in the machine learning field, where we separate the learning phase for the prediction phase. The prediction step cannot be invoked on an object that hasn’t been fitted.

For some algorithms, one of these steps is trivial. A brute force Nearest Neighbors classifier can be instantaneously fit, but prediction takes a while. On the opposite end we have linear models, with tons of complicated algorithms to fit them, but evaluation is a simple matrix-vector product that Numpy handles perfectly.

But many of scikit-learn’s estimators have both steps interesting. Let’s take Non-negative Matrix Factorization. It has three interesting functions: The fit that computes $$X = WH$$, the transform that computes a non-negative projection on the components learned in fit, and fit_transform that takes advantage of the observation that when fitting, we also get the transformed $$X$$ for free.

When benchmarking NMF, we initially had to design 3 benchmarks:

• setup = standard, code = obj.fit(X)
• setup = standard, code = obj.fit_transform(X)
• setup = standard + obj.fit(X), code = obj.transform(X)

## How much time were we wasting?

Let’s say it takes 10 seconds. For every benchmark, we time the code by running it 3 times. We run it once more to measure memory usage, once more for cProfile and one last time for line_profiler. This is a total of 6 times per benchmark. We need to multiply this by 2 again for running on two datasets. So when benchmarking NMF, because we need to fit before predicting, we do it 12 extra times. If a fit takes 5 seconds, this means one minute wasted on benchmarking just one estimator. Wouldn’t it be nice to fit, fit_transform and transform in a sequence?

## Behind the scenes

We made the PythonBenchmark code parameter also support getting a sequence of strings, instead of just a string. On the database side, every benchmark result entry gets an extra component in the primary key, the number of the step it measures.

In the benchmark description files, nothing is changed:

{
'obj': 'NMF',
'init_params': {'n_components': 2},
'datasets': ('blobs',),
'statements': ('fit_unsup', 'transform_unsup', 'fit_transform')
},

But before, we would take the cartesian product of datasets and statements, and build a Benchmark object for every pairing. Now, we just pass the tuple as it is, and vbench is smart enough to do the right thing.
We avoided the extra calls to fit in a lot of benchmarks. The whole suite now takes almost half the time to run!

Note: This trick is currently hosted in the abstract_multistep_benchmarks vbench branch in my fork.

# Profiler output, benchmark standard deviation and other goodies in scikit-learn-speed

Posted by & filed under benchmarking, python, scikit-learn.

This post is about the scikit-learn benchmarking project that I am working on, called scikit-learn-speed. This is a continuous benchmarking suite that runs and generates HTML reports using Wes McKinney’s vbench framework, to which I had to make some (useful, I hope) additions.

## What it looks like now

You can check out a teaser/demo that was run on equidistant releases from the last two months. What has changed since the last version? Here’s a list in order of obviousness:

• We now use the lovely scikit-learn theme
• Timing graphs now show the ±1 standard deviation range
• cProfile output is displayed for all the benchmarks, so we can easily see at a glance what’s up
• Said profiler output is collapsible using JQueryUI goodness
• There now is an improved Quick Start guide to running vbench on your machine

## What made this possible

I have done some more refactoring in my vbench fork, because I didn’t want to have a huge, monolithic Benchmark class that was specific to what we want in scikit-learn-speed. So on this branch, I set up a mixin/multiple inheritance hierarchy of benchmark classes.

The Benchmark class in vbench is now an abstract base class, with some common functionality and structure.
Our SklBenchmark class is defined in scikit-learn-speed as:

class SklBenchmark(CProfileBenchmarkMixin, MemoryBenchmarkMixin, PythonBenchmark):

Let’s read this from right to left:

• PythonBenchmark: This class stores code, setup and cleanup Python code as strings, and implements simple timing mechanisms using the time module.
• Bonus: TimeitBenchmark: This class extends PythonBenchmark with the timeit micro-benchmark timing method previously used in vbench. We turned this off in scikit-learn-speed.
• MemoryBenchmarkMixin: This adds memory benchmarking using memory_profiler.
• CProfileBenchmarkMixin: This runs the code through cProfile and implements mechanisms to report the output.

The database is not flexible enough to adapt to arbitrary benchmark structure right now, so if anybody would like to help the effort, it would be very appreciated.

# Scikit-learn-speed HTML reports teaser

Posted by & filed under benchmarking, python, scikit-learn.

EDIT: I made the plots a little more readable, check it out!

Last time, I teased you with a screenshot of local output. Now, I will tease you with the benchmarks run on a couple of recent commits, along with some from earlier this year.

After some effort and bugfixes, the project now reliably runs on different machines, so the next step to host it on a remote server and invoke it daily is getting closer. In the mean time, you can have a look at the sample output.

Note that just last time, the plots look jagged but the differences are mostly minor and significant conclusions cannot be drawn yet, but as the suite will start running daily, the plots will become much more meaningful. I could waste time running the suite on more previous commits, but the results wouldn’t be comparable with the ones from the deployed system, because of hardware differences.

Playing around with this makes me want a couple of features in vbench. One is the possibility to overlay related benchmarks on the same plot (for example, different parameters for the same algorithm and data): this could be useful to spot patterns. A second one is some query / sorting support: see what are the most expensive benchmarks, see what benchmarks show the biggest jump in performance (but this could become a historical wall of fame or shame).

# Memory benchmarking with vbench

Posted by & filed under benchmarking, python, scikit-learn.

The scikit-learn-speed project now has memory usage benchmarking!

This was accomplished by building on what I described in my recent posts, specifically the extensions to Fabian’s memory_profiler that you can find in my fork, but they will be merged upstream soon. The key element is the %magic_memit function whose development I blogged about on several occasions. I plugged this into vbench in a similar way to how the timings are computed, all with great success.

Here is a screenshot of the way a simple benchmark looks now, with just a few data points.

Memory benchmarking in scikit-learn-speed powered by vbench.

You can check it out and use it yourself for your benchmarks, but you need to use the vbench from the memory branch on my fork.

Of course, there are some important caveats. I am running this on my laptop, which runs OS X Lion, so, under the effect of this bug, I hardcoded the ‘-i‘ so the memory benchmarks are not realistic. Also, the y-range should probably be forced wider, because the plots look erratic, showing the very small noise at a large-scale.

# On why my %memit fails on OSX

Posted by & filed under benchmarking, python.

In my last post I mentioned that I’m not satisfied with the current state of %memit, because some more complicated numerical function calls make it crash. I will start this post with a reminder of a pretty important bug:

On MacOS X (10.7 but maybe more), after forking a new process, there is a segfault in Grand Central Dispatch on the BLAS DGEMM function from Accelerate.

EDIT 1: In a hurry, I forgot to mention how Olivier Grisel and David Cournapeau spent some time narrowing down this issue, starting from an odd testing bug in scikit-learn. They reported it to Apple, but there was, as of the date of this post, no reaction.

EDIT 2: MinRK confirms, and I verified shortly after, that this bug is fixed in Mountain Lion (10.8). Still not sure how far back it goes, though, so feedback is welcome.

When I first tried to make the %memit magic, I thought about simply measuring the current memory, running the command, and measuring the memory again. The problem is the results are not consistent, because Python tries to reuse already allocated memory whenever it can.

Using memory_profiler, here’s an example illustrating this elastic memory management:

# mem_test.py
import numpy as np

def make_a_large_array():
return np.ones((1000, 1000))

def main():
make_a_large_array()
make_a_large_array()
make_a_large_array()

# in IPython:
In [1]: import mem_test

In [2]: %mprun -f mem_test.main mem_test.main()
Filename: mem_test.py

Line #    Mem usage  Increment   Line Contents
==============================================
8   24.8477 MB  0.0000 MB   def main():
9   24.8633 MB  0.0156 MB       make_a_large_array()
10   32.4688 MB  7.6055 MB       make_a_large_array()
11   32.4688 MB  0.0000 MB       make_a_large_array()

If this was in an IPython environment, and one would like to see how much memory make_a_large_array() uses, you could say we can simply run it a few times and take the maximum. However, if you happened to accidentally call main() once before, you will no longer get a good result:

In [3]: %mprun -f mem_test.main mem_test.main()
Filename: mem_test.py

Line #    Mem usage  Increment   Line Contents
==============================================
8   32.4922 MB  0.0000 MB   def main():
9   32.5234 MB  0.0312 MB       make_a_large_array()
10   32.5234 MB  0.0000 MB       make_a_large_array()
11   32.5234 MB  0.0000 MB       make_a_large_array()

So how can we get consistent results for the memory usage of an instruction? We could run it in a fresh, new process. I implemented this in %memit and it shows:

In [5]: %memit mem_test.make_a_large_array()
maximum of 3: 8.039062 MB per loop

In [6]: %memit mem_test.make_a_large_array()
maximum of 3: 8.035156 MB per loop

In [7]: %memit mem_test.make_a_large_array()
maximum of 3: 8.042969 MB per loop

This way you can also realistically benchmark assignments:

In [8]: %memit X = mem_test.make_a_large_array()
maximum of 3: 8.054688 MB per loop

In [9]: %memit X = mem_test.make_a_large_array()
maximum of 3: 8.058594 MB per loop

In [10]: %memit X = mem_test.make_a_large_array()
maximum of 3: 8.058594 MB per loop

If we don’t spawn a subprocess, del doesn’t help, but allocating new variables does:

In [11]: %memit -i X = mem_test.make_a_large_array()
maximum of 3: 7.632812 MB per loop

In [12]: del X

In [13]: %memit -i X = mem_test.make_a_large_array()
maximum of 3: 0.000000 MB per loop

In [14]: %memit -i Y = mem_test.make_a_large_array()
maximum of 3: 7.632812 MB per loop

In [15]: %memit -i Z = mem_test.make_a_large_array()
maximum of 3: 7.632812 MB per loop

Now, the problem is that when the function that you are benchmarking contains calls to np.dot (matrix multiplication), the subprocess will consistently fail with SIGSEGV on affected OS X systems. These are actually pretty much all the functions that I intended %memit for: numerical applications. For that reason, I have made %memit notify the user when all subprocesses fail, and to suggest the usage of the -i flag.

I think that, with this update, %memit is flexible and usable enough for actual use, and therefore for merging into memory_profiler.

# More on memory benchmarking

Posted by & filed under benchmarking, python.

Following up on my task to make it easier to benchmark memory usage in Python, I updated Fabian’s memory_profiler to include a couple of useful IPython magics. While in my last post, I used the new IPython 0.13 syntax for defining magics, this time I used the backwards-compatible one from the previous version.

You can find this work-in-progress as a pull request on memory_profiler from where you can trace it to my GitHub repo. Here’s what you can do with it:

## %mprun

Copying the spirit of %lprun, since imitation is the most sincere form of flattery, you can use %mprun to easily view line-by-line memory usage reports, without having to go in and add the @profile decorator.

For example:

In [1]: import numpy as np

In [2]: from sklearn.linear_model import ridge_regression

In [3]: X, y = np.array([[1, 2], [3, 4], [5, 6]]), np.array([2, 4, 6])

In [4]: %mprun -f ridge_regression ridge_regression(X, y, 1.0)

(...)

109   41.6406 MB  0.0000 MB           if n_features > n_samples or \
110   41.6406 MB  0.0000 MB              isinstance(sample_weight, np.ndarray) or \
111   41.6406 MB  0.0000 MB              sample_weight != 1.0:
112
113                                       # kernel ridge
114                                       # w = X.T * inv(X X^t + alpha*Id) y
115                                       A = np.dot(X, X.T)
116                                       A.flat[::n_samples + 1] += alpha * sample_weight
117                                       coef = np.dot(X.T, _solve(A, y, solver, tol))
118                                   else:
119                                       # ridge
120                                       # w = inv(X^t X + alpha*Id) * X.T y
121   41.6484 MB  0.0078 MB               A = np.dot(X.T, X)
122   41.6875 MB  0.0391 MB               A.flat[::n_features + 1] += alpha
123   41.7344 MB  0.0469 MB               coef = _solve(A, np.dot(X.T, y), solver, tol)
124
125   41.7344 MB  0.0000 MB       return coef.T

## %memit

As described in my previous post, this is a %timeit-like magic for quickly seeing how much memory a Python command uses.
Unlike %timeit, however, the command needs to be executed in a fresh process. I have to dig in some more to debug this, but if the command is run in the current process, very often the difference in memory usage will be insignificant, I assume because preallocated memory is used. The problem is that when running in a new process, some functions that I tried to bench crash with SIGSEGV. For a lot of stuff, though, %memit is currently usable:

In [1]: import numpy as np

In [2]: X = np.ones((1000, 1000))

In [3]: %memit X.T
worst of 3: 0.242188 MB per loop

In [4]: %memit np.asfortranarray(X)
worst of 3: 15.687500 MB per loop

In [5]: Y = X.copy('F')

In [6]: %memit np.asfortranarray(Y)
worst of 3: 0.324219 MB per loop

It is very easy, using this small tool, to see what forces memory copying and what does not.

## Installation instructions

First, you have to get the source code of this version of memory_profiler. Then, it depends on your version of IPython. If you have 0.10, you have to edit ~/.ipython/ipy_user_conf.py like this: (once again, instructions borrowed from line_profiler)

# These two lines are standard and probably already there.
import IPython.ipapi
ip = IPython.ipapi.get()

# These two are the important ones.
import memory_profiler
ip.expose_magic('mprun', memory_profiler.magic_mprun)
ip.expose_magic('memit', memory_profiler.magic_memit)

If you’re using IPython 0.11 or newer, the steps are different. First create a configuration profile:

\$ ipython profile create

Then create a file named ~/.ipython/extensions/memory_profiler_ext.py with the following content:

import memory_profiler

ip.define_magic('mprun', memory_profiler.magic_mprun)
ip.define_magic('memit', memory_profiler.magic_memit)

Then register it in ~/.ipython/profile_default/ipython_config.py, like this. Of course, if you already have other extensions such as line_profiler_ext, just add the new one to the list.

c.TerminalIPythonApp.extensions = [
'memory_profiler_ext',
]
c.InteractiveShellApp.extensions = [
'memory_profiler_ext',
]

Now launch IPython and you can use the new magics like in the examples above.