text:
PaO2 Arterial oxygen tension, or partial pressure Alveolar oxygen tension, or partial pressure Arterial carbon dioxide tension, or partial pressure Alveolar carbon dioxide tension, or partial pressure Oxygen tension of mixed venous blood Alveolar-arterial oxygen tension difference. The term formerly used (A-a D.mw-parser-output .template-chem2-su{display:inline-block;font-size:80%;line-height:1;vertical-align:-0.35em}.mw-parser-output .template-chem2-su>span{display:block}.mw-parser-output sub.template-chem2-sub{font-size:80%;vertical-align:-0.35em}.mw-parser-output sup.template-chem2-sup{font-size:80%;vertical-align:0.65em}O2) is discouraged. Alveolar-arterial tension ratio; PaO2:PAO2 The term oxygen exchange index describes this ratio. Arteriovenous oxygen content difference Oxygen saturation of the hemoglobin of arterial blood Oxygen saturation as measured by pulse oximetry Oxygen content of arterial blood Symbol relating the hydrogen ion concentration or activity of a solution to that of a standard solution; approximately equal to the negative logarithm of the hydrogen ion concentration. pH is an indicator of the relative acidity or alkalinity of a solution .mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}

Dead space is the volume of air that is inhaled that does not take part in the gas exchange, because it either remains in the conducting airways or reaches alveoli that are not perfused or poorly perfused. It means that not all the air in each breath is available for the exchange of oxygen and carbon dioxide. Mammals breathe in and out of their lungs, wasting that part of the inhalation which remains in the conducting airways where no gas exchange can occur.

## Components

Benefits do accrue to a seemingly wasteful design for ventilation that includes dead space.[1]

1. Carbon dioxide is retained, making a bicarbonate-buffered blood and interstitium possible.
2. Inspired air is brought to body temperature, increasing the affinity of hemoglobin for oxygen, improving O2 uptake.[2]
3. Particulate matter is trapped on the mucus that lines the conducting airways, allowing its removal by mucociliary transport.
4. Inspired air is humidified, improving the quality of airway mucus.[2]

In humans, about a third of every resting breath has no change in O2 and CO2 levels. In adults, it is usually in the range of 150 mL.[3]

Dead space can be increased (and better envisioned) by breathing through a long tube, such as a snorkel. Although one end of the snorkel is open to the air, when the wearer breathes in, they inhale a significant quantity of air that remained in the snorkel from the previous exhalation. Therefore, a snorkel increases the person's dead space by adding even more airway that does not participate in gas exchange.

Anatomical dead space is the volume of the conducting airways (from the nose, mouth and trachea to the terminal bronchioles). These conduct gas to the alveoli but no gas exchange occurs here. In healthy lungs where the alveolar dead space is small, Fowler's method accurately measures the anatomic dead space using a single breath nitrogen washout technique.[4][5]

The normal value for dead space volume (in mL) is approximately the lean mass of the body (in pounds), and averages about a third of the resting tidal volume (450-500 mL). In Fowler's original study, the anatomic dead space was 156 ± 28 mL (n=45 males) or 26% of their tidal volume.[4] Despite the flexibility of the trachea and smaller conducting airways, their overall volume (i.e. the anatomic dead space) changes little with bronchoconstriction or when breathing hard during exercise.[4][6]

As birds have a longer and wider trachea than mammals the same size, they have a disproportionately large anatomic dead space, reducing the airway resistance. This adaptation does not impact gas exchange because birds flow air through their lungs - they do not breathe in and out like mammals.[7]

Alveolar dead space is defined as the difference between the physiologic dead space and the anatomic dead space. It is contributed to by all the terminal respiratory units that are over-ventilated relative to their perfusion. Therefore it includes, firstly those units that are ventilated but not perfused, and secondly those units which have a ventilation-perfusion ratio greater than one.

Alveolar dead space is negligible in healthy individuals, but it can increase dramatically in some lung diseases due to ventilation-perfusion mismatch.

## Calculating

Just as dead space wastes a fraction of the inhaled breath, dead space dilutes alveolar air during exhalation. By quantifying this dilution, it is possible to measure physiological dead space, employing the concept of mass balance, as expressed by Bohr equation.[8][9]

${\displaystyle {\frac {V_{d}}{V_{t}}}={\frac {P_{a\,{\ce {CO2}}}-P_{e\,{\ce {CO2}}}}{P_{a\,{\ce {CO2}}}}}}$
where ${\displaystyle V_{d}}$ is the dead space volume and ${\displaystyle V_{t}}$ is the tidal volume;
${\displaystyle P_{a\,{\ce {CO2}}}}$ is the partial pressure of carbon dioxide in the arterial blood, and
${\displaystyle P_{e\,{\ce {CO2}}}}$ is the partial pressure of carbon dioxide in the mixed expired (exhaled) air.

The Bohr equation is used to measure physiological dead space. Unfortunately, the concentration of carbon dioxide (CO2) in alveoli is required to use the equation but this is not a single value as the ventilation-perfusion ratio is different in different lung units both in health and in disease. In practice, the arterial partial pressure of CO2 is used as an estimate of the average alveolar partial pressure of CO2, a modification introduced by Henrik Enghoff in 1938 (Enghoff H. Volumen inefficax. Bemerkungen zur Frage des schadlichen Raumes. Upsala Läkarefören Forhandl., 44:191-218, 1938). In effect, the single arterial pCO2 value averages out the different pCO2 values in the different alveoli, and so makes the Bohr equation useable.

The quantity of CO2 exhaled from the healthy alveoli is diluted by the air in the conducting airways (anatomic dead space) and by gas from alveoli that are over-ventilated in relation to their perfusion. This dilution factor can be calculated once the mixed expired pCO2 in the exhaled breath is determined (either by electronically monitoring the exhaled breath or by collecting the exhaled breath in a gas impermeant bag (a Douglas bag) and then measuring the pCO2 of the mixed expired gas in the collection bag). Algebraically, this dilution factor will give us the physiological dead space as calculated by the Bohr equation:

${\displaystyle {\frac {V_{\ce {physiological\,dead\,space}}}{V_{t}}}={\frac {P_{a\,{\ce {CO2}}}-P_{\ce {mixed\,expired\,CO2}}}{P_{a\,{\ce {CO2}}}}}}$

The alveolar dead space is determined as the difference between the physiological dead space (measured using the Enghoff modification of the Bohr equation) and the anatomic dead space (measured using Fowler's single breath technique).

A clinical index of the size of the alveolar dead space is the difference between the arterial partial pressure of CO2 and the end-tidal partial pressure of CO2.

A different maneuver is employed in measuring anatomic dead space: the test subject breathes all the way out, inhales deeply from a 0% nitrogen gas mixture (usually 100% oxygen) and then breathes out into equipment that measures nitrogen and gas volume. This final exhalation occurs in three phases. The first phase (phase 1) has no nitrogen as that is gas that is 100% oxygen in the anatomic dead space. The nitrogen concentration then rapidly increases during the brief second phase (phase 2) and finally reaches a plateau in the third phase (phase 3). The anatomic dead space is equal to the volume exhaled during the first phase plus the volume up to the mid-point of the transition from phase 1 to phase 3.

## Ventilated patient

The depth and frequency of our breathing is determined by chemoreceptors and the brainstem, as modified by a number of subjective sensations. When mechanically ventilated using a mandatory mode, the patient breathes at a rate and tidal volume that is dictated by the machine. Because of dead space, taking deep breaths more slowly (e.g. ten 500 ml breaths per minute) is more effective than taking shallow breaths quickly (e.g. twenty 250 ml breaths per minute). Although the amount of gas per minute is the same (5 L/min), a large proportion of the shallow breaths is dead space, and does not allow oxygen to get into the blood.

Mechanical dead space is dead space in an apparatus in which the breathing gas must flow in both directions as the user breathes in and out, increasing the necessary respiratory effort to get the same amount of usable air or breathing gas, and risking accumulation of carbon dioxide from shallow breaths. It is in effect an external extension of the physiological dead space.

It can be reduced by:

• Using separate intake and exhaust passages with one-way valves placed in the mouthpiece. This limits the dead space to between the non return valves and the user's mouth or nose. The additional dead space can be minimized by keeping the volume of this external dead space as small as possible, but this should not unduly increase work of breathing.
• With a full face mask or demand diving helmet:
• Keeping the inside volume small
• Having a small internal orinasal mask inside the main mask, which separates the external respiratory passage from the rest of the mask interior.
• In a few models of full face mask a mouthpiece like those used on diving regulators is fitted, which has the same function as an orinasal mask, but can further reduce the volume of the external dead space, at the cost of forcing mouth-breathing (and acting like a gag, preventing clear talking).

## References

1. ^ West, John B. (2011). Respiratory physiology : the essentials (9th ed.). Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins. ISBN 978-1-60913-640-6.
2. ^ a b Williams, R; Rankin, N; Smith, T; Galler, D; Seakins, P (November 1996). "Relationship between the humidity and temperature of inspired gas and the function of the airway mucosa". Critical Care Medicine. 24 (11): 1920–9. doi:10.1097/00003246-199611000-00025. PMID 8917046.
3. ^ "Wasted Ventilation". Ccmtutorials.com. Retrieved 2013-11-27.
4. ^ a b c Fowler W.S. (1948). "Lung Function studies. II. The respiratory dead space". Am. J. Physiol. 154 (3): 405–416. doi:10.1152/ajplegacy.1948.154.3.405. PMID 18101134.
5. ^ Heller H, Könen-Bergmann M, Schuster K (1999). "An algebraic solution to dead space determination according to Fowler's graphical method". Comput Biomed Res. 32 (2): 161–7. doi:10.1006/cbmr.1998.1504. PMID 10337497.
6. ^ Burke, TV; Küng, M; Burki, NK (1989). "Pulmonary gas exchange during histamine-induced bronchoconstriction in asthmatic subjects". Chest. 96 (4): 752–6. doi:10.1378/chest.96.4.752. PMID 2791669.
7. ^ West, JB (2009). "Comparative physiology of the pulmonary blood-gas barrier: the unique avian solution". American Journal of Physiology. Regulatory, Integrative and Comparative Physiology. 297 (6): R1625-34. doi:10.1152/ajpregu.00459.2009. PMC 2803621. PMID 19793953.
8. ^ Bohr, C. (1891). Über die Lungenathmung. Skand. Arch. Physiol. 2: 236-268.
9. ^ Klocke R (2006). "Dead space: simplicity to complexity". J Appl Physiol. 100 (1): 1–2. doi:10.1152/classicessays.00037.2005. PMID 16357075. article


text:
Two Columns whose content is positioned relative to each other

In theatre (and film) there's a tendency for actors to let each other finish what they're saying, possibly even making a small pause before starting their turn. But that is not how real life conversation work and it leads to a particular stiff "theatre tone".

To counter that I'm letting my actors practise speaking at the same time (intentionally, that is, in real life they're doing it all the time). And in order to help them gain an idea of what the result is supposed to be like, I want to give them their text already in a format that visualises this.

I made a text/plain example for what I need:

ALICE: I‘m saying something in
this mock drama of ours, it is      BOB: Yes.
supposed to be a little             CHARLES: No doubt.
monologue of medium length. I‘m
still supposed to be talking,       BOB: I can see that.
because our author needs an
example for you trying to
interrupt me. But, while I‘m
certainly perceiving what you‘re
saying, I‘m not yet letting you
take what linguists call your       BOB: I want to ...
„turn“, for I really want to
finish my thought.                  BOB: I want to say something right
now. I have to admit, I don‘t really
CHARLES: I feel that, too ...       have something to say. But I would
feel left out, if I did not get my
ALICE: Of course.                   turn. I would feel that I‘d be
vanishing in this piece, if I
wouldn‘t say something. I feel that
my sole purpose is to have a little
CHARLES: That‘s ...                 speech right here. And although I
rather feel compelled to talk right
CHARLES: That‘s great, because      now, I also derive some strange kind
it expresses exactly my             of joy from it.
sentiments also. You‘re really
giving voice to ...                 ALICE: Of course, Bob, and now we
all had our turn, except Charles,
whom I‘m interrupting right now, but
that is intentional. And now we
can finish this little farce,
right? I mean, like: Stop it
right now!

So, basically, I need two columns. The content of each column needs to be positioned relative to the text in the other column. In the example, when Bob says "Yes." in the right column, this should occur on the same line as the word "drama" in Alice's speech in the left column. And the other way around: When Charles says "I feel that, too ..." in the left column, this should occur after Bob's "I don't really have" in the right column.

I guess, that, provided it is feasible at all, this is rather non-trivial to achieve in LaTeX. Basically, I need a solution right now, but also a possibly more thorough one for future projects. I have a nagging suspicion that for right now I'm better off with Inkscape or Scribus. I'd love to be wrong about this. For the future, I'd like to find a solution with LaTeX or TeX. I have little to no experience with LaTeX hacking, but I'm confident that I can get into it, given enough time. Pointers on where to look and tips as for the general logic on how to achieve it could greatly facilitate it. Also, I'm an org-mode user and feel quite at home with Emacs-Lisp hacking. So, one possibility would be to write an emacs interface and produce TeX code programmatically.

A possible LaTeX interface could, for instance, include a command \turn{LABEL} to indicate the start of a new turn in the other column and \turnlabel{LABEL} to indicate where it should be vertically positioned in relation to the previous turn. The interface could, again just for instance, include a command \interruption for interruptions that don't constitute a turn. For instance:

%% mock LaTeX

\turn{Alice}

ALICE: I‘m saying something in this mock drama
\interruption{BOB: Yes.} \interruption{CHARLES: No doubt.}
of ours, it is supposed to be a little monologue of medium
length. I‘m still supposed to be talking \interruption{BOB:
I can see that.}, because our author needs an example for
you trying to interrupt me. But, while I‘m certainly
perceiving what you‘re saying, I‘m not yet letting you take
what linguists \interruption{BOB: I want to ...} call your
„turn“, for I really want to finish \turnlabel{Bob} my
thought.

\turn{Bob}
BOB: I want to say something right now. [...]

• Try »paracol«, »parallel« or »parcolumns«. – Thorsten Donig Sep 10 '13 at 10:17
• could you edit your question to include that text as text rather than as a screen dump. – David Carlisle Sep 10 '13 at 10:49
• @DavidCarlisle: I'd love to; in fact, I tried originally both with a <pre> tag and; but the text/plain example is based on absolute positioning of the right column based on characer count. In the preview, this appeared ragged, and because of the amount of text, completely messy. – Oliver Scholz Sep 10 '13 at 10:59
• @ThorstenDonig Great! Thank you very much, this might be a starting point. – Oliver Scholz Sep 10 '13 at 10:59
• You could use the code button {} (or simply indent by 4 spaces) to lay out the text exactly – David Carlisle Sep 10 '13 at 11:20

This isn't too smart about multi-line interruptions near a page break, or interruptions too close together.

Commented out you will see some code that records the vertical start and end positions in a .pos file at the end of the run, this could be used on a later pass to automatically move things around but that gets tricky and unstable, and possibly for a play you ultimately want manual control anyway, so this uses a simpler scheme that \turn and \interruption take an optional first parameter than shifts the text by that number of baselineskip.

\documentclass[a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\makeatletter

\setlength\columnsep{20pt}

\newwrite\iterpos

%\immediate\openout\iterpos=\jobname.pos

\raggedbottom
\parskip\z@

\newcount\turncnt
\newcount\intercnt

\def\dialog{%
\par
\global\turncnt\z@
\linewidth=\dimexpr(\textwidth-\columnsep)/2\relax
\hsize\linewidth}

\def\enddialog{\par}

\newcommand\turn[2][0]{%
\setbox0\hbox{\textbf{#2: }}%
\everypar{\setbox2\lastbox\box0\everypar{}}
\par
\ifnum#1=\z@\else\vskip#1\baselineskip\fi
\ifodd\turncnt
\hsize\linewidth
\leftskip\z@
\else
\hsize\dimexpr2\linewidth+\columnsep\relax
\leftskip\dimexpr\linewidth+\columnsep\relax
\fi
\ignorespaces}

\newcommand\interruption[3][0]{%
\@bsphack
\ifodd\turncnt
\hspace*{\dimexpr\linewidth+\columnsep\relax}%
\else
\hspace*{\dimexpr-\linewidth-\columnsep\relax}%
\fi
\raisebox{\dimexpr\dp\strutbox
\ifnum#1=\z@\else-(#1\baselineskip)\fi
\relax}[\z@][\z@]{\parbox[t]{\linewidth}{%
%\pdfsavepos
%\edef\tmp{\write\iterpos{\string\def\string\intterstart\romannumeral\intercnt{\noexpand\the\pdflastypos}}}\tmp
\textbf{#2: }#3%
\ifhmode\unskip\fi
%\pdfsavepos
%\edef\tmp{\write\iterpos{\string\def\string\intterend\romannumeral\intercnt{\noexpand\the\pdflastypos}}}\tmp
}}}%
\@esphack}

\makeatother

\begin{document}
\begin{dialog}
\turn{ALICE}I‘m saying something in
this mock drama
\interruption{BOB}{Yes.} \interruption[1]{CHARLES}{No doubt.}
of ours, it is supposed to be a little
monologue of medium length. I‘m
still supposed to be talking\interruption{BOB}{I can see that.},
because our author needs an
example for you trying to
interrupt me. But, while I‘m
certainly perceiving what you‘re
saying, I‘m not yet letting you
take what linguists\interruption{BOB}{I want to ...} call your
„turn“, for I really want to
finish my thought.                  \turn[-2]{BOB}

I want to say something right
now. I have to admit, I don‘t really
have something to say.\interruption{CHARLES}{ feel that, too ...} But I would
feel left out, if I did not get my
turn.\interruption{ALICE}{Of course.} I would feel that I‘d be
vanishing in this piece, if I
wouldn‘t say something. I feel that
my sole purpose is to have a little
speech right here.\interruption{CHARLES}{That‘s ...} And although I
rather feel compelled to talk right
now, I also derive some strange kind
of joy from it.
\turn[-2]{CHARLES}
That‘s great, because
it expresses exactly my
sentiments also. You‘re really
giving voice to ...               \turn{ALICE}

Of course, Bob, and now we
all had our turn, except Charles,
whom I‘m interrupting right now, but
that is intentional. And now we
can finish this little farce,
right? I mean, like: Stop it
right now!

\end{dialog}
\end{document}

• I'm speechless. Thank you very much! Yes, for a play I need manual control anyways, since the exact positioning of cues is part of the creative work. This definitely gets the job done; and it's a HUGE help for me. Again: Thank you! – Oliver Scholz Sep 11 '13 at 10:36

format: plain


text:
# It Just Keeps Piling Up!

We'll start off by looking at three basic types of deposition. Much (but not everything) is known about them, and together they cover a wide range of real world phenomena...

Bill Casselman
University of British Columbia, Vancouver, Canada
Email Bill Casselman

### Introduction

Stuff keeps falling all the time. Rain falls everywhere, snow falls on the ground, snow falls on snow, river sediments fall on the ocean floor, and in industrial work all kinds of materials are laid down in a somewhat random manner on all kinds of other materials. The technical phrase is that stuff is deposited on other stuff, and the process involved is deposition. It is important to understand different deposition processes in nature and in industry. Of course real-world deposition can be very complicated, so mathematicians look at simplified (often extremely simplified) models. For example, in real life most depositions take place onto a surface, whereas in most mathematical models they take place onto a line, and already for these very simplified models what happens is often not easy to understand.

The effect of rainfall is very simple. Water is a liquid of low viscosity, which means that it flows freely. In practice, the effect is just to form a level surface of puddles. But other kinds of deposition exhibit some kind of stickiness, and it is this that makes things difficult.

We'll start off by looking at three basic types of deposition. Much (but not everything) is known about them, and together they cover a wide range of real world phenomena.

### What are the basic types?

The three basic types of deposition we'll look at will have some features in common. First of all, the deposition will be onto a one-dimensional region of width $W$. At various times, a small square will appear somewhere in the sky above the region and fall onto whatever edge has already formed. What happens at that point depends on which type of process we are looking at. In each case, we ask, how do the stacks of squares grow?

We are looking at a finite region. Without some special assumption, something special can be expected at the boundary. To eliminate these effects, I'll assume the boundary to be circular, or in other words the process is periodic. We'll see what the consequences are in just a moment.

The time scale is normalized so that $W$ squares are dropped in $1$ unit of time. In $t$ units of time, the number of squares dropped is $Wt$.

Random deposition

The simplest of all deposition processes is that in which the unit square just drops into place at the lowest unoccupied point. This is called random deposition. (The term 'random' is not all that justified. All the models we'll look at are random. For reasons that will become apparent, a better term might be independently random'.)

This is not a type of deposition that occurs in nature, and it is too simple even to suggest what happens in nature, but it serves well as something to compare other processes to.

Random deposition with relaxation

An interesting variation of this is one in which the square drops onto an unoccupied site, then drops into the nearest site that has minimum height. (If both sides are lower, it tosses a coin to choose.) The system tries to find minimal energy through purely local shifts. This is called random deposition with relaxation. In these figures, you can see how periodicity comes in.

It will fit intuition better if the squares are replaced by disks.

Ballistic deposition

The most complicated of the three models is called ballistic deposition. In this, the square falls onto the first site where it sticks onto the first available neighbour. (The term ballistic' is common in the literature. I have no idea why.) Here, to, periodicity should be apparent.

### The problem

The problem is to figure out what the growing interface at the top looks like. In the real world, this is important because what actually interacts with the outside world is that growing interface, and one wants to deduce the underlying mechanisms from what one can see. One might hope to match a model one is familiar with.

We'll look now at some sample runs of each type.

Random deposition

Random deposition with relaxation

Ballistic deposition

### A closer look

Let's make another pass.

Random deposition

In this example, I start with a site of size $50$, dropping $40,000$ squares. On the left I have marked alternate generations with two colors.

The heights of successive generations are very, very rough, but in some sense they grow uniformly. That is to say, the average height grows very uniformly. I show this in the right-hand figure, where horizontal lines indicate the averages of selected height distributions. The growth of the averages is very uniform, indeed. However, what is also happening as the stacks grow is that the size of the fluctuations from the mean also grows. This has a well-defined measure--the standard deviation: $$\sigma^{2} = { \sum_{1}^{n} (x_{i} - \mu)^{2} \over n } \quad \hbox{ where \mu is the mean } \quad { \sum_{1}^{n} x_{i} \over n }, \quad$$ Here is a table of time, mean, standard deviation: $$\matrix { \hbox{t} & \mu & \sigma \cr 0 & 0 & 0 \cr 20 & 20.0 & 4.70 \cr 40 & 40.0 & 6.78 \cr 60 & 60.0 & 6.75 \cr 80 & 80.0 & 7.29 \cr 100 & 100.0 & 8.17 \cr }$$

Since $W$ squares fall in one unit of time, the average growth of the stacks can plausibly be expected to be at least roughly the same as the time elapsed. The table supports this hypothesis. But as far as the sizes of fluctuations in stack heights goes, all we can easily tell from the table is that they increase with time. We'd like to have a quantitative estimate of the rate of increase. But to do this with the run we are presently looking at would not be so informative, because the size of the base (i.e. $50$ sites) is not very large, and it turns out that quirky effects of small samples occur.

So I now make a run on a larger base, of size $500$, up to $t = 200$. We now plot a sequence of pairs $(t, \sigma)$ with $\Delta t = 10$ to see if we can guess how $\sigma$ depends on $t$:

It looks like $s$ might be proportional to $\sqrt{t}$, say $s = c \sqrt{t}$. If so, then $\log s = \log c + (1/2) \log m$. We check this by plotting $\log \sigma$ against $\log t$:

Sure enough, we see a line of slope $1/2$.

We can summarize: In the random deposition process, the average height of stacks grows with an approximately fixed velocity, but fluctuations in the height of stacks is large, and the standard deviation of the fluctuations is very near $\sqrt{t}$.

In fact, these claims can be rigourously proved. Each stack grows independently of every other stack, and grows something like the total number of 'heads' in tossing pennies. As the size of the base and the length of time for which the deposition proceeds, the fluctuations from the average $\mu = t$ are distributed very nearly according to the normal distribution $${ 1 \over \sigma \sqrt{2 \pi} } \cdot \mathstrut e^{ -(x - \mu)^{2} \over \phantom{\big |} 2 \sigma^{2} } \quad \hbox{ with } \quad \sigma^{2} = t \, ( 1 - 1/W ) \, .$$

There is another way to see what is going on. In the common random walk, a particle starts off at the origin of a line, and at each subsequent time $i$ goes either one step backwards or one step forwards, each equally likely. In the process at hand, each stack can either remain the same, with probability $(W-1)/W$, or advance by one unit, with probability $1/W$. In other words, it is a kind of skewed random walk. This accounts for all we see. The important point is that the stack above each site behaves completely independently of every other stack.

Random deposition with relaxation

For the other deposition processes, the first question is, how do they differ from the random deposition?

Here again is the relaxed deposition:

There are a number of things to observe. The first is that the process fills up the region below its top. It seems to progress with a uniform velocity, and in fact this velocity is essentially the same for all runs. The envelope of the growth is much, much smoother than it is for random deposition. Here is a typical scan:

A little thought will tell you that there are certain necessary features that explain the smoothness. Most important is that you will never see a jump of more than one in the scan. Furthermore, long successions of increments or decrements are rare, and you might therefore expect a lot of fairly flat segments, which is in fact what you see.

As with random deposition, an analysis of the standard deviation of interfaces is instructive.

What we see here is a plot of standard deviation versus time, on a log-log graph. In this run, the number of sites was $100$, and the total elapsed time was $2500$, making $250,000$ drops in all. It happens, however, that individual runs exhibit a great deal of randomness that one might think to be inessential, so the data represents an average over many runs. The process divides roughly into three regimes: (1) The stacks are nearly all empty, and relaxation hardly plays a role; the process looks much like a random deposition. (2) What I call the normal regime, in which the standard deviation increases approximately proportionally to $t^{1/4}$. This is far more slowly than for random deposition. (3) A period in which the sites are in some sense in a steady state. During this saturated regime, which presumably goes on forever, the probability distribution of standard deviation remains more or less constant.

What does 'saturated' mean? Unlike with random deposition, neighbouring sites are not independent of each other; the heights of stacks are correlated. Intuitively, the distance at which one site 'feels' another grows, until the correlation extends to roughly all sites. From that point on we are presumably looking at an essentially stationary process.

In a real physical process the number of sites can be enormous and the process approximates closely a limiting process, just as a random walk over a long time period approximates a continuous Brownian motion. The limit possesses a certain scale invariance and, based on a reasonable guess as to what it is, there are good theoretical reasons for seeing why the exponent $1/4$ appears. The limiting process is called the Edwards-Wilkinson process, after the physicists who discovered it. It also predicts when the cross-over between the various regimes takes place.

The continuous limit process is well understood, although how exactly discrete approximations relate to it are not. Many variations of this process can be imagined. It is asserted often that they all share the same continuous process but there seems to be little idea of how to prove this rigourously.

One mathematical question that immediately springs to mind is, how smooth are these interfaces?

Ballistic deposition

I now turn to the third process. The scale of its fluctuations lies somewhere between that of the two other processes.

I shan't give details as I did for the others, but computer experiments and some theoretical reasoning lead to the conjecture that standard deviation is proportional to $t^{1/3}$. But whereas the distribution of the other fluctuations is known or at least conjectured to be normal, the distribution here is likely to be that first defined by the mathematicians Craig Tracy and Harold Widom in a totally different context. This is an extremely active topic of research, part of ongoing exploration of what is called Kardar-Parisi-Zhang (KPZ) universality, which is all about figuring out what processes give rise to similar distributions. Verifying particular cases, even experimentally, is not easy.

### More ... ?

One might consider a mathematical version of TETRIS®. There are seven basic shapes, called tetrominoes.

Every now and then, a random tetromino drops from the sky above a random site, after a random rotation. It falls as far as it can. (In this theoretical game, no rows are deleted.)

And here is what the result looks like, after a while:

There is much in common with the other deposition schemes we have seen, particularly ballistic. In particular, a small amount of experiment indicates that the interface travels overall with a velocity essentially independent of any particular run involved. But, as Ivan Corwin asks, how much in other ways? For example, it seems that the fluctuations here are measurably wilder than those for ballistic deposition. As far as I know, there has not been any serious exploration of the question. The point is to try to see how universal KPZ universality actually is.

Anybody contemplating this question should first read the note by Farmudi and Vvedensky referred to below.

The popular literature on this subject is sparse and often misleading. This is forgiveable, because the mathematics involved is very difficult.

• A-L. Barabási and H. E. Stanley, Fractal concepts in surface growth, Cambridge University Press, 1995.

This is the standard text in the subject, and it is quite readable, although not mathematically deep. But it looks to me as though many of the diagrams of simulations are not accurate, or perhaps not labeled accurately. For one example, I have never seen data as 'good' as those in Figure 2.3 on a single run, so this probably illustrates an average of several runs. For another, all of the interfaces of the graph of the simulation of random deposition with relaxation in Figure 5.2 look far too rough. Among other things, they seem to have several jumps of more than two steps, which is impossible for the simulation they describe. So the simulation in play is perhaps one with relaxation to just a nearest neighbour.

Random deposition is treated in Chapter 4, random deposition with relaxation is discussed in Chapter 5, and ballistic deposition in Chapters 6-7.

• Ivan Corwin, Kardar-Parisi-Zhang universality, NOTICES of the American Mathematical Society, March 2016.

Corwin has written many surveys of this subject. This is the most recent. This article was my initial stimulus to try to learn about the subject of depositions, and Corwin has offered me much useful help. He is of course not to be blamed for my misconceptions.

• Bahman Farmudi and Dimitri Vvedensky, Large-scale simulations of ballistic deposition: the approach to asymptotic scaling, Physical Review E 83 020103 (2011)

This is the most thorough discussion I have seen of the enormous amount of computation necessary to get really good estimates of parameters of deposition processes. It is not cheerful reading, but the problem it presents, of understanding exactly why computations are so taxing, is intriguing.

Bill Casselman
University of British Columbia, Vancouver, Canada
Email Bill Casselman

The AMS encourages your comments, and hopes you will join the discussions. We review comments before they're posted, and those that are offensive, abusive, off-topic or promoting a commercial product, person or website will not be posted. Expressing disagreement is fine, but mutual respect is required.

Welcome to the Feature Column!

These web essays are designed for those who have already discovered the joys of mathematics as well as for those who may be uncomfortable with mathematics.

Search Feature Column

Feature Column at a glance


text:
# Determinacy interchanging the roles of both players

Let me refer to Jech's "Set Theory" Chap. 33 Determinacy:

"With each subset A of $\omega^\omega$ we associate the following game $G_A$, played by two players I and II. First I chooses a natural number $a_0$, then II chooses a natural number $b_0$, then I chooses a1, then II chooses b1, and so on. The game ends after $\omega$ steps; if the resulting sequence $<a_0, b_0, a_1, b_1, ...>$ is in A, then I wins, otherwise II wins.

A strategy (for I or II) is a rule that tells the player what move to make depending on the previous moves of both players. A strategy is a winning strategy if the player who follows it always wins. The game $G_A$ is determined if one of the players has a winning strategy.

The Axiom of Determinacy (AD) states that for every subset A of $\omega^\omega$, the game $G_A$ is determined."

Now, there is some apparent lack of symmetry in the definition of the $G_A$ game: the player who plays first (I) attempts for a sequence in A.

What happens if we interchange the roles of both players? I. e. if we let the player who plays first attempt for a sequence not in A? Let us call this game $G'_A$

Is it the case that for every subset A, A is determined wrt $G_A$ iff A is determined wrt $G'_A$?

-
In what axiom system are you asking this question? The game $G'_A$ is just the standard game played on $\omega^\omega \backslash A$, so under Determinacy this is trivially true. – Oliver Jul 22 '10 at 16:34
I forgot to mention it. In ZFC. – Marc Alcobé García Jul 23 '10 at 7:11

The answer is no. The game $G'(A)$ you describe is just the same as the complementary game $G(A^c)$ from I's point of view. So the question is: if one of the players has a winning strategy in $G(A)$, does one of the players have a winning strategy in $G(A^c)$? Using AC one can construct a set $A$ for which this is not the case.

We do a recursive construction, much as in the usual recursive construction of a nondetermined set. The main difference is set out in advance a strategy $\sigma$ for I to be a winning strategy for $A$. Fix any strategy $\sigma$ for I (so it takes a finite sequences of even length as input and gives natural numbers as output). For a given infinite sequence $y$ of moves for II we let $\sigma*y$ denote the member of $\omega^\omega$ resulting when I follows $\sigma$ and II plays out $y$.

Set $Z=\{\sigma*y:y\in\omega^\omega\}$. We will make sure $Z\subseteq A$, then certainly $\sigma$ will be a winning strategy for I in $G(A)$.

So we also define $\langle a_\alpha\rangle_{\alpha<2^\omega}$ and $\langle b_\alpha\rangle_{\alpha<2^\omega}$ so that no $a_\alpha$ is equal to any $b_\beta$ and both sets are disjoint from $Z$. Let $\langle\sigma_\alpha\rangle_{\alpha<2^\omega}$ enumerate all of I's strategies and let $\\langle\tau_\alpha\rangle_{\alpha<2^\omega}$ all of II's strategies. Suppose $\\langle a_\beta\rangle_{\beta<\alpha}$ and $\langle b_\beta\\rangle_{\beta<\alpha}$ have been defined. First consider $\sigma_\alpha$; if it equals $\sigma$ we don't do anything. Otherwise it disagrees with $\sigma$ on some $s$; there are $2^\omega$ many $x$ which are extensions of $s$, for each such $x$ we have $\sigma_\alpha*x\not\in Z$ so pick one which also isn't equal to an $a$ or a $b$ so far and make it $a_\alpha$.

Next consider $\tau_\alpha$. There are continuum many $x$ that I can play so that $\tau_\alpha*x$ does not land in $Z$ (just have $x$ give a first move different from that prescribed by $\sigma$). Using a cardinality argument we get some $x$ such that $\tau_\alpha*x$ is not in $Z$ and not equal to anything chosen so far, and set it equal to $b_\alpha$.

At the end let $A$ equal $Z$ together with the set of $a_\alpha$. Then $\sigma$ is a winning strategy for I in $G(A)$ because every play according to $\sigma$ lands in $Z$. But no strategy wins $A^c$. We've made sure that for every strategy for I there is a play which equals an $a_\alpha$ and thus lands in $A$ and so that strategy cannot be winning in $G(A^c)$. And similarly we've made sure that for every strategy for II there is a play equalling one of the $b_\beta$ and thus landing in $A^c$ and not winning for II in $G(A^c)$.

-
There was an error in the part of the argument considering the $\tau_\alpha$. I have corrected it. – Justin Palumbo Jul 27 '10 at 19:39

There is a general construction that swaps players in a game by ignoring the first move. Given $A$, define a set $B$ such that $x \in B$ if and only if $\sigma(x) \in A$, where $\sigma$ is the one-sided shift that discards the first element. That is, $(\sigma x)(n) = x(n+1)$. Now player II has a winning strategy $s_{II}$ for game $G_B$ (in the usual sense) if and only if player I would have a winning strategy $s'_{I}$ for the game $G'_A$ in the reversed sense. The strategies can be defined from each other in the following concrete way.

• $s_{II}(\tau) = s'_I(\sigma(\tau))$, where $\sigma$ is again a left shift
• $s_{I}'(\tau) = s_{II}(0\smallfrown\tau)$, where $0$ is any fixed, legal first move for player I

So if $B$ is determined, in the usual sense, then $A$ is determined in the opposite sense. The nice thing about this construction is that $B$ has the same classification as $A$ in the arithmetical and analytical hierarchies. So if we have a typical fragment of determinacy that shows $G(A)$ is determined, that same fragment will show that $G(B)$ is determined, and then the strategy translation above will show that $G'(A)$ is determined.

As Oliver pointed out, the game $G'(A)$ is also the same as the game $G(A^c)$. The construction I sketched here is better known in that light. It lets us prove that determinacy for closed sets is equivalent to determinacy for open sets, that determinacy for $\Pi^1_1$ sets is equivalent to determinacy for $\Sigma^1_1$ sets, etc., relative to very weak axiom systems.

Justin Palumbo's answer shows that if you look at games one set at a time, instead of looking at determinacy for reasonable pointclasses, then things are much more messy.

-
There is a subtlety here... It is true that if $\Gamma$ is a pointclass closed under continuous substitution then determinacy of sets in $\Gamma$ holds if and only if determinacy of sets whose complement lies in $\Gamma$ holds. But the question was about arbitrary sets. Under AC it is possible to construct using a diagonal argument a set $A$ such that $G(A)$ is determined and $G(A^c)$ is not. So I think actually the answer to the OPs question as stated is no... – Justin Palumbo Jul 22 '10 at 20:56
You're right; I misread the question as being about pointclasses (about why we don't consider both variations of the game). I'm going to clarify that in my answer. Marc Alcobé García, please consider accepting Justin Palumbo's answer instead of this one. – Carl Mummert Jul 22 '10 at 22:44


text:
This document presents the statistical catch-at-age stock assessment model developed as part of the Assessment For All (a4a) initiative of the European Commission Joint Research Centre. The stock assessment model framework is a non-linear catch-at-age model implemented in R and FLR, and using ADMB that can be applied rapidly to a wide range of situations with low parametrization requirements. The model structure is defined by submodels, which are the different parts that require structural assumptions. There are 5 submodels in operation: a model for F-at-age, a model for the initial age structure, a model for recruitment, a (list) of model(s) for abundance indices catchability-at-age, and a list of models for the observation variance of catch-at-age and abundance indices. The submodels form use linear models. This opens the possibility of using the linear modelling tools available in R: see for example the mgcv (http://cran.r-project.org/web/packages/mgcv/index.html) gam formulas, or factorial design formulas using. Detailed model formulas, several diagnostic tools and a large set of models are presented in the document. Additionaly, advanced features like external weighting of the likelihood components and MCMC fits are also described. The target audience for this document are readers with some experience in R and some background on stock assessment. The document explains the approach being developed by a4a for fish stock assessment and scientific advice. It presents a mixture of text and code, where the first explains the concepts behind the methods, while the last shows how these can be run with the software provided.

## Required packages

To follow this tutorial you should have installed the following packages:

You can do so as follows,

install.packages(c("copula","triangle", "coda", "XML", "reshape2", "latticeExtra"))
# from FLR
install.packages(c("FLCore", "FLa4a"), repos="http://flr-project.org/R")
# This chunk loads all necessary packages
library(FLa4a)
library(latticeExtra)
# datasets
data(ple4)
data(ple4.indices)
data(ple4.index)

# Background

The stock assessment model framework is a non-linear catch-at-age model implemented in R/FLR/ADMB that can be applied rapidly to a wide range of situations with low parametrization requirements.

In the a4a assessment model, the model structure is defined by submodels, which are the different parts of a statistical catch at age model that require structural assumptions.

There are 5 submodels in operation: - a model for F-at-age, - a (list) of model(s) for abundance indices catchability-at-age, - a model for recruitment, - a list of models for the observation variance of catch-at-age and abundance indices, - a model for the initial age structure,

In practice, we fix the variance models and the initial age structure models, but in theory these can be changed.

The submodels form use linear models. This opens the possibility of using the linear modelling tools available in R: see for example the mgcv gam formulas, or factorial design formulas using lm(). In R’s linear modelling language, a constant model is coded as $$\sim 1$$, while a slope over age would simply be $$\sim age$$. For example, we can write a traditional year/age separable F model like $$\sim factor(age) + factor(year)$$.

The ‘language’ of linear models has been developing within the statistical community for many years, and constitutes an elegant way of defining models without going through the complexity of mathematical representations. This approach makes it also easier to communicate among scientists:

There are two basic types of assessments available in a4a: the management procedure fit and the full assessment fit. The management procedure fit does not compute estimates of covariances and is therefore quicker to execute, while the full assessment fit returns parameter estimates and their covariances at the expense of longer fitting time.

# Stock assessment model details

Modelled catches $$C$$ are defined in terms of the three quantities, natural mortality $$M$$, fishing mortality $$F$$ and recruitment $$R$$, using a modified form of the well known Baranov catch equation:

$C_{ay} = \frac{\bf{F}_{ay}}{\bf{F}_{ay}+M_{ay}}\left(1 - e^{-(\bf{F}_{ay}+M_{ay})}\right) \bf{R}_{y}e^{-\sum (\bf{F}_{ay} + M_{ay})}$

where $$a$$ and $$y$$ denote age and year. Modelled survey indices $$I$$ are defined in terms of the same three quantities with the addition of survey catchability $$Q$$:

$I_{ays} = \bf{Q}_{ays} \bf{R}_{y}e^{-\sum (\bf{F}_{ay} + M_{ay})}$

where $$s$$ denotes survey or abundance index and allows for multiple surveys to be considered. Observed catches $$C^{(obs)}$$ and the observed survey indices $$I^{(obs)}$$ are assumed to be log-normally distributed, or equivalently, normally distributed on the log-scale, with age, year and survey specific observation variance:

$\log C^{(obs)}_{ay} \sim \text{Normal} \Big( \log C_{ay}, \bf{\sigma}^2_{ay}\Big) \qquad \log I^{(obs)}_{ays} \sim \text{Normal} \Big( \log I_{ays}, \bf{\tau}^2_{ays} \Big)$

The full log-likelihood for the a4a statistical catch at age model can now be defined as the sum of the log-likelihood of the observed catches ($$\ell_N$$ is the log-likelihood of a normal distribution)

$\ell_C = \sum_{ay} w^{(c)}_{ay}\ \ell_N \Big( \log C_{ay}, \bf{\sigma}^2_{ay} ;\ \log C^{(obs)}_{ay} \Big)$

and the log-likelihood of the observed survey indices

$\ell_I = \sum_s \sum_{ay} w^{(s)}_{ays}\ \ell_N \Big( \log I_{ays}, \bf{\tau}_{ays}^2 ;\ \log I^{(obs)}_{ays} \Big)$

giving the total log-likelihood

$\ell = \ell_C + \ell_I$

which is defined in terms of the strictly positive quantites, $$M_{ay}$$, $$F_{ay}$$, $$Q_{ays}$$ and $$R_{y}$$, and the observation variances $$\sigma_{ay}$$ and $$\tau_{ays}$$. As such, the log-likelihood is over-parameterised as there are many more parameters than observations. In order to reduce the number of parameters, $$M_{ay}$$ is assumed known (as is common), and the remaining parameters are written in terms of a linear combination of covariates $$x_{ayk}$$, e.g.

$\log F_{ay} = \sum_k \beta_k x_{ayk}$

where $$k$$ is the number of parameters to be estimated and is sufficiently small. Using this tecnique the quantities $$\log F$$, $$\log Q$$, $$\log \sigma$$ and $$\log \tau$$ %$$\log \text{initial\,age\,structure}$$ % this is not present in the above (in bold in the equations above) can be described by a reduced number of parameters. The following section has more discussion on the use of linear models in a4a.

## Stock recruitment relationships

The a4a statistical catch at age model can addiionally allow for a functional relationship to be imposed that links predicted recruitment $$\tilde{R}$$ based on spawning stock biomass and modelled recruitment $$R$$, included as a fixed variance random effect. Options for the relationship are the hard coded models Ricker, Beverton Holt, smooth hockeystick or geometric mean. This is implemented by including a third component in the log-likelihood

$\ell_{SR} = \sum_y \ell_N \Big( \log \tilde{R}_y(a, b), \phi_y^2 ;\ \log R_y \Big)$

giving the total log-likelihood

$\ell = \ell_C + \ell_I + \ell_{SR}$

Using the (time varying) Ricker model as an example, predicted recruitment is

$\tilde{R}_y(a_y,b_y) = a_y S_{y-1} e^{-b_y S_{y-1}}$

where $$S$$ is spawning stock biomass derived from the model parameters $$F$$ and $$R$$, and the fixed quantites $$M$$ and mean weights by year and age. It is assumed that $$R$$ is log-normally distributed, or equivalently, normally distributed on the log-scale about the (log) recruitment predicted by the SR model $$\tilde{R}$$, with known variance $$\phi^2$$, i.e.

$\log R_y \sim \text{Normal} \Big( \log \tilde{R}_y, \phi_y^2 \Big)$

which leads to the definition of $$\ell_{SR}$$ given above. In all cases $$a$$ and $$b$$ are strictly positive, and with the quantities $$F$$, $$R$$, etc. linear models are used to parameterise $$\log a$$ and/or $$\log b$$, where relevant.

By default, recruitment $$R$$ as apposed to the reruitment predicted from a stock recruitment model $$\tilde{R}$$, is specified as a linear model with a parameter for each year, i.e.

$\log R_y = \gamma_y$

This is to allow modelled recruitment $$R_y$$ to be shrunk towards the stock recruitment model. However, if it is considered appropriate that recruitment can be determined exactly by a relationship with covariates, it is possible, to instead define $$\log R$$ in terms of a linear model in the same way as $$\log F$$, $$\log Q$$, $$\log \sigma$$ and $$\log \tau$$. %But this is pretty much the same as taking a geometric mean, with a model on log a, and making the variance very small.

# Quick and dirty

Here we show a simple example of using the assessment model using plaice in the North Sea. The default settings of the stock assessment model work reasonably well. It’s an area of research that will improve with time. Note that because the survey index for plaice has missing values we get a warning saying that we assume these values are missing at random.

data(ple4)
data(ple4.indices)
fit <- sca(ple4, ple4.indices)

To inspect the stock assessment summary constituted of trends of fishing mortality (harvest), spawning stock biomass (SSB), catch and recruits, the user may add the a4aFit object to the original FLStock object using the + method and plot the result (Figure 1).

stk <- ple4 + fit
plot(stk)


text:
# Why does PyTorch use a different formula for the cross-entropy?

In my understanding, the formula to calculate the cross-entropy is

$$H(p,q) = - \sum p_i \log(q_i)$$

But in PyTorch nn.CrossEntropyLoss is calculated using this formula:

$$loss = -\log\left( \frac{\exp(x[class])}{\sum_j \exp(x_j)} \right)$$

that I think it only addresses the $$\log(q_i)$$ part in the first formula.

Why does PyTorch use a different formula for the cross-entropy?

• You are using the Softmax CE loss, use BCE loss or Binary CE loss for your formula.
– user9947
Jan 15 '20 at 8:09

When you one-hot-encode your labels with $$p_i \in \{0,1\}$$ you get $$p_i = 0$$ iff $$i$$ is not correct and, equivalently, $$p_i =1$$ iff $$i$$ is correct.

Hence, $$p_i \log(q_i) = 0 \log(q_i) = 0$$ for all classes except the "truth" and $$p_i \log(q_i) = 1 \log(q_i) = \log(q_i)$$ for the correct prediction.

Therefore, your loss reduces to: $$H(p,q) = - \sum p_i \log(q_i) = - \log(q_{truth})$$

• I'm not sure whether OP wanted to know this.
– user9947
Jan 15 '20 at 8:11
• @DuttaA have a look at the last part of the question: "that I think it only addresses the $\log(q_i)$ part in the first formula. So is that means Pytorch using different Cross entropy formula?". Reads to me that it actually is exactly about this. . Jan 15 '20 at 8:23
• OMG! yes, this is what I meant, I accidentally mix language modeling cross-entropy (that have continuous values of $p_i$) and PyTorch nn.CrossEntropyLoss. I forgot in classification $p_i$ will be 0 or 1 so the PyTorch function is valid, but of course, I can't use it directly for Language Modeling case. Thank You! Jan 15 '20 at 8:35
• @Sammy This answer partially answers the question, but I think you should also explain the function inside the log (in the PyTorch version). What function is that? Note that I know what function that is, but, for completeness, I think you should also explain it. Maybe you should also refer to the documentation. Afterward, I will upvote your answer ;)
– nbro
Jan 15 '20 at 13:17


text:
# Tag Info

## Hot answers tagged wing-sweep

41

The wing was swept to move the center of lift back, closer to the center of gravity. When the design of the DC-1 was advanced to a point where the center of gravity became clear, a redesign of the wing-fuselage intersection would had been more effort, so the outer wing panels were swept back a little. From this website by Jeff Lucker: the unique and ...

29

Sweeping a wing costs weight, for several reasons. Therefore, if the goal is to fly high and achieve long range, an unswept wing is the better choice. Only when transport performance needs to be maximized will a moderately swept wing look better. Why is a swept wing heavier? Sweep reduces aerodynamic efficiency, so a larger wing is needed to create the ...

22

Disadvantages of wing sweep Lift curve slope is reduced by the cosine of the quarter chord sweep angle. This means more angle of attack for takeoff, which requires a longer take-off run and a longer landing gear to avoid a tail strike on rotation. If you rotate the airplane, the tips of backward swept wings come down when the aircraft rotates for take-off. ...

20

The tail sweep helps to extend its range of angles of attack. For both stability and control, it is very important that the tail doesn't stall before the wing in any circumstances (for a classic design; for canards it may be beneficial). Generally, for high positive angles of attack, the horizontal tail already has an edge because it is set at a lower angle ...

17

It was obvious to the design team that conventional aircraft could not satisfy the required specification; knowing little about high-speed flight and unable to glean much from the Royal Aircraft Establishment or the US, the team at Avro investigated German Second World War swept wing research. The team estimated that an otherwise conventional aircraft, with ...

15

Wing sweep improves the performance by delaying the shock waves and accompanying aerodynamic drag rise caused by fluid compressibility at high (near sonic) speeds. However, there are some disadvantages associated with the wing sweep like: Wing sweep reduces the slope of the lift curve and the maximum lift coefficient of the wing. This means that the swept ...

15

MiG-23 For the MiG-23, it was manual. [The] MiG-23 had a completely manual wing sweep control. But there was no gauge to indicate optimum wing position for the surrounding conditions. "You had to manually put the wings into position to safely fly the airplane," (...) "They had a very nice gauge that showed where the wings were commanded to, ...

13

Mark is right when he says that there is no induced drag at supersonic speeds, but it is an invitation for misunderstandings. Induced drag is replaced by lift wave drag, and all what happens is that aerodynamicists choose to use two different names for basically the same effect: Air gets pushed down. As usual when I post a long answer, I was not quite happy ...

12

This was an intentional test run by Grumman in response to US Navy concerns about asymmetric wing sweep. From The story of F-14A, Aircraft No. 3, BuNo. 157982 One of these [flight tests] was in response to concerns raised by the US Navy regarding asymmetrical wing sweep. No. 3 is best remembered for photo shown at right. A series of flight tests were ...

10

The two pictures in the question only show how to decompose the speed vector into the normal and parallel components. They are nowhere near "real" flow vectors. First I recommend to read this answer so you get an idea how air is accelerated and decelerated when flowing around an airfoil. Note that the low pressure area over the wing will suck in ...

10

I finally found several arguments to answer this question from different sources. Sweep and Mach Number from Airbus This argument is developed in an Airbus magazine from 1985. To grasp it it is important to understand the purpose of swept wing. By sweeping the main wings one introduces an angle between the leading edge of the wing and a line perpendicular ...

10

The oblique wing and the swing wing aim to reduce the drag over a wide range of speeds by changing the wing sweep. While the swing wing rotates a part of the wing to achieve this, the oblique wing rotates the whole wing. The first production aircraft with variable sweep wings was the F111 Aardvark, while the most famous one is the F-14 Tomcat. "F-14 Tomcat ...

10

When looking at statements like these, it's important to consider which parameters are varied and which parameters remain fixed. Let's get this out of the way first by correcting your statements: Lift increases with increase in airspeed with constant angle of attack but induced drag reduces with increase in airspeed with lift force remaining constant ...

9

Advantages of an oblique wing over swing wings: Low wave drag due to favorable volume distribution over length. Lower structural mass. The hinge of a swing wing is heavy! No shift in the center of lift (if the sweep angle is higher than the Mach angle). A swing wing experiences a severe shift in the center of lift. Disadvantages of an oblique wing over a ...

8

Basically, because of its mission requirements. It wasn't intended to fly fast, and because it didn't need to go particularly fast, there was no need to sweep the wing. Instead, it was supposed to provide a massive amount of data (by 20th century standards) along its flight path, which necessitated long flight times: thus the long, straight wings providing a ...

8

I would not say that no other bomber ever had delta wings. Convair B-58 Hustler and Mirage IV both have delta wings. They are both supersonic though. The B-52 did not have to worry too much about MTOW and empty weight since SAC already had bases with long runways capable of really heavy aircraft. That's thanks to B-52's predecessor B-36. The B-36 was the ...

7

The vertical tail and the horizontal tail are wings, and follow the same construction rules. The span-wise lift distribution is much better if the wing is tapered: more lift at the root, less at the tip. The wing can be built lighter, root area is larger, torsion stiffness is higher. Vortex induced drag is lower. As a reference, from Torenbeek: pages 232 &...

7

From Wikipedia, the cruise speeds of the both airplanes are fairly similar: A380, nominal wing sweep angle 33.5 degrees, cruise speed Mach 0.85 Boeing 747: nominal wing sweep angle 37.5 degrees, cruise speed Mach 0.84 .. 0.9, depending on the model. So those numbers would contradict the claim in your question, but align with your expectations. There are ...

6

Yes it could, but don't hold your breath. Forward sweep means that the aeroelastic forces will twist the wing at high lift into an increased angle of attack, so such a wing needs to be stiff in order to avoid divergence (PDF!). Also, aileron effectiveness is excellent, so a forward-swept configuration is a good choice for dogfighting. Since backward sweep ...

6

Lift increases with increase in airspeed but induced drag reduces with increase in airspeed. If you keep weight ($W$) constant, the total lift ($L$) does not change with change in airspeed ($V$). $L=W$ for quasi-steady flight, right? What is changed is the lift coefficient, $C_L$. As speed increases, $C_L$ decreases. Induced drag coefficient ($C_{D_i}$) is ...

5

There is no magical switch that gets flipped once you cross a specific sweep angle. Things are gradually changing with the cosine of the local sweep angle. Now let's look at your questions one by one: What happens to the wingtip wake on a forward-swept wing? Nothing special. It is part of the full wake system and behaves like in any other aircraft of ...

5

A swept wing has less torsional strength than a straight wing, ie there's less structure at 90d to the root supporting the tip of a swept wing. Thus a swept wing twists when it's loaded, which unloads the tips and thus generates less lift at higher weights.

5

In supplement to @Dave's answer and as he said depending on the airplane, the complexity of calculating the c.g. can vary greatly. The manufacturer of every large aircraft I've programmed weight & balance for has had tables for each tank. In Boeing's case for the 747-400, each table line is a tank volume in gallons and the tank c.g. at that volume in ...

5

It depends on the rigidity of the upper and lower skins. As far as large jets go, you see different variations; ribs parallel to the airflow, ribs perpendicular to one of the spars, ribs perpendicular to mean chord. In the 737 cutaway below, they appear to be more or less perpendicular to the rear spar. Aerodynamically it's not important on large swept wing ...

4

A swept wing would most likely adversely affect the handling of a U2 during a typical mission profile, and would adversely affect the structure required (weight) of the aircraft. A swept wing adversely impacts the stall characteristics and stability when flying near a stall. It should be noted that while a swept wing helps manage effects due to the ...

4

One aerodynamic behavior appears at high angle of attack : dihedral or anhedral wake, which affects (in some weird mixture) both roll and yaw stability. Forward swept trailing edge is much more stable than backward swept trailing edge. Some jet fighters (let's say F-22) and most aerobatic airplanes (let's say SU-31) adopted forward swept trailing edges in ...

4

The pivot wing, also called oblique wing is a type of wing design that tries to minimize the drag over the entire speed range of the aircraft. At low speeds, the main concern is the induced drag, which can be reduced by increasing the aspect ratio. That is why low speed aircrafts like gliders have very long slender wings. As the speed increases, the ...

4

Yes. Variable sweep airplanes exist and it shouldn't be a problem to design a system able to sweep the wings forward and rearwards. However, the few aerodynamic advantages of the forward swept wing at low speeds doesn't justify the added complexity, weight and loss of space inside the fuselage.

4

Sweep is not strictly required, but helpful to increase supersonic L/D. Just witness the length to which aircraft manufacturers went to sweep wings for Mach 2+ flight. However, sweep comes with its own set of problems.

4

The tail is swept to move the aerodynamic center aft to provide more "leverage" for the tail in generating down force. Same as sweeping the fin, although that is also for looks. If the airplane had a straight fin and straight horizontal tail, the tail surfaces would have to be larger to have the same tail "volume" (area x moment arm) or the fuselage itself ...

Only top voted, non community-wiki answers of a minimum length are eligible


text:
Gaussian Bivariate Distribution

The Gaussian bivariate distribution is given by

 (1)

where
 (2)

and
 (3)

is the Covariance. Let and be normally and independently distributed variates with Mean 0 and Variance 1. Then define
 (4) (5)

These new variates are normally distributed with Mean and , Variance
 (6) (7)

and Covariance
 (8)

The Covariance matrix is
 (9)

where
 (10)

The joint probability density function for and is
 (11)

However, from (4) and (5) we have
 (12)

Now, if
 (13)

then this can be inverted to give
 (14)

Therefore,

 (15)

Expanding the Numerator gives
 (16)
so

 (17)
But

 (18)

The Denominator is
 (19)
so
 (20)

and

 (21)

Solving for and and defining
 (22)

gives
 (23) (24)

The Jacobian is
 (25)

Therefore,
 (26)

and

 (27)

where

 (28)

Now, if
 (29)

then
 (30)

 (31) (32)

so
 (33) (34)

where
 (35)

The Characteristic Function is given by

 (36)

where
 (37)

and
 (38)

Now let
 (39) (40)

Then

 (41)

where
 (42)

Complete the Square in the inner integral

 (43)
Rearranging to bring the exponential depending on outside the inner integral, letting

 (44)

and writing
 (45)

gives

 (46)
Expanding the term in braces gives

 (47)
But is Odd, so the integral over the sine term vanishes, and we are left with

 (48)
Now evaluate the Gaussian Integral

 (49)

to obtain the explicit form of the Characteristic Function,

 (50)

Let and be two independent Gaussian variables with Means and for , 2. Then the variables and defined below are Gaussian bivariates with unit Variance and Cross-Correlation Coefficient :

 (51)

 (52)

The conditional distribution is
 (53)

where
 (54) (55)

The marginal probability density is
 (56)

References

Abramowitz, M. and Stegun, C. A. (Eds.). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, 9th printing. New York: Dover, pp. 936-937, 1972.

Spiegel, M. R. Theory and Problems of Probability and Statistics. New York: McGraw-Hill, p. 118, 1992.


text:
### Still have math questions?

a. Use the binomial series and the fact that $$\frac { d } { d x } \sin ^ { - 1 } x = ( 1 - x ^ { 2 } ) ^ { - 1 / 2 }$$
$$\left. \begin{array} { l } { \text { (a) } x + \frac { x ^ { 3 } } { 6 } + \frac { 3 x ^ { 5 } } { 40 } + \frac { 5 x ^ { 7 } } { 112 } , \text { radius of convergence } = 1 } \\ { \text { (b) } \frac { \pi } { 2 } - x - \frac { x ^ { 3 } } { 6 } - \frac { 3 x ^ { 5 } } { 40 } - \frac { 5 x ^ { 7 } } { 112 } } \end{array} \right.$$


text: —We present a methodology for calibrating multiple light source locations in 3D from images. The procedure involves the use of a novel calibration object that consists of three spheres at known relative positions. The process uses intensity images to find the positions of the light sources. We conducted experiments to locate light sources in 51 different positions in a laboratory setting. Our data shows that the vector from a point in the scene to a light source can be measured to within <tmath>$2.7 \pm .4 ^{\circ }$</tmath> at <tmath>$\alpha =.05$</tmath> (6 percent relative) of its true direction and within <tmath>$0.13 \pm .02$</tmath> m at <tmath>$\alpha =.05$</tmath> (9 percent relative) of its true magnitude compared to empirically measured ground truth. Finally, we demonstrate how light source information is used for color correction.


text:
## Annals of Functional Analysis

### Nonlinear isometries between function spaces

#### Abstract

We demonstrate that any surjective isometry $T\colon \mathcal{A}\to \mathcal{B}$ not assumed to be linear between unital, completely regular subspaces of complex-valued, continuous functions on compact Hausdorff spaces is of the form $\begin{equation*}T(f)=T(0)+\operatorname{Re}[\mu \cdot(f\circ\tau)]+i\operatorname{Im}[\nu \cdot(f\circ\rho)],\end{equation*}$ where $\mu$ and $\nu$ are continuous and unimodular, there exists a clopen set $K$ with $\nu=\mu$ on $K$ and $\nu=-\mu$ on $K^{c}$, and $\tau$ and $\rho$ are homeomorphisms.

#### Article information

Source
Ann. Funct. Anal. Volume 8, Number 4 (2017), 460-472.

Dates
Accepted: 13 December 2016
First available in Project Euclid: 2 June 2017

https://projecteuclid.org/euclid.afa/1496368961

Digital Object Identifier
doi:10.1215/20088752-2017-0010

Keywords
isometry nonlinear function spaces

#### Citation

Roberts, Kathleen; Lee, Kristopher. Nonlinear isometries between function spaces. Ann. Funct. Anal. 8 (2017), no. 4, 460--472. doi:10.1215/20088752-2017-0010. https://projecteuclid.org/euclid.afa/1496368961

#### References

• [1] J. Araujo and J. J. Font,Linear isometries between subspaces of continuous functions, Trans. Amer. Math. Soc.349(1997), no. 1, 413–428.
• [2] S. Banach,Théorie des opérations linéaires, Chelsea, New York, 1955.
• [3] S. Eilenberg,Banach space methods in topology, Ann. of Math. (2)43(1942), 568–579.
• [4] R. J. Fleming and J. E. Jamison,Isometries on Banach Spaces: Function Spaces, Chapman & Hall/CRC Monogr. Surv. Pure Appl. Math.129, Chapman & Hall, Boca Raton, FL, 2003.
• [5] A. Jamshidi and F. Sady,Extremely strong boundary points and real-linear isometries, Tokyo J. Math.38(2015), no. 2, 477–490.
• [6] H. Koshimizu, T. Miura, H. Takagi, and S.-E. Takahasi,Real-linear isometries between subspaces of continuous functions, J. Math. Anal. Appl.413(2014), no. 1, 229–241.
• [7] S. Mazur and S. Ulam,Sur les transformationes isométriques d’espaces vectoriels, normés, C. R. Math. Acad. Sci. Paris194(1932), 946–948.
• [8] T. Miura, “Surjective isometries between function spaces” inFunction Spaces in Analysis, Contemp. Math.645, Amer. Math. Soc., Providence, 2015, 191–197.
• [9] S. B. Myers,Banach spaces of continuous functions, Ann. of Math. (2)49(1948), 132–140.
• [10] M. H. Stone,Applications of the theory of boolean rings in topology, Trans. Amer. Math. Soc.41(1937), no. 3, 375–481.


text:
Convolutional Neural Networks (CNN) explained

Posted May 21, 2019 by Rokas Balsys

##### Residual Networks:

Welcome to another tutorial! Now we will learn how to build very deep convolutional networks, using Residual Networks (ResNets). This model was the winner of ImageNet challenge in 2015. The fundamental breakthrough with ResNet was it allowed us to train extremely deep neural networks with 150+layers successfully. Prior to ResNet training very deep neural networks was difficult due to the problem of vanishing gradients.

However, increasing network depth does not work by simply stacking layers together. Deep networks are hard to train because of the vanishing gradient problem — as the gradient is back-propagated to earlier layers, repeated multiplication may make the gradient extremely small. As a result, as the network goes deeper, its performance gets saturated or even starts degrading rapidly.

In this tutorial, we will:

• Implement the basic building blocks of ResNets.
• Put together these building blocks to implement and train a state-of-the-art neural network for image classification.

Let's run the cell below to load the required packages:

import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'
import cv2
import numpy as np
from keras import layers
from keras.layers import Input, Add, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D, AveragePooling2D, MaxPooling2D
from keras.initializers import glorot_uniform
from keras.utils import plot_model
from IPython.display import SVG
from keras.utils.vis_utils import model_to_dot
import keras.backend as K
import tensorflow as tf

ROWS = 64
COLS = 64
CHANNELS = 3
CLASSES = 2

def read_image(file_path):
return cv2.resize(img, (ROWS, COLS), interpolation=cv2.INTER_CUBIC)

def prepare_data(images):
m = len(images)
X = np.zeros((m, ROWS, COLS, CHANNELS), dtype=np.uint8)
y = np.zeros((1, m), dtype=np.uint8)
for i, image_file in enumerate(images):
if 'dog' in image_file.lower():
y[0, i] = 1
elif 'cat' in image_file.lower():
y[0, i] = 0
return X, y

def convert_to_one_hot(Y, C):
Y = np.eye(C)[Y.reshape(-1)].T
return Y

TRAIN_DIR = 'Train_data/'
TEST_DIR = 'Test_data/'

train_images = [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR)]
test_images =  [TEST_DIR+i for i in os.listdir(TEST_DIR)]

train_set_x, train_set_y = prepare_data(train_images)
test_set_x, test_set_y = prepare_data(test_images)

X_train = train_set_x/255
X_test = test_set_x/255

Y_train = convert_to_one_hot(train_set_y, CLASSES).T
Y_test = convert_to_one_hot(test_set_y, CLASSES).T

print ("number of training examples =", X_train.shape[0])
print ("number of test examples =", X_test.shape[0])
print ("X_train shape:", X_train.shape)
print ("Y_train shape:", Y_train.shape)
print ("X_test shape:", X_test.shape)
print ("Y_test shape:", Y_test.shape)

Output:

number of training examples = 6002
number of test examples = 1000
X_train shape: (6002, 64, 64, 3)
Y_train shape: (6002, 2)
X_test shape: (1000, 64, 64, 3)
Y_test shape: (1000, 2)

##### 2 - Building a Residual Network:

In ResNets, a "shortcut" or a "skip connection" allows the gradient to be directly backpropagated to earlier layers:

The image on the left shows the "main path" through the network. The image on the right adds a shortcut to the main path. By stacking these ResNet blocks on top of each other, you can form a very deep network.

ResNet blocks with the shortcut also makes it very easy for one of the blocks to learn an identity function. This means that you can stack on additional ResNet blocks with little risk of harming training set performance. (There is also some evidence that the ease of learning an identity function--even more than skip connections helping with vanishing gradients--accounts for ResNets' remarkable performance.)

Two main types of blocks are used in a ResNet, depending mainly on whether the input/output dimensions are same or different. We are going to implement both of them.

Why do Skip Connections work?

• They mitigate the problem of vanishing gradient by allowing this alternate shortcut path for gradient to flow through
• They allow the model to learn an identity function which ensures that the higher layer will perform at least as good as the lower layer, and not worse

##### 2.1 - The identity block:

The identity block is the standard block used in ResNets, and corresponds to the case where the input activation (say $a^{[l]}$) has the same dimension as the output activation (say $a^{[l+2]}$). To flesh out the different steps of what happens in a ResNet's identity block. We'll actually implement a slightly more powerful version of this identity block, in which the skip connection "skips over" 3 hidden layers rather than 2 layers.

The upper path is the "shortcut path." The lower path is the "main path." In this diagram, we have also made explicit the CONV2D and ReLU steps in each layer.

Here're the individual steps:

First component of main path:

• The first CONV2D has $F_1$ filters of shape (1,1) and a stride of (1,1). Its padding is "valid" and its name should be conv_name_base + '2a'. Use 0 as the seed for the random initialization.
• The first BatchNorm is normalizing the channels axis. Its name should be bn_name_base + '2a'.
• Then apply the ReLU activation function. This has no name and no hyperparameters.

Second component of main path:

• The first CONV2D has $F_2$ filters of shape (1,1) and a stride of (1,1). Its padding is "valid" and its name should be conv_name_base + '2b'. Use 0 as the seed for the random initialization.
• The first BatchNorm is normalizing the channels axis. Its name should be bn_name_base + '2b'.
• Then apply the ReLU activation function. This has no name and no hyperparameters.

Third component of main path:

• The first CONV2D has $F_3$ filters of shape (1,1) and a stride of (1,1). Its padding is "valid" and its name should be conv_name_base + '2c'. Use 0 as the seed for the random initialization.
• The third BatchNorm is normalizing the channels axis. Its name should be bn_name_base + '2c'. Note that there is no ReLU activation function in this component.

Final step:

• The shortcut and the input are added together.
• Then apply the ReLU activation function. This has no name and no hyperparameters.

Arguments:
X - input tensor of shape (m, n_H_prev, n_W_prev, n_C_prev)
f - integer, specifying the shape of the middle CONV's window for the main path
filters - python list of integers, defining the number of filters in the CONV layers of the main path
stage - integer, used to name the layers, depending on their position in the network
block - string/character, used to name the layers, depending on their position in the network

Returns:
X - output of the identity block, tensor of shape (n_H, n_W, n_C)

def identity_block(X, f, filters, stage, block):
# defining name basis
conv_name_base = 'res' + str(stage) + block + '_branch'
bn_name_base = 'bn' + str(stage) + block + '_branch'

# Retrieve Filters
F1, F2, F3 = filters

# Save the input value. We'll need this later to add back to the main path.
X_shortcut = X

# First component of main path
X = Conv2D(filters = F1, kernel_size = (1, 1), strides = (1,1), padding = 'valid', name = conv_name_base + '2a', kernel_initializer = glorot_uniform(seed=0))(X)
X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X)
X = Activation('relu')(X)

# Second component of main path
X = Conv2D(filters = F2, kernel_size = (f, f), strides = (1,1), padding = 'same', name = conv_name_base + '2b', kernel_initializer = glorot_uniform(seed=0))(X)
X = BatchNormalization(axis = 3, name = bn_name_base + '2b')(X)
X = Activation('relu')(X)

# Third component of main path
X = Conv2D(filters = F3, kernel_size = (1, 1), strides = (1,1), padding = 'valid', name = conv_name_base + '2c', kernel_initializer = glorot_uniform(seed=0))(X)
X = BatchNormalization(axis = 3, name = bn_name_base + '2c')(X)

# Final step: Add shortcut value to main path, and pass it through a RELU activation
X = Activation('relu')(X)

return X

tf.reset_default_graph()
with tf.Session() as test:
A_prev = tf.placeholder("float", [3, 4, 4, 6])
X = np.random.randn(3, 4, 4, 6)
A = identity_block(A_prev, f = 2, filters = [2, 4, 6], stage = 1, block = 'a')
test.run(tf.global_variables_initializer())
out = test.run([A], feed_dict={A_prev: X, K.learning_phase(): 0})
print("out = ", out[0][1][1][0])

out =  [1.2611859 2.0174022 2.8489313 2.459698  4.33884   2.1486042]

##### 2.2 - The convolutional block:

The ResNet "convolutional block" is the other type of block:

The CONV2D layer in the shortcut path is used to resize the input 𝑥 to a different dimension, so that the dimensions match up in the final addition needed to add the shortcut value back to the main path. For example, to reduce the activation dimensions's height and width by a factor of 2, we can use a 1x1 convolution with a stride of 2. The CONV2D layer on the shortcut path does not use any non-linear activation function. Its main role is to just apply a (learned) linear function that reduces the dimension of the input, so that the dimensions match up for the later addition step.

The details of the convolutional block are as follows:

First component of main path:

• The first CONV2D has $F_1$ filters of shape (1,1) and a stride of (1,1). Its padding is "valid" and its name should be conv_name_base + '2a'. Use 0 as the seed for the random initialization.
• The first BatchNorm is normalizing the channels axis. Its name should be bn_name_base + '2a'.
• Then apply the ReLU activation function. This has no name and no hyperparameters.

Second component of main path:

• The first CONV2D has $F_2$ filters of shape (1,1) and a stride of (1,1). Its padding is "valid" and its name should be conv_name_base + '2b'. Use 0 as the seed for the random initialization.
• The first BatchNorm is normalizing the channels axis. Its name should be bn_name_base + '2b'.
• Then apply the ReLU activation function. This has no name and no hyperparameters.

Third component of main path:

• The first CONV2D has $F_3$ filters of shape (1,1) and a stride of (1,1). Its padding is "valid" and its name should be conv_name_base + '2c'. Use 0 as the seed for the random initialization.
• The third BatchNorm is normalizing the channels axis. Its name should be bn_name_base + '2c'. Note that there is no ReLU activation function in this component.

Shortcut path:

• The CONV2D has $F_3$ filters of shape (1,1) and a stride of (s,s). Its padding is "valid" and its name should be conv_name_base + '1'.
• The BatchNorm is normalizing the channels axis. Its name should be bn_name_base + '1'.

Final step:

• The shortcut and the input are added together.
• Then apply the ReLU activation function. This has no name and no hyperparameters.

Arguments:
X - input tensor of shape (m, n_H_prev, n_W_prev, n_C_prev)
f - integer, specifying the shape of the middle CONV's window for the main path
filters - python list of integers, defining the number of filters in the CONV layers of the main path
stage - integer, used to name the layers, depending on their position in the network
block - string/character, used to name the layers, depending on their position in the network
s - Integer, specifying the stride to be used
Returns:
X - output of the convolutional block, tensor of shape (n_H, n_W, n_C)

def convolutional_block(X, f, filters, stage, block, s = 2):
# defining name basis
conv_name_base = 'res' + str(stage) + block + '_branch'
bn_name_base = 'bn' + str(stage) + block + '_branch'

# Retrieve Filters
F1, F2, F3 = filters

# Save the input value
X_shortcut = X

##### MAIN PATH #####
# First component of main path
X = Conv2D(F1, (1, 1), strides = (s,s), name = conv_name_base + '2a', kernel_initializer = glorot_uniform(seed=0))(X)
X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X)
X = Activation('relu')(X)

# Second component of main path
X = Conv2D(filters=F2, kernel_size=(f, f), strides=(1, 1), padding='same', name=conv_name_base + '2b', kernel_initializer=glorot_uniform(seed=0))(X)
X = BatchNormalization(axis=3, name=bn_name_base + '2b')(X)
X = Activation('relu')(X)

# Third component of main path
X = Conv2D(filters=F3, kernel_size=(1, 1), strides=(1, 1), padding='valid', name=conv_name_base + '2c', kernel_initializer=glorot_uniform(seed=0))(X)
X = BatchNormalization(axis=3, name=bn_name_base + '2c')(X)

##### SHORTCUT PATH ####
X_shortcut = Conv2D(F3, (1, 1), strides = (s,s), name = conv_name_base + '1', kernel_initializer = glorot_uniform(seed=0))(X_shortcut)
X_shortcut = BatchNormalization(axis = 3, name = bn_name_base + '1')(X_shortcut)

# Final step: Add shortcut value to main path, and pass it through a RELU activation
X = Activation('relu')(X)

return X

tf.reset_default_graph()
with tf.Session() as test:
A_prev = tf.placeholder("float", [3, 4, 4, 6])
X = np.random.randn(3, 4, 4, 6)
A = convolutional_block(A_prev, f = 2, filters = [2, 4, 6], stage = 1, block = 'a')
test.run(tf.global_variables_initializer())
out = test.run([A], feed_dict={A_prev: X, K.learning_phase(): 0})
print("out = ",out[0][1][1][0])

out =  [0.45051736 0.         0.8701941  0.         1.8204272  0.        ]

##### 3 - Building our first ResNet model (50 layers):

We now have the necessary blocks to build a very deep ResNet. The following figure describes in detail the architecture of this neural network. "ID BLOCK" in the diagram stands for "Identity block," and "ID BLOCK x3" means you should stack 3 identity blocks together.

The details of this ResNet-50 model are:

• Stage 1:
• The 2D Convolution has 64 filters of shape (7,7) and uses a stride of (2,2). Its name is "conv1".
• BatchNorm is applied to the channels axis of the input.
• MaxPooling uses a (3,3) window and a (2,2) stride.
• Stage 2:
• The convolutional block uses three set of filters of size [64,64,256], "f" is 3, "s" is 1 and the block is "a".
• The 2 identity blocks use three set of filters of size [64,64,256], "f" is 3 and the blocks are "b" and "c".
• Stage 3:
• The convolutional block uses three set of filters of size [128,128,512], "f" is 3, "s" is 2 and the block is "a".
• The 3 identity blocks use three set of filters of size [128,128,512], "f" is 3 and the blocks are "b", "c" and "d".
• Stage 4:
• The convolutional block uses three set of filters of size [256, 256, 1024], "f" is 3, "s" is 2 and the block is "a".
• The 5 identity blocks use three set of filters of size [256, 256, 1024], "f" is 3 and the blocks are "b", "c", "d", "e" and "f".
• Stage 5:
• The convolutional block uses three set of filters of size [512, 512, 2048], "f" is 3, "s" is 2 and the block is "a".
• The 2 identity blocks use three set of filters of size [512, 512, 2048], "f" is 3 and the blocks are "b" and "c".
• The 2D Average Pooling uses a window of shape (2,2) and its name is "avg_pool".
• The flatten doesn't have any hyperparameters or name.
• The Fully Connected (Dense) layer reduces its input to the number of classes using a softmax activation. Its name should be 'fc' + str(classes).

So we will implement the ResNet with 50 layers described in the figure above into following architecture:
CONV2D -> BATCHNORM -> RELU -> MAXPOOL -> CONVBLOCK -> IDBLOCK2 -> CONVBLOCK -> IDBLOCK3 -> CONVBLOCK -> IDBLOCK5 -> CONVBLOCK -> IDBLOCK2 -> AVGPOOL -> TOPLAYER

Arguments:
input_shape - shape of the images of the dataset
classes - integer, number of classes

Returns:
model - a Model() instance in Keras

def ResNet50(input_shape = (64, 64, 3), classes = 2):
# Define the input as a tensor with shape input_shape
X_input = Input(input_shape)

# Stage 1
X = Conv2D(64, (7, 7), strides = (2, 2), name = 'conv1', kernel_initializer = glorot_uniform(seed=0))(X)
X = BatchNormalization(axis = 3, name = 'bn_conv1')(X)
X = Activation('relu')(X)
X = MaxPooling2D((3, 3), strides=(2, 2))(X)

# Stage 2
X = convolutional_block(X, f = 3, filters = [64, 64, 256], stage = 2, block='a', s = 1)
X = identity_block(X, 3, [64, 64, 256], stage=2, block='b')
X = identity_block(X, 3, [64, 64, 256], stage=2, block='c')

# Stage 3
X = convolutional_block(X, f = 3, filters = [128, 128, 512], stage = 3, block='a', s = 2)
X = identity_block(X, 3, [128, 128, 512], stage=3, block='b')
X = identity_block(X, 3, [128, 128, 512], stage=3, block='c')
X = identity_block(X, 3, [128, 128, 512], stage=3, block='d')

# Stage 4
X = convolutional_block(X, f = 3, filters = [256, 256, 1024], stage = 4, block='a', s = 2)
X = identity_block(X, 3, [256, 256, 1024], stage=4, block='b')
X = identity_block(X, 3, [256, 256, 1024], stage=4, block='c')
X = identity_block(X, 3, [256, 256, 1024], stage=4, block='d')
X = identity_block(X, 3, [256, 256, 1024], stage=4, block='e')
X = identity_block(X, 3, [256, 256, 1024], stage=4, block='f')

# Stage 5
X = convolutional_block(X, f = 3, filters = [512, 512, 2048], stage = 5, block='a', s = 2)
X = identity_block(X, 3, [512, 512, 2048], stage=5, block='b')
X = identity_block(X, 3, [512, 512, 2048], stage=5, block='c')

# AVGPOOL.
X = AveragePooling2D((2, 2), name='avg_pool')(X)

# output layer
X = Flatten()(X)
X = Dense(classes, activation='softmax', name='fc' + str(classes), kernel_initializer = glorot_uniform(seed=0))(X)

# Create model
model = Model(inputs = X_input, outputs = X, name='ResNet50')

return model

Run the following code to build the model's graph. If your implementation is not correct you will know it by checking your accuracy when running model.fit(...) below.

model = ResNet50(input_shape = (ROWS, COLS, CHANNELS), classes = CLASSES)

Now we need to configure the learning process by compiling the model.

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

The model is now ready to be trained. Run the following cell to train your model on 100 epochs with a batch size of 64:

model.fit(X_train, Y_train, epochs = 100, batch_size = 64)

6002/6002 [=================] - 12s 2ms/step - loss: 0.0403 - acc: 0.9900
Epoch 99/100
6002/6002 [=================] - 13s 2ms/step - loss: 0.0148 - acc: 0.9947
Epoch 100/100
6002/6002 [=================] - 14s 2ms/step - loss: 0.0289 - acc: 0.9902

Let's see how this model (trained on 100 epochs) performs on the test set.

preds = model.evaluate(X_test, Y_test)
print ("Loss = " + str(preds[0]))
print ("Test Accuracy = " + str(preds[1]))

1000/1000 [=================] - 2s 2ms/step
Loss = 1.3275088095664977
Test Accuracy = 0.724

We can also print a summary of our model by running the following code:

model.summary()

For future use we can save our model to file:

model.save('ResNet50.h5')

Finally, we can run the code below to visualize our ResNet50:

plot_model(model, to_file='model.png')
SVG(model_to_dot(model).create(prog='dot', format='svg'))

What you should remember:

• Very deep "plain" networks don't work in practice because they are hard to train due to vanishing gradients.
• The skip-connections help to address the Vanishing Gradient problem. They also make it easy for a ResNet block to learn an identity function.
• There are two main type of blocks: The identity block and the convolutional block.
• Very deep Residual Networks are built by stacking these blocks together.

##### Conclusion:
• ResNet is a powerful backbone model that is used very frequently in many computer vision tasks
• ResNet uses skip connection to add the output from an earlier layer to a later layer. This helps it mitigate the vanishing gradient problem
• You can use Keras to load their pretrained ResNet 50 or use the code I have shared to code ResNet yourself.

Full tutorial code and cats vs dogs image data-set can be found on my GitHub page.