text:
**Contents**

**Cover**

**Half Title page**

**Title page**

**Copyright page**

**Preface**

**Part One: Overview and Motivation**

**Chapter One: Introduction to Monte Carlo Methods**

1.1 Historical origin of Monte Carlo simulation

1.2 Monte Carlo simulation vs. Monte Carlo sampling

1.3 System dynamics and the mechanics of Monte Carlo simulation

1.4 Simulation and optimization

1.5 Pitfalls in Monte Carlo simulation

1.6 Software tools for Monte Carlo simulation

1.7 Prerequisites

For further reading

References

**Chapter Two: Numerical Integration Methods**

2.1 Classical quadrature formulas

2.2 Gaussian quadrature

2.3 Extension to higher dimensions: Product rules

2.4 Alternative approaches for high-dimensional integration

2.5 Relationship with moment matching

2.6 Numerical integration in R

For further reading

References

**Part Two: Input Analysis: Modeling and Estimation**

**Chapter Three: Stochastic Modeling in Finance and Economics**

3.1 Introductory examples

3.2 Some common probability distributions

3.3 Multivariate distributions: Covariance and correlation

3.4 Modeling dependence with copulas

3.5 Linear regression models: A probabilistic view

3.6 Time series models

3.7 Stochastic differential equations

3.8 Dimensionality reduction

3.9 Risk-neutral derivative pricing

For further reading

References

**Chapter Four: Estimation and Fitting**

4.1 Basic inferential statistics in R

4.2 Parameter estimation

4.3 Checking the fit of hypothetical distributions

4.4 Estimation of linear regression models by ordinary least squares

4.5 Fitting time series models

4.6 Subjective probability: The Bayesian view

For further reading

References

**Part Three: Sampling and Path Generation**

**Chapter Five: Random Variate Generation**

5.1 The structure of a Monte Carlo simulation

5.2 Generating pseudorandom numbers

5.3 The inverse transform method

5.4 The acceptance-rejection method

5.5 Generating normal variates

5.6 Other ad hoc methods

5.7 Sampling from copulas

For further reading

References

**Chapter Six: Sample Path Generation for Continuous-Time Models**

6.1 Issues in path generation

6.2 Simulating geometric Brownian motion

6.3 Sample paths of short-term interest rates

6.4 Dealing with stochastic volatility

6.5 Dealing with jumps

For further reading

References

**Part Four: Output Analysis and Efficiency Improvement**

**Chapter Seven: Output Analysis**

7.1 Pitfalls in output analysis

7.2 Setting the number of replications

7.3 A world beyond averages

7.4 Good and bad news

For further reading

References

**Chapter Eight: Variance Reduction Methods**

8.1 Antithetic sampling

8.2 Common random numbers

8.3 Control variates

8.4 Conditional Monte Carlo

8.5 Stratified sampling

8.6 Importance sampling

For further reading

References

**Chapter Nine: Low-Discrepancy Sequences**

9.1 Low-discrepancy sequences

9.2 Halton sequences

9.3 Sobol low-discrepancy sequences

9.4 Randomized and scrambled low-discrepancy sequences

9.5 Sample path generation with low-discrepancy sequences

For further reading

References

**Part Five: Miscellaneous Applications**

**Chapter Ten: Optimization**

10.1 Classification of optimization problems

10.2 Optimization model building

10.3 Monte Carlo methods for global optimization

10.4 Direct search and simulation-based optimization methods

10.5 Stochastic programming models

10.6 Stochastic dynamic programming

10.7 Numerical dynamic programming

10.8 Approximate dynamic programming

For further reading

References

**Chapter Eleven: Option Pricing**

11.1 European-style multidimensional options in the BSM world

11.2 European-style path-dependent options in the BSM world

11.3 Pricing options with early exercise features

11.4 A look outside the BSM world: Equity options under the Heston model

11.5 Pricing interest rate derivatives

For further reading

References

**Chapter Twelve: Sensitivity Estimation**

12.1 Estimating option greeks by finite differences

12.2 Estimating option greeks by pathwise derivatives

12.3 Estimating option greeks by the likelihood ratio method

For further reading

References

**Chapter Thirteen: Risk Measurement and Management**

13.1 What is a risk measure?

13.2 Quantile-based risk measures: Value-at-risk

13.3 Issues in Monte Carlo estimation of V@R

13.4 Variance reduction methods for V@R

13.5 Mean–risk models in stochastic programming

13.6 Simulating delta hedging strategies

13.7 The interplay of financial and nonfinancial risks

For further reading

References

**Chapter Fourteen: Markov Chain Monte Carlo and Bayesian Statistics**

14.1 Acceptance–rejection sampling in Bayesian statistics

14.2 An introduction to Markov chains

14.3 The Metropolis–Hastings algorithm

14.4 A re-examination of simulated annealing

For further reading

References

**Index**
**Handbook in Monte Carlo Simulation**

Copyright © 2014 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey. All rights reserved.
Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at <http://www.wiley.com/go/permission>.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www.wiley.com.

**_Library of Congress Cataloging-in-Publication Data:_**

Brandimarte, Paolo.
Handbook in Monte Carlo simulation : applications in financial engineering, risk management, and economics / Paolo Brandimarte.
pages cm
Includes bibliographical references and index.
ISBN 978-0-470-53111-2 (cloth)
1. Finance—Mathematical models. 2. Economics—Mathematical models. 3. Monte Carlo method. I. Title. HG106.B735 2014
330.01′518282—dc23
2013047832
**Preface**

The aim of this book is to provide a wide class of readers with a low- to intermediate-level treatment of Monte Carlo methods for applications in finance and economics. The target audience consists of students and junior practitioners with a quantitative background, and it includes not only students in economics and finance, but also in mathematics, statistics, and engineering. In fact, this is the kind of audience I typically deal with in my courses. Not all of these readers have a strong background in either statistics, financial economics, or econometrics, which is why I have also included some basic material on stochastic modeling in the early chapters, which is typically skipped in higher level books. Clearly, this is not meant as a substitute for a proper treatment, which can be found in the references listed at the end of each chapter. Some level of mathematical maturity is assumed, but the prerequisites are rather low and boil down to the essentials of probability and statistics, as well as some basic programming skills.1 Advanced readers may skip the introductory chapters on modeling and estimation, which are also included as a reminder that no Monte Carlo method, however sophisticated, will yield useful results if the input model is flawed. Indeed, the power and flexibility of such methods may lure us into a false sense of security, making us forget some of their inherent limitations.

Option pricing is certainly a relevant application domain for the techniques we discuss in the book, but this is not meant to be a book on financial engineering. I have also included a significant amount of material on optimization in its many guises, as well as a chapter related to computational Bayesian statistics. I have favored a wide scope over a deeply technical treatment, for which there are already some really excellent and more demanding books. Many of them, however, do not quite help the reader to really "feel" what she is learning, as no ready-to-use code is offered. In order to allow anyone to run the code, play with it, and hopefully come up with some variations on the theme, I have chosen to develop code in R. Readers familiar with my previous book written in MATLAB might wonder whether I have changed my mind. I did not: I never use R in research or consulting, but I use it a lot for teaching. When I started writing the book, I was less than impressed by the lack of an adequate development environment, and some design choices of the language itself left me a bit puzzled. As an example, the * operator in MATLAB multiplies matrices row by column; whenever you want to work elementwise, you use the . operator, which has a clear and uniform meaning when applied to other operators. On the contrary, the operator * works elementwise in R, and row-by-column matrix product is accomplished by the somewhat baroque operator %*%. Furthermore, having to desperately google every time you have to understand a command, because documentation is a bit poor and you have to make your way in a mess of packages, may be quite frustrating at times. I have also found that some optimization functions are less accurate and less clever in dealing with limit cases than the corresponding MATLAB functions. Having said that, while working on the book, I have started to appreciate R much more. Also my teaching experience with R has certainly been fun and rewarding. A free tool with such a potential as R is certainly most welcome, and R developers must be praised for offering all of this. Hopefully, the reader will find R code useful as a starting point for further experimentation. I did _not_ assemble R code into a package, as this would be extremely misleading: I had no plan to develop an integrated and reliable set of functions. I just use R code to illustrate ideas in concrete terms and to encourage active learning. When appropriate, I have pointed out some programming practices that may help in reducing the computational burden, but as a general rule I have tried to emphasize clarity over efficiency. I have also avoided writing an introduction to R programming, as there are many freely available tutorials (and a few good books2). A reader with some programming experience in any language should be able to make her way through the code, which has been commented on when necessary. My assumption is that a reader, when stumbling upon an unknown function, will take advantage of the online help and the example I provide in order to understand its use and potentiality. Typically, R library functions are equipped with optional parameters that can be put to good use, but for the sake of conciseness I have refrained from a full description of function inputs.

### Book structure

The book is organized in five parts.

**1.** Part I, Overview and Motivation, consists of two chapters. Chapter 1 provides an introduction to Monte Carlo methods and applications. The different classes of dynamic models that are encountered in simulation are outlined, and due emphasis is placed on pitfalls and limitations of Monte Carlo methods. Chapter 2 deals with numerical integration methods. Numerical integration is quite relevant, as it provided most of the historical motivation to develop Monte Carlo methods; furthermore, there are cases in which one is much better off using good quadrature formulas than throwing random numbers around. Finally, framing Monte Carlo methods within numerical integration provides the necessary discipline to understand and properly use low-discrepancy sequences, sometimes referred to as quasi–Monte Carlo methods.

**2.** Part II, Input Analysis: Modeling and Estimation, is specifically aimed at students and newcomers, as it includes two introductory chapters dealing with stochastic model building (Chapter 3) and model fitting (Chapter 4). Essentially, in this part of the book we are concerned with the modeling of inputs of a Monte Carlo simulation. Many advanced books on Monte Carlo methods for finance skip and take for granted these concepts. I have preferred to offer a limited treatment for the sake of unfamiliar readers, such as students in engineering or practitioners without an econometrics background. Needless to say, space does not allow for a deep treatment, but I believe that it is important to build at least a framework for further study. In order to make this part useful to intermediate readers, too, I have taken each topic as an excuse for a further illustration of R functionalities. Furthermore, some more advanced sections may be useful to students in economics and finance as well, such as those on stochastic calculus, copulas, and Bayesian statistics.

**3.** Part III, Sampling and Path Generation, is more technical and consists of two chapters. In Chapter 5 we deal with pseudorandom number and variate generation. While it is certainly true that in common practice one takes advantage of reliable generators provided by software tools like R, and there is no need for an overly deep treatment, some basic knowledge is needed in order to select generators and to manage simulation runs properly. We also outline scenario generation using copulas. In Chapter 6 we deal with sample path generation for continuous-time models based on stochastic differential equations. This is an essential tool for any financial engineer and is at the heart of many derivative pricing methods. It is important to point out that this is also relevant for risk managers, insurers, and some economists as well.

**4.** Part IV, Output Analysis and Efficiency Improvement, looks at the final step of the simulation process. Monte Carlo methods are extremely powerful and flexible; yet, their output may not be quite reliable, and an unreasonable computational effort may be called for, unless suitable countermeasures are taken. Chapter 7 covers very simple, and possibly overlooked, concepts related to confidence intervals. Counterexamples are used to point out the danger of forgetting some underlying assumptions. Chapter 8 deals with variance reduction strategies that are essential in many financial engineering and risk management applications; indeed, the techniques illustrated here are applied in later chapters, too. Chapter 9 deals with low-discrepancy sequences, which are sometimes gathered under the quasi–Monte Carlo nickname. Actually, there is nothing stochastic in low-discrepancy sequences, which should be regarded as deterministic numerical integration strategies. For a certain range of problem dimensionality, they are a good alternative to pseudorandom sampling.

**5.** Part IV, Miscellaneous Applications, includes five more or less interrelated chapters dealing with:

  The interplay between optimization and Monte Carlo methods, including stochastic methods for deterministic global optimization, scenario generation for stochastic programming with recourse, and stochastic dynamic programming (Chapter 10)

  Option pricing, with an emphasis on variance reduction methods (Chapter 11)

  Obtaining sensitivity of performance measures with Monte Carlo simulation (Chapter 12)

  Risk measurement and management, with an emphasis on value-at-risk and related risk measures for financial risk management (Chapter 13)

  Markov chain Monte Carlo (MCMC) methods, which are relevant for different applications, most notably computational Bayesian statistics (Chapter 14)

There are some logical precedences among these final chapters, but they need not be read in a strict sequential order. Chapter 14 is independent of the others, and the only link is represented by the possibility of regarding simulated annealing, a stochastic approach for both global and combinatorial optimization, as an MCMC method. Stochastic dynamic programming, dealt with in Chapter 10, is needed to understand American-style option pricing in Chapter 11. Measuring option price sensitivity is used as a motivating example in Chapter 12, but the methods outlined there have a much more general applicability, as they can also be used within optimization procedures. Finally, there is certainly a natural link between option pricing and the risk management examples discussed in Chapter 13.

### Supplements and R code

The code has been organized in two files per chapter. The first one contains all of the function definitions and the reference to required packages, if any; this file should be sourced before running the scripts, which are included as chunks of code in a second file. An archive including all of the R code will be posted on a webpage. My current URL is

  http://staff.polito.it/paolo.brandimarte/

A hopefully short list of errata will be posted there as well. One of the many corollaries of Murphy's law states that my URL is going to change shortly after publication of the book. An up-to-date link will be maintained on the Wiley webpage:

  http://www.wiley.com/

For comments, suggestions, and criticisms, all of which are quite welcome, my e-mail address is

  paolo.brandimarte@polito.it

PAOLO BRANDIMARTE

_Turin, February 2014_

1In case of need, the mathematical prerequisites are covered in my other book: _Quantitative Methods: An Introduction for Business Management._ Wiley, 2011.

2Unfortunately, I have also run into very bad books using R; hopefully, this one will not contribute to the list.

# Part One

# Overview and Motivation

# Chapter One

# Introduction to Monte Carlo Methods

The term _Monte Carlo_ is typically associated with the process of modeling and simulating a system affected by randomness: Several random scenarios are generated, and relevant statistics are gathered in order to assess, e.g., the performance of a decision policy or the value of an asset. Stated as such, it sounds like a fairly easy task from a conceptual point of view, even though some programming craft might be needed. Although it is certainly true that Monte Carlo methods are extremely flexible and valuable tools, quite often the last resort approach for overly complicated problems impervious to a more mathematically elegant treatment, it is also true that running a _bad_ Monte Carlo simulation is very easy as well. There are several reasons why this may happen:

  We are using a wrong model of uncertainty:

– Because we are using an unrealistic probability distribution

– Or because we are missing some link among the underlying risk factors

– Or because some unknown parameters have been poorly estimated

– Or because the very nature of uncertainty in our problem does not lend itself to a stochastic representation

  The output estimates are not reliable enough, i.e., the estimator variance is so large that a much larger sample size is required.

  There is a systematic error in the estimates, which could be low or high biased.

  The way we generate sample paths, possibly discretizing a continuous-time model, induces a non-negligible error.

  We are using poor random variate generators.

  There is some possibly subtle bug in the computer program implementing the method.

Some of these issues are technical and can be addressed by the techniques that we will explore in this book, but others are more conceptual in nature and point out a few intrinsic and inescapable limitations of Monte Carlo methods; it is wise not to forget them, while exploring the richness and power of the approach.

The best countermeasure, in order to avoid the aforementioned pitfalls, is to build reasonably strong theoretical foundations and to gain a deep understanding of the Monte Carlo approach and its variants. To that end, a good first step is to frame Monte Carlo methods as a numerical integration tool. Indeed, while the term _simulation_ sounds more exciting, the term Monte Carlo _sampling_ is often used. The latter is more appropriate when we deal with Monte Carlo sampling as a tool for numerical integration or statistical computing. Granted, the idea of simulating financial markets is somewhat more appealing than the idea of computing a multidimensional integral. However, a more conceptual framework helps in understanding powerful methods for reducing, or avoiding altogether, the difficulties related to the variance of random estimators. Some of these methods, such as low-discrepancy sequences and Gaussian quadrature, are actually deterministic in nature, but their understanding needs a view integrating numerical integration and statistical sampling.

In this introductory chapter, we consider first the historical roots of Monte Carlo; we will see in Section 1.1 that some early Monte Carlo methods were actually aimed at solving deterministic problems. Then, in Section 1.2 we compare Monte Carlo sampling and Monte Carlo simulation, showing their deep relationship. Typical simulations deal with dynamic systems evolving in time, and there are three essential kinds of dynamic models:

  Continuous-time models

  Discrete-time models

  Discrete-event models

These model classes are introduced in Section 1.3, where we also illustrate how their nature affects the mechanics of Monte Carlo simulation. In this book, a rather relevant role is played by applications involving optimization. This may sound odd to readers who associate simulation with performance evaluation; on the contrary, there is a multiway interaction between optimization and Monte Carlo methods, which is outlined in Section 1.4.

In this book we illustrate a rather wide range of applications, which may suggest the idea that Monte Carlo methods are almost a panacea. Unfortunately, this power may hide many pitfalls and dangers. In Section 1.5 we aim at making the reader aware of some of these traps. Finally, in Section 1.6 we list a few software tools that are commonly used to implement Monte Carlo simulations, justifying the choice of R as the language of this book, and in Section 1.7 we list prerequisites and references for readers who may need a refresher on some background material.

# 1.1 Historical origin of Monte Carlo simulation

Monte Carlo methods involve random sampling, but the actual aim is to estimate a deterministic quantity. Indeed, a well-known and early use of Monte Carlo-like methods is Buffon's needle approach to estimate π.1 The idea, illustrated in Fig. 1.1, is to randomly throw _n_ times a needle of length _l_ on a floor consisting of wood strips of width _t_ > _l_ , and to observe the number of times _h_ that the needle crosses the border between two strips. Let _X_ be the distance from the center of the needle to the closest border; _X_ is a uniform random variable on the range [0, _t_ /2], and its probability density function is 2/ _t_ on that interval and 0 outside.2 By a similar token, let θ be the acute angle between the needle and that border; this is another uniformly distributed variable, on the range [0, π/2], with density 2/π on that support. Using elementary trigonometry, it is easy to see that the needle will cross a border when

**FIGURE 1.1** **An illustration of Buffon's needle**.

The probability of this event is found by integrating the joint density of the random variables _X_ and θ over the relevant domain. Assuming independence, the joint density is just the product of the two marginals. Therefore, we have to compute a double integral as follows:

If we want to estimate π, we can use a sample estimate of this probability, i.e., the ratio of _h_ over _n_ :

We see that the original problem is purely deterministic, but we use Monte Carlo sampling as an estimation tool.

A slightly more practical approach, taking advantage of R, is illustrated in the code of Fig. 1.2. The idea is shooting _n_ random bullets on the square [−1, 1] × [−1, 1] and evaluating the fraction _h/n_ of points falling inside the unit circle _x_ 2 \+ _y_ 2 = 1. Since the ratio between the area of the circle and the area of the square is π/4, we have

**FIGURE 1.2** **Using R to estimate** π.

Running the code,3 we obtain

    > set.seed(55555)
    > estPi(1000)
    [1] 3.132
    > estPi(1000)
    [1] 3.14
    > estPi(1000)
    [1] 3.084
    > estPi(1000000)
    [1] 3.141388
    > pi
    [1] 3.141593

Clearly, a huge sample size is needed to obtain a fairly accurate estimate, and this is certainly not the smartest way to estimate π.

In more recent times, in the period between 1930 and 1950, Monte Carlo methods were used by the likes of Enrico Fermi, Stanislaw Ulam, and John von Neumann to solve problems related to physics. Indeed, such methods were used in the 1950s when the hydrogen bomb was developed at the Los Alamos laboratories. It was then that the term _Monte Carlo_ was coined after the well-known gambling house. Then, luckily enough, the idea moved to other domains, including operations research and economics. At present, Monte Carlo methods are extremely widespread in many domains, including risk management and financial engineering.

# 1.2 Monte Carlo simulation vs. Monte Carlo sampling

The terms _simulation_ and _sampling_ are both used with reference to Monte Carlo methods. Indeed, there is no 100% sharp line separating the two concepts, and a couple of examples are the best way to get the point.

###   Example 1.1 Shortfall probability in wealth management

Consider an investor who allocates her wealth _W_ 0 to _n_ risky financial assets, such as stock, bonds, derivatives, and whatnot. At time _t_ = 0, the asset price of asset _i, i_ = 1,..., _n_ , is _P_ _i_ 0, and the (integer) number of shares of assets _i_ in the portfolio is _h_ _i_. Therefore, the initial wealth is

The portfolio is held up to time _t_ = _T_ , and the price of asset _i_ at the end of this investment horizon depends on a set of underlying risk factors, such as the term structure of interest rates, spreads due to credit risk, inflation, oil price, the overall state of the economy, etc. Let X be a vector random variable, with joint density _f_ x(x), representing these factors. If we assume that the price of each asset at time _T_ is given by a function of the underlying factors,

then the terminal wealth is a function of X,

which is itself a random variable. On the one hand, we are interested in the expected value of _W_ _T_. However, since risk is a quite relevant issue, we might also be interested in the shortfall probability with respect to a target wealth _H_ :

Whatever the case, we are lead to the expected value of some function _g_ (X):

(1.1)

If we choose _g_ (X) = _W_ _T_ we obtain the expected value of future wealth; if we choose the indicator function of the event { _W_ _T_ < _H_ }, i.e.,

then we obtain the shortfall probability. Other risk measures can be selected and will be discussed in Chapter 13. Evaluating the multidimensional integral in Eq. (1.1) is quite difficult, even when we know the analytical form of the involved functions. As we shall see, by using Monte Carlo we may draw a sample of _m_ observations X _k_ , _k_ = 1,..., _m_ , and estimate the quantities of interest. We are using random sampling as a tool for numerical integration.

In the example above, no simulation is involved at all, assuming that we are able to sample directly the random variable **X** given its density. In other cases, we may have to cope with a more complicated dynamic model that describes the dynamics of the underlying risk factors over the time interval [0, _T_ ]. In such a case, we may not be able to sample risk factors directly, and we have to generate a set of sample paths. Dynamic models may be represented by continuous-time stochastic differential equations4 or by relatively simple discrete-time processes as in the following example.

###   Example 1.2 An autoregressive process

Time series models are quite often used in financial econometrics to describe the evolution of a quantity of interest over time. A simple example of this class of models is an autoregressive (AR) process of order 1 like

(1.2)

where _X_ 0 is given and  _t_ is the driving stochastic process, i.e., a sequence of random variables  _t_ , _t_ = 1, 2, 3,.... In the simplest models, this noise term consists of a sequence of mutually independent and identically distributed normal random variables. We use the notation _X_ ~ N(μ, σ2) to state that _X_ is a normal random variable with expected value μ and variance σ2. Unfolding the recursion, we immediately see that

which implies

The process described by Eq. (1.2) is very easy to analyze, but it is not the most general AR process of order 1. A more general example is

whose properties critically depend on the coefficient α (see Example 1.4). In other cases, we may have a system of such equations, with mutually correlated random driving factors and a more complicated structure, preventing us from obtaining the probability distribution of the variable of interest in an explicit form. In such a case, we may have to simulate the dynamical system.

On the basis of the above examples, we may argue that the term _sampling_ could be reserved to those cases in which there is no dynamics to be simulated over time, whereas _simulation_ entails the generation of sample paths. We may need to generate sample paths because the dynamical model is complicated and prevents an analytical solution, or because it is important to check the underlying variables over time. For instance, when pricing a European-style option,5 we just need to sample the underlying variable at the option maturity. In the easy case of geometric Brownian motion (GBM for short) this boils down to sampling a lognormal random variable, but we have to resort to sample path generation when we step outside that safe and comforting domain, and we enter the realm populated by stochastic volatilities and price jumps. On the contrary, when we are pricing an American-style option, featuring early exercise opportunities, we need a whole sample path, whatever price model we adopt.

As is usually the case, also the above distinction is not so clear-cut. In both settings we are actually evaluating a possibly high-dimensional integral of a function, in order to estimate a probability or an expectation. The function may be given in analytical form, or it may be defined by a possibly complicated black box, requiring several lines of code to be evaluated. Conceptually, there is no difference. Indeed, numerical integration plays such a pivotal role for a full appreciation of Monte Carlo methods that we will devote the whole of Chapter 2 to it.

# 1.3 System dynamics and the mechanics of Monte Carlo simulation

When we have to describe the dynamic evolution of a system over time, we typically resort to one of the following model classes:

  Discrete-time models

  Continuous-time models

  Discrete-event models

From a technical point of view, as usual, the distinction is not sharp. For instance, even if we choose a continuous-time model, we have to discretize it in some way for the sake of computational feasibility. This implies that we actually simulate a discrete-time approximation, but given the issues involved in sample path generation, which involves nontrivial concepts related to the numerical solution of stochastic differential equations, it is better to stick to the classification above. It is also possible to create somewhat hybrid models, such as stochastic processes including both a diffusion component, which is continuous, and jumps, which are related to discrete-event dynamics.

## 1.3.1 DISCRETE-TIME MODELS

In discrete-time models we assume that the time horizon [0, _T_ ] is discretized in time intervals (time buckets) of width δ _t_. In principle, nothing forbids nonuniform time steps, which may actually be required when dealing with certain financial derivatives; nevertheless, we usually stick to the uniform case for the sake of simplicity. The system is actually observed only at time instants of the form

where _T_ = _M_ δ _t._ What happens between two discrete-time instants is not relevant. Typically, we forget about the time bucket size δ _t_ , which could be a day, a week, or a month, and we use discrete-time subscripts, like _t_ = 0, 1, 2,..., directly.

###   Example 1.3 Cash on hand

Consider the cash management problem for a retail bank. During each day _t_ , we observe cash inflows _w_ _t_ + and cash outflows _w_ _t_ −, corresponding to cash deposits and withdrawals by customers, respectively. Hence, if we denote by _H_ _t_ the cash holding at the end of day _t_ , we have the dynamic equation

Such a model makes sense if we are only interested in the balance at the end of each day.

The AR model of Example 1.2 is a very basic example of time series models. From a technical point of view, simulating a time series model is usually no big deal.

###   Example 1.4 Simulating a simple AR process

Let us consider the following autoregressive process:

where  _t_ ~ N(0, σ2). The R function in Fig. 1.3 can be used for its simulation. In the specific case

(1.3)

we run the function as follows:

    > set.seed(55555)
    > AR(40,.8, 8, 1, 50)

obtaining the plot in Fig. 1.4.

**FIGURE 1.3** **R code to simulate a simple autoregressive process.**

**FIGURE 1.4** **Sample path of the AR process ofEq. (1.3).**

In the above examples we had the purely random evolution of a system. In a more general setting, we may have the possibility of influencing, i.e., of partially controlling its dynamics. For instance, in discrete-time stochastic optimization models, we deal with a system modeled by the state transition equation

(1.4)

where

  s _t_ is a vector of state variables, observed at the end of time period _t._

  x _t_ is a vector of control variables, i.e., decisions made _after_ observing the state s _t_.

   _t_ +1 is a vector of disturbances, realized _after_ we apply the control variable x _t_.

If the sequence of disturbances  _t_ is intertemporally independent, we obtain a Markov process. Markov processes are relatively easy to represent, simulate, and control. We should also observe that the state need not be a continuous variable. We may have a system with discrete states, resulting in a discrete-time Markov chain.6 As an example, we may consider a Markov chain modeling the migration in the credit rating of a bond issue. Sometimes, we use a discrete-state Markov chain to approximate a continuous-state system by aggregation: We partition the state space with a mutually exclusive and collectively exhaustive collection of subsets and identify each one with a single representative value.

## 1.3.2 CONTINUOUS-TIME MODELS

The typical representation of a continuous-time model relies on differential equations. We are all familiar with the differential equation _F_ = _ma_ from elementary physics, where _F_ is force, _m_ is mass, and _a_ is acceleration, i.e., the second-order derivative of position with respect to time. As a more financially motivated example, imagine that you deposit an amount of cash _B_ (0) = _B_ 0, at time _t_ = 0, in a safe bank account. If _r_ is the _continuously compounded_ interest rate on your deposit,7 the evolution in time of wealth over time can be represented as

(1.5)

whose solution, given the initial condition _B_ (0) = _B_ 0, is

If the interest rate is not constant, and it is given by _r_ ( _t_ ), we obtain

(1.6)

This solution is easily obtained by rewriting Eq. (1.5) in the differential form

which, by recalling the derivative of the natural logarithm, can be transformed to

Integrating on the interval [0, _t_ ] and taking the exponential to get rid of the logarithm immediately yields Eq. (1.6). In less lucky cases, we have to resort to numerical methods, which typically yield a discrete-time approximation of the continuous-time system.

Monte Carlo methods come into play when randomness is introduced into the differential equation, yielding a stochastic differential equation. A well-known model in this vein is the GBM

(1.7)

which is arguably the most elementary model used to represent the random evolution of stock prices. With respect to the previous case, we have an additional term _W_ ( _t_ ), which is a Wiener process. We will deal later with the underpinnings of this kind of stochastic process,8 but for now suffice to say that the increment of the Wiener process over a finite time interval [ _t,t_ \+ δ _t_ ] is a normal random variable:

Loosely speaking, the differential _dW_ ( _t_ ) is the limit of this increment for δ _t_ → 0. Thus, we might consider simulating random sample paths of GBM by a straightforward discretization scheme:

This is called the Euler discretization scheme, and it immediately leads to the discrete-time process

where  ( _t_ \+ δ _t_ ) ~ N(0, 1). This kind of process, in terms of simulation, is not unlike an AR process and is easy to deal with. Unfortunately, this rather naive discretization suffers from a significant inconvenience: We obtain the wrong probability distribution. Indeed, it is easy to see that we generate normally distributed prices; therefore, at least in principle, we may generate a negative price, which does not make sense for limited liability assets like stock shares.

Using tools from stochastic calculus, we will see that we may solve the stochastic differential equation exactly and find an exact discretization:

(1.8)

This results in a lognormal distribution, ruling out negative prices. We also notice that μ is related to the drift of the process, i.e., its average rate of change over time, whereas σ plays the role of a volatility.

###   Example 1.5 Simulating geometric Brownian motion

The R code in Fig. 1.5 can be used to generate sample paths of GBM. The function returns a matrix whose rows correspond to numRepl sample paths starting from initial state s0, over numSteps time steps from 0 to _T_ , for a GBM with parameters μ and σ. The R snapshot of Fig. 1.6 yields the plots in Fig. 1.7. We immediately appreciate the role of the volatility σ.

**FIGURE 1.5** **R code to generate sample paths of geometric Brownian motion.**

**FIGURE 1.6** **R script to plot GBM sample paths for different volatilities.**

**FIGURE 1.7** **Sample paths of geometric Brownian motion for different levels of volatility.**

This solution is exact in the sense that there is no discretization error in the sample path generation; we are left with sampling errors that are typical of Monte Carlo methods. In other cases, things are not that easy: Even though, in the end, continuous-time models may boil down to discrete-time models, they do have some peculiarities, which may be hard to deal with.

## 1.3.3 DISCRETE-EVENT MODELS

The GBM sample paths depicted in Fig. 1.7 look noisy enough, but continuous. Indeed, it can be shown that they are continuous, in a stochastic sense, but nondifferentiable everywhere. The sample paths of a discrete-event dynamic system are quite different in nature. It is important to avoid a possible confusion between _discrete event_ and _discrete time._ Discrete-event systems are continuous-time models and, unlike discrete-time models, there is no constant time step δ _t_. Like discrete-time systems, the state changes only at some time instants, corresponding to the occurrence of some event, but the time between events is in general a continuous random variable.

Perhaps, the simplest discrete-event model, to be further discussed in Section 3.2.2, is the Poisson process. This is an instance of a counting process: The value _N_ ( _t_ ) counts the number of events that occurred during the time interval [0, _t_ ], where the times _X_ _k_ elapsing between events _k_ − 1 and _k, k_ = 1, 2, 3,..., are a sequence of independent and identically distributed exponential variables. By convention, _X_ 1 is the time at which the first event occurs after the start time _t_ = 0. A sample path of a Poisson process is illustrated in Fig. 1.8; we see that the process "jumps" whenever an event occurs, so that sample paths are piecewise constant. The underlying exponential distribution is characterized by a single parameter λ that can be interpreted as a "rate," i.e., the average number of events occurring per unit time.9 This basic Poisson process can be generalized by allowing for a nonconstant rate λ( _t_ ), which yields an inhomogeneous Poisson process. We may also allow for arbitrary jump sizes. If we associate a continuous probability distribution with the jump size, we obtain a continuous-state system, whereas unit jumps yield a discrete state space corresponding to integer numbers. This second generalization is called compound Poisson process, which is a discrete-event, continuous-state model.

**FIGURE 1.8** **Sample path of the Poisson process.**

Financially motivated examples of such models are:

  Models of stock prices allowing for jumps. Pure-jump models have been proposed, but we may also integrate diffusion processes, like GBM, with jumps. Stochastic processes in this more general form are known as Lévy processes.

  Models of credit rating migration, where the rating of a bond issue changes across a finite set of states at random times. This is a discrete-event, discrete-state model. Continuous-time Markov chains are an example in this family.

Discrete-event models are the rule in other application domains, like the simulation of manufacturing systems and logistic networks.10 These kinds of simulation problems are, in a sense, dual to the simulation problems faced in economics and finance: The mathematics involved is generally trivial, but there is an incredible complexity in data bookkeeping, resulting in huge computer programs, unless one resorts to a special-purpose simulation tool. On the contrary, when dealing with a stochastic differential equation, the program often consists of just a few lines of code, possibly hiding a quite sophisticated mathematical complexity. In the next section we give a simple example of a discrete-event system, so that the reader can appreciate the mechanics involved.

## 1.3.3.1 Simulation of discrete-event systems: a single queue

Let us consider a very simple system consisting of a single server (e.g., a bank teller), which must satisfy a set of incoming requests for service (e.g., customers entering a bank). The time between arrivals of consecutive customers is random; we might assume that it is a plain Poisson process or some time-varying process. The time needed to serve each request is random as well. The system dynamics is not difficult to describe:

  If the arriving customer finds the server idle, it starts her service immediately and will free the resource after service completion; the server state changes from idle to busy.

  If the server is idle, the new customer joins a queue of requests waiting for service. The simplest queuing discipline is first-in, first-out (FIFO), i.e., requests are handled according to the order of their arrivals.

  When the server terminates a service, it starts servicing the first customer in the queue, if any; otherwise, its state changes from busy to idle.

In order to describe the state of this system, we would certainly use _L_ ( _t_ ) = 0, 1, 2,..., the number of customers in the system, which includes the one being served, if any, and the state of the server, _S_ ( _t_ )   {0, 1}, where we assume that 1 corresponds to busy and 0 to idle. We also need some information about the future departure of the currently served customer, if any. A sample path of this single-server queuing system is depicted in Fig. 1.9. Statistics that we might be interested to collect are the average queue length or the average server utilization, which are essentially the integrals of the sample paths in Fig. 1.9 divided by the simulation time horizon. We immediately see that some bookkeeping is required, as these are not the familiar sample means, but rather integrals over time. A performance measure that can be evaluated as a sample mean is the average customer waiting time.

**FIGURE 1.9** **Sample path of queuing system consisting of a single server.**

In some specific cases, there is no need to resort to simulation, as the system can be analyzed mathematically. For instance, if the service and the interarrival times are both exponentially distributed, the system is essentially a continuous-time Markov chain, thanks to the memoryless property of the exponential distribution.11 Such a queue is denoted by _M/M_ /1, where the _M_ refers to the memoryless nature of the two stochastic processes involved, customer arrivals and customer service, and the 1 refers to a system consisting of a single server. If we denote the arrival and service rates by λ and μ, it can be shown that

(1.9)

where ρ = λ/μ is the utilization rate; clearly, stability requires μ > λ, i.e., the service rate, i.e., the average number of requests served per unit time, must be larger than the arrival rate. In more complicated cases, typically involving network of servers, or more difficult distributions, or different queuing disciplines, one may be forced to resort to simulation.

Let us denote by _A_ _j_ , _S_ _j_ , and _W_ _j_ the arrival time, service time, and waiting time, respectively, for customer _j._ It is possible to find a recursive equation for the waiting time of customer _j._ Customer _j_ has to wait if it arrives before the completion time of customer _j_ − 1. Denoting the latter quantity by _C_ _j_ −1, we have

(1.10)

However, the completion time of customer _j_ − 1 is

Plugging this into Eq. (1.10), we find

(1.11)

where _I_ _j_ ≡ _A_ _j_ − _A_ _j_ −1 is the interarrival time between customers ( _j_ − 1) and _j_. We note that, in a more general setting possibly involving priorities, balking, or multiple servers, complicated data bookkeeping would be needed to track customer waiting times, requiring suitable data structures and a corresponding programming effort. In this lucky case, it is easy to implement the recursion in R, as shown in Fig. 1.10. In Fig. 1.11 we also show an R script running the simulator in order to check the results against the exact formula of Eq. (1.9). With ρ = 90.90%, which is a fairly high value for the utilization rate, we observe a lot of variability around the correct value:

**FIGURE 1.10** **Implementation of Lindley's recursion for a _M/M_ /1 queue.**

**FIGURE 1.11** **Script to check the function** MM1_Queue ().

    > rho/mu/(1-rho)
    [1] 9.090909
    > MM1_Queue(lambda, mu, 100000)
    [1] 9.026053
    > MM1_Queue(lambda,mu,100000)
    [1] 10.34844
    > MM1_Queue(lambda,mu,100000)
    [1] 8.143615

For a low utilization rate, such as ρ = 50%, we get more reliable results:

    > rho/mu/(1-rho)
    [1] 0.5
    > MM1_Queue(lambda,mu, 100000)
    [1] 0.497959
    > MM1_Queue(lambda,mu,100000)
    [1] 0.5044687
    > MM1_Queue(lambda,mu,100000)
    [1] 0.4961641

We need to quantify the reliability of the above estimates, possibly by confidence intervals, as we show in Chapter 7 on simulation output analysis.

# 1.4 Simulation and optimization

There is a considerable interplay between Monte Carlo methods and optimization problems. A generic finite-dimensional optimization problem can be stated as

(1.12)

where x    _n_ is an _n_ -dimensional vector collecting the decision variables, _S_ ⊆  _n_ is the feasible set, and _f_ is the objective function. The feasible set _S_ is usually represented by a collection of equalities and inequalities. In recent years, there has been an astonishing progress in the speed and reliability of commercial software tools to solve wide classes of optimization problems. As a general rule that we shall illustrate in more detail in Chapter 10, convex optimization problems, whereby the function _f_ and the set _S_ are both convex, can be efficiently solved. In particular, large-scale linear programming problems can be solved in a matter of seconds or minutes. Other families of convex problems, such as second-order cone programming models, can be solved rather efficiently by relatively recent interior-point methods.

The joint progress in algorithmic sophistication and hardware speed makes optimization modeling a more and more relevant tool for many practical problems, including those motivated by economical or financial applications. Nevertheless, there are classes of problems that still are a very hard nut to crack. They include the following:

**1.** Nonconvex problems, i.e., either problems with a nonconvex objective function featuring many local optima, or problems with a nonconvex feasible set, like integer programming and combinatorial optimization problems

**2.** Stochastic optimization problems involving uncertainty in the problem data

**3.** Dynamic decision problems affected by uncertainty, which preclude the determination of an optimal static solution, but on the contrary require a dynamic strategy that adapts decisions to the unfolding of uncertainty

We will see much more on this kind of models, and the related solution algorithms, in Chapter 10. Here we just want to point out that Monte Carlo methods play a key role in tackling these problems, which we also illustrate in the next subsections.

## 1.4.1 NONCONVEX OPTIMIZATION

In a minimization problem with no constraints on the decision variables, if the objective function is convex, then a locally optimal solution is also a global one. This property considerably helps in devising solution algorithms. Equivalently, maximizing a concave function is relatively easy. This is good news for an economist wishing to solve a decision problem featuring a concave utility function or a financial engineer minimizing a convex risk measure. However, there are cases in which such a nice condition is not met, and the need for global optimization methods arises. The following example illustrates a typical function featuring a large number of local optima.

###   Example 1.6 The Rastrigin function

The Rastrigin function is a sort of benchmark to test global optimization algorithms, as well as to illustrate how bad an objective function can be. In the general _n_ -dimensional case, the Rastrigin function is defined as

where _A_ is some constant. In a bidimensional case, if we choose _A_ = 10, we may boil down the function to

depending on the two variables _x_ and _y._ A surface plot of the Rastrigin function is displayed in Fig. 1.12, where its nonconvexity can be appreciated. Another view is offered by the contour plot of Fig. 1.13. These two plots have been obtained by the R script in Fig. 1.14.

**FIGURE 1.12** **Surface plot of Rastrigin function.**

**FIGURE 1.13** **Contour plot of Rastrigin function.**

**FIGURE 1.14** **Script to produce the plots of the Rastrigin function.**

The Rastrigin function does illustrate why nonconvexities make an optimizer's life difficult, but it definitely looks artificial. To see how nonconvexities may arise in a practical context, let us consider model calibration in asset pricing. Suppose that we have a model for the price of _n_ assets indexed by _k, k_ = 1,..., _n_. The pricing function

predicts the price _ __k_ depending on some specific characteristics of the asset _k_ itself, and a set of general parameters β _j_ , _j_ = 1,..., _m_ , common to several assets, where _m_ << _n._ Model calibration requires finding a set of parameters that fit as much as possible a set of observed prices _P_ _k_ _o_. After model calibration, we can move on and price other assets in an internally consistent way. To be concrete, an option pricing model is calibrated on the basis of quoted prices of exchange-traded derivatives, and it can then be used to price an over-the-counter derivative for which a market quote is not available. The model can be used, e.g.,

  By the client of an investment bank to check if the price proposed by the bank for a specifically tailored contract makes sense

  By an arbitrageur to spot price inconsistencies paving the way for arbitrage opportunities

  By a risk manager to predict the impact of changes in the underlying factors on the value of a portfolio

A standard way to tackle the problem is by minimizing the average squared distance between the observed price _P_ _k_ _o_ and the theoretical price _ __k_ predicted by the model, which leads to the nonlinear least-squares problem:

(1.13)

(1.14)

where β is a vector collecting the parameters and _S_ is a feasible set representing restrictions on parameters, such as non-negativity. While ordinary least-squares problems are trivial to solve, in this case the pricing functions may be nonlinear; hence, there is no guarantee in general that the resulting problem is convex, and a local solver may well get trapped in a locally optimal solution yielding a set of unreasonable parameters. A similar consideration applies in parameter estimation, when a complicated likelihood function is maximized.12

Several global optimization algorithms have been proposed, and some of them do guarantee the global optimality of the reported solution. Unfortunately, these algorithms are typically restricted to a specific class of functions, and the optimality guarantee may require demanding computations. As an alternative, if we are not able to exploit specific structural properties of the problem, and a near-optimal solution is enough, possibly the optimal one but with no proof of its status, we may resort to general-purpose stochastic search algorithms. Among the proposed classes of methods, we mention:

  Pure random sampling of points in the feasible domain

  Using a local optimizer along with random multistart strategies

  Using strategies to escape from local optima, like simulated annealing

  Using strategies based on the evolution of a population of solutions, like genetic algorithms and particle swarm optimization

These methods may also be applied when the complementary kind of nonconvexity arises, i.e., we have to deal with a nonconvex set of feasible solutions, as is the case with combinatorial optimization problems, where we have to explore a finite, but huge discrete set of solutions. The above methods come in a possibly confusing assortment of variants, also including hybrid strategies. The effectiveness of each approach may actually depend on some problem features, most notably the presence or absence of difficult constraints. However, all of them rely on some random exploration behavior, based on Monte Carlo sampling.

## 1.4.2 STOCHASTIC OPTIMIZATION

Most financial problems involve some form of uncertainty. Hence, we may have to tackle an optimization problem where some data defining the objective function or the feasible set are uncertain. We face the difficult task of choosing an optimal solution x* in _S_ before knowing the value of a vector random variable ξ taking values on  . If so, two questions arise:

  Is x* robust with respect to optimality? In other words, what can we say about the quality of the solution across possible realizations of ξ?

  Is x* robust with respect to feasibility? This is an even thornier issue, as the solution we choose might be infeasible for some particularly unlucky scenarios. For instance, we may have solvency issues for a pension fund.

For now, let us focus only on the first issue and consider a problem of the form

(1.15)

The objective function _f_ (x, ξ) depends on both decisions x and random factors ξ, but by taking an expectation with respect to ξ we obtain a deterministic function _F_ (x). Assuming that the components of ξ are jointly continuous and distributed according to a density function _g_ ξ(y), with support  , the objective function can be written as

As we point out in Chapter 2, such a multidimensional integral can be extremely difficult to evaluate for a given x, let alone to optimize in general, even when we are able to prove that _F_ (x) enjoys such nice properties as continuity, differentiability, or even convexity.

A possible approach is to boil down the integral to a sum, by sampling _K_ scenarios characterized by realizations ξ _k_ and probabilities π _k_ , _k_ = 1,..., _K._ Then, we solve

(1.16)

which is a more familiar optimization problem. We note that Monte Carlo sampling may play a key role in optimization as a scenario generation tool. However, _crude_ sampling is typically not a viable solution and the following issues must be tackled:

  How can we generate a limited number of scenarios, in order to capture uncertainty without having to solve a huge and intractable problem?

  How can we prove convergence, i.e., that when the sample size goes to infinity, the solution of problem (1.16) converges to the true optimal solution of problem (1.15)?

  The previous question is actually theoretical in nature and definitely beyond the scope of this book. A more practical, but obviously related question is: How can we assess the quality of the solution for a finite sample size?

  How can we assess the robustness of the solution with respect to sampling variability and, possibly, with respect to higher level uncertainty on the parameters defining the uncertainty model itself?

Some of these hard questions will be addressed in this book, when we deal with variance reduction strategies, numerical integration by Gaussian quadrature, and deterministic sampling based on low-discrepancy sequences. If all this does not sound challenging enough, recall that we may also have feasibility issues. What can we do if a solution proves to be infeasible after we observe the true value of the uncertain parameters? More generally, what if the optimization problem is dynamic and we have to make a _sequence_ of decisions while we gather information step by step? One approach to cope with these multistage stochastic problems relies on multistage stochastic programming with recourse, discussed in Chapter 10; sometimes, they can also be tackled by stochastic dynamic programming.

## 1.4.3 STOCHASTIC DYNAMIC PROGRAMMING

In the previous section we have considered how uncertainty issues may affect an optimization problem, but we disregarded a fundamental point: the dynamics of uncertainty over time. In real life, when making decisions under uncertainty, we do start with a plan, but then we adjust our decisions whenever new pieces of information are progressively discovered. Rather than coming up with a plan to be rigidly followed over time, we pursue a dynamic strategy in which decisions are adapted on the basis of contingencies. This kind of consideration suggests the opportunity of formulating multistage, dynamic stochastic optimization models, whereby we have to find a sequence of decisions x _t_ , _t_ = 0, 1,..., _T_ , which are not deterministic but depend on the realized future states. As we shall see, there are different approaches to tackle these quite challenging problems, depending on their nature and on our purpose.

  We may be interested in the first decision x0, because this is the decision that we have to implement here and now. The next decisions enter into the optimization model in order to avoid an overly myopic behavior, but when time advances, we will solve the model again according to a rolling horizon strategy. Note that this requires the solution of a sequence of multistage models. This approach is common in management science and in some engineering problems, and it is exemplified by multistage stochastic programming models with recourse.

  In other cases, we want to find a _policy_ in feedback form, i.e., a rule mapping the current state s _t_ , which evolves according to the state transition equations (1.4), to an optimal decision. The mapping x _t_ * = _F_ _t_ (s _t_ ) can be found by following another approach, based on stochastic dynamic programming.

To see an example in which the second kind of approach may be preferable, consider pricing American-style options.13. In that case, we are interested in the expected _value_ of the objective function, resulting from the application of an optimal exercise policy in the future. This requires a strategy, possibly a suboptimal one, to come up with lower bounds on the option value, to be complemented by upper bounds.

Another common kind of problem whereby a dynamic programming approach may be preferable can be found in economics. To illustrate the idea, let us consider a stylized consumption–saving problem.14 A decision maker must decide, at each discrete-time instant, the fraction of current wealth that she consumes immediately, deriving a corresponding utility, and the fraction of current wealth that is saved and invested. The return on the investment can be random and depending on the asset allocation decisions. Even if we assume that return is deterministic, we should consider uncertainty in labor income, which determines the level of wealth available for consumption/saving. A possible model is the following:

where

  E _t_ [·] denotes a _conditional_ expectation given the current state at time _t._

  _U_ (·) is a utility function related to individual preferences.

  β   (0, 1) is a subjective time discount factor.

  _X_ τ is the cash on hand (wealth) at time τ, before making the consumption/saving decision.

  _C_ τ is the amount of wealth that is consumed.

  _S_ τ is the amount of wealth that is saved.

  _R_ τ is the return from period τ − 1 to τ; note that this return is usually random and that _R_ τ applies to _S_ τ−1, not _S_ τ.

  _P_ τ is the nominal labor income, which may vary over time according to a deterministic or stochastic trajectory.

  _Y_ τ is the noncapital income, i.e., the income from labor that depends on the nominal labor income and multiplicative shocks  τ.

As we shall see in Chapter 10, we may solve such a problem by dynamic programming, which is based on a recursive functional equation for the value function

(1.17)

where _V_ _t_ ( _X_ _t_ ) is the expected utility if we start the decision process at time _t_ , from a state with current wealth _X_ _t_. In the above model, wealth is the only state variable and randomness is due to random financial return and shocks on nominal income. However, if we model income in a more sophisticated way, including permanent and transitory shocks, we may need additional state variables. From the above equation we see that the problem is decomposed stage by stage, and if we are able to find the sequence of value functions, we have just to solve a static optimization problem to find the optimal decision at each step, given the current state. The value function is the tool to trade off immediate utility against future benefits.

Also note that we make our decision after observing the actually realized state, which does not depend only on our decisions, since it is also affected by external random factors. The policy is implicitly defined by the set of value functions _V_ _t_ (·) for each time instant. This is definitely handy if we have to carry out an extensive set of simulations in order to investigate the impact of different economic assumptions on the decision maker's behavior, and it is an advantage of dynamic programming over stochastic programming models with recourse. Nevertheless, dynamic programming has some definite limitations on its own, as it relies on a decomposition that is valid only if some Markovian properties hold for the system states, and it is plagued by the aptly called "curse of dimensionality." This refers to the possibly high dimensionality of the state space. If we have only one state variable, it is easy to approximate value functions numerically on a grid, but if there are many state variables, doing so is practically impossible. Actually, other curses of dimensionality may refer to the need to discretize the expectation in Eq. (1.17) and the computational complexity of the optimization required at each step. Nevertheless, there are approximate dynamic programming approaches that do not guarantee optimality, but typically provide us with very good decisions. The exact approach should be chosen case by case, but what they have in common is the role of Monte Carlo methods:

  To generate scenarios

  To learn an optimal policy

  To evaluate the performance of a policy by simulation

# 1.5 Pitfalls in Monte Carlo simulation

Monte Carlo methods are extremely flexible and powerful tools; furthermore, they are conceptually very simple, at least in their naive and crude form. All of these nice features are counterbalanced by a significant shortcoming: They can be very inefficient, when each single simulation run is computationally expensive and/or a large sample size is needed to obtain reliable estimates. These computational difficulties are technical in nature and can be (at least partially) overcome by clever strategies. However, there are deeper issues that should not be disregarded and are sometimes related to the very meaning of uncertainty.

## 1.5.1 TECHNICAL ISSUES

Each Monte Carlo replication yields an observation of a random variable, say _X_ _k_ , for _k_ = 1,..., _N_ , which can be thought of as a realization of a random variable _X_ , whose expected value μ = E[ _X_ ] we want to estimate. Basic inferential statistics suggests using the sample mean   and sample standard deviation _S_ to build a confidence interval with confidence level 1 − α:

(1.18)

where we assume that the sample size _N_ is large enough to warrant the use of quantiles _z_ 1−α/2 from the standard normal distribution.15 This form of a confidence interval is so deceptively simple that we may forget the conditions for its validity.

###   Example 1.7 Estimating average waiting time

Consider the queuing system of Section 1.3.3.1, but now assume that there are multiple servers, possibly bank tellers. We might be interested in investigating the trade-off between the number of servers, which has a significant impact on cost, and the average waiting time in the queue, which is a measure of service quality. Using Monte Carlo, we may simulate a fairly large number _N_ of customers and tally the waiting time _W_ _k_ , _k_ = 1,..., _N_ , for each of them. Based on this sample, we may estimate the average waiting time by the sample mean  . Now can we use Eq. (1.18) to build a confidence interval for the average waiting time?

By doing so, we would commit a few mistakes:

**1.** The confidence interval is exact for a normal population, i.e., when observations are normally distributed. Clearly, there is no reason to believe that waiting times are normally distributed.

**2.** What is the waiting time of the first customer entering the system? Clearly, this is zero, as the first customer is so lucky that she finds all of the servers idle. In such a system, we must account for a transient phase, and we should collect statistics only when the system is in steady state.

**3.** Last but not least, the common way to estimate standard deviation assumes independence among the observations. However, successive waiting times are obviously positively correlated (the degree of correlation depends on the load on the system; for high utilization levels, the correlation is rather strong).

We may find a remedy to overcome these issues, based on a batching strategy, as we shall see in Chapter 7.

The last two mistakes underlined in Example 1.7 remind us that standard inferential statistics rely on a sequence of i.i.d. (independent and identically distributed) random variables. In nontrivial Monte Carlo methods this condition may not apply and due care must be exerted. Another issue that must not be disregarded is bias: Is it always the case that E[ ] = θ, i.e., that the expected value of the sample mean we are taking is the parameter in which we are interested? This is certainly so if we are sampling a population and the parameter is actually the population mean μ. In general, estimators can be biased.

###   Example 1.8 Bias in option pricing

Monte Carlo methods are commonly used to price options, i.e., financial derivatives written on an underlying asset. Consider, for instance, an American-style put option, written on a stock that pays no dividends during the option life. This option gives its holder the right to sell the underlying stock at the strike price _K_ , up to option maturity _T._ Whenever the stock price _S_ ( _t_ ) at time _t_ , for _t_   [0, _T_ ], is smaller than _K_ , the option is "in-the-money," and the holder can exercise her right immediately, earning a payoff _K_ − _S_ ( _t_ ); to see why this is the payoff, consider that the holder can buy the stock share at the current (spot) price _S_ ( _t_ ) and sell it immediately at the strike price _K_ > _S_ ( _t_ ). This is a somewhat idealized view, as it neglects transaction costs and the fact that market prices move continuously. However, for some derivatives that are settled in cash, like index options, this is really the payoff. However, when is it really optimal to exercise? Should we wait for better opportunities? Finding the answer requires the solution of a stochastic dynamic optimization problem, whereby we seek to maximize the payoff, whose output is an optimal exercise policy. The fair option value is related to the expected payoff resulting from the application of this optimal policy. When the problem is challenging, we may settle for an approximate policy. Unfortunately, a suboptimal policy introduces a low bias, since we fall short of achieving the truly optimal payoff.

Leaving the aforementioned issues aside, the form of the confidence interval in Eq. (1.18) shows the role of estimator's variance:

where σ is the (usually unknown) variance of each observation. This implies that the half-length of the confidence interval is given by  , which is both good and bad news:

  The nice fact is that when the sample size _N_ is increased, the confidence interval shrinks in a way that does not depend on problem dimensionality.16 This is why Monte Carlo methods are naturally suitable to high-dimensional problems.

  The not so nice fact is that this convergence is related to the square root of _N._ To get the message loud and clear, this implies that to gain one order of precision, i.e., to divide the length of the confidence interval by a factor of 10, we should multiply _N_ by a factor of 100.

The last observation explains why a huge number of replications may be needed in order to obtain reliable estimates. This brute force approach may be feasible thanks to another nice feature of Monte Carlo methods: They lend themselves to parallelization, and the availability of multicore machines at cheap prices is certainly helpful. However, a smarter approach does not rely on brute computational force. Using proper variance reduction techniques, discussed in Chapter 8, we may be able to reduce σ without introducing any bias in the estimator. Even this improvement may not be enough when dealing with complex multistage optimization problems, though.

## 1.5.2 PHILOSOPHICAL ISSUES

The issues that we have discussed in the previous section are technical in nature, and can be at least partially overcome by suitable tricks of the trade. However, there are some deeper issues that one must be aware of when tackling a problem by Monte Carlo, especially in risk management.17 While in other cases we might be interested in an average performance measure, in risk management we are interested in the "bad tail" of a distribution, where extreme events may occur, such as huge losses in a financial portfolio.

###   Example 1.9 Value-at-risk

One of the most common (and most criticized) financial risk measures is value-at-risk (V@R). Technically, V@R is the quantile of the probability distribution of loss on a portfolio of assets, over a specified time horizon, with a given confidence level. In layman's terms, if the one-day V@R at 99% level is $1000, it means that we are "99% sure" that we will not lose more than $1000 the next day. If we know the probability density of loss, finding the quantile is a rather easy matter. In general, however, we may hold a portfolio with widely different asset classes, including stock shares, bonds, and perhaps quite sophisticated derivatives, whose value depends on a large number of underlying factors. Monte Carlo methods can be used to sample a suitable set of scenarios in terms of factors that are mapped to scenarios in terms of portfolio value; then, we may estimate the quantile in which we are interested. It stands to reason that estimating V@R at 99.9% will take a larger computational effort than estimating V@R at 90%, since we have to deal with extreme events. Using brute force, or clever variance reduction, we might be able to tackle such a task. However, there are more serious difficulties:

  Can we really trust our probabilistic model when we deal with very rare, but potentially disastrous events?

  What about the reliability of the estimates of small probabilities?

  What happens to correlations under extreme conditions?

  Did we include every risk factors in our model? Assuming we did so in financial terms, what about the consequence of an earthquake?

Sometimes, a line is drawn between _uncertainty_ and _risk._ The latter term is reserved to cases in which the rules of the game are clear and perfectly known, as in the case of dice throwing and coin flipping. Uncertainty, in this framework, is a more general concept that is not really reduced to the uncertainty in the realization of random variables. To get the picture, let us consider the scenario tree depicted in Fig. 1.15. The tree can be interpreted in terms of a discrete probability distribution: Future scenarios _S_ _k_ are associated with probabilities π _k_. Risk has to do with the case in which both possible scenarios and their probabilities are known. However, we may have another level of uncertainty when the probabilities are not really known, i.e., we have uncertainty about the uncertainty. Indeed, when dealing with extreme risks, the very notion of probability may be questioned: Are we talking about frequentist probabilities, or subjective/Bayesian views? We may even fail to consider all of the scenarios. The black scenario in the figure is something that we have failed to consider among the plausible contingencies. The term _black swan_ is commonly used, as well as _unk-unks_ , i.e., unknown unknowns. To add some complexity to the picture, in most stochastic models uncertainty is exogenous. In social systems, however, uncertainty is at least partially endogenous. To see concrete examples, think of the effect of herding behavior in finance and trading in thin and illiquid markets.

**FIGURE 1.15** **Schematic illustration of different kinds of uncertainty.**

To see the issue within a more general setting, imagine a mathematical model of a physical system described by a difficult partial differential equation. Can we obtain brand new knowledge by solving the equation using accurate numerical methods? Arguably, the knowledge itself is encoded in the equation. Solving it may be useful in understanding the consequence of that knowledge, possibly invalidating the model when results do not match observed reality. By a similar token, Monte Carlo methods are numerical methods applied to a stochastic model. There is high value in flexible and powerful methods that allow tackling complex models that defy analytical solution. But if the very assumptions of the model are flawed, the results will be flawed as well. If we analyze risks using sophisticated Monte Carlo strategies, we may fall into the trap of a false sense of security, and pay the consequence if some relevant source of uncertainty has not been included in the model or if probabilities have been poorly assessed. Monte Carlo is an excellent tool, but it is just that: a tool.

# 1.6 Software tools for Monte Carlo simulation

In general, Monte Carlo simulations require the following:

**1.** The possibility of describing the system behavior using an imperative programming language or a connection of graphical blocks

**2.** Random number generators

**3.** Statistical tools to collect and analyze results

**4.** Visualization/animation facilities

There is a wide array of software systems that can be used to meet the above requirements, which may be more or less relevant, depending on the underlying kind of model:

  General-purpose programming languages like C++, Visual Basic, or Java.

  Spreadsheet add-ins like @Risk18

  Software environments like R and MATLAB,19 which also allow for classical imperative programming, in a relatively high-level language

  Graphical editors for the description of a system in terms of interconnected building blocks like Simulink, which is an outgrowth of MATLAB, and Arena20

All of these options have advantages and disadvantages in terms of cost, ease of use, flexibility, readability, and ease of maintenance of the models. Clearly, the maximum power is associated with general-purpose languages. By using proper object-oriented methods, we may also build reusable libraries, and compiled general-purpose languages are definitely the best option in terms of execution speed. However, they require a non-negligible coding effort, particularly if some essential modeling infrastructure must be built from scratch. Furthermore, with respect to more powerful programming languages like MATLAB, they require many more lines of code. This is due to the need of declaring variables, which allows for good type checking and efficient compiled code, but it is definitely bad for rapid prototyping. Many lines of code may also obscure the underlying model. Furthermore, visualization and random number generation require a significant coding effort as well, unless one relies on external libraries.

Spreadsheets provide the user with a familiar interface, and this can be exploited both by writing code, e.g., in VBA for Microsoft Excel, or by using add-ins, which automate some tasks and typically provide the user with much better quality random number generators. However, if VBA code is written, we essentially fall back on general-purpose programming languages; if a ready-to-use tool is adopted, we may lack flexibility.

On the opposite side of the spectrum we find tools relying on graphical editors to connect building blocks. This is really essential when dealing with complex discrete-event systems. When modeling manufacturing systems, building everything from scratch is a daunting and error-prone effort. A similar consideration applies when simulating nonlinear systems with complicated dynamics. Indeed, this is a domain where such tools like Arena or Simulink are quite helpful. Unfortunately, these tools are not really aimed at problems in finance and economics, and are rather expensive.

The kind of simulation models we are concerned with in this book are somewhere in between, as they do not involve overly complex event bookkeeping, but may require careful discretization of stochastic differential equations. They do not involve the level of sophistication needed to simulate complicate engineering systems featuring nonlinearities, but we do face the additional issues related with uncertainty. Hence, we need a set of functions enabling us to sample a variety of probability distributions, as well as to statistically analyze the output. Therefore, in most financial and some economic applications, a good compromise between flexibility and power is represented by numerical and statistical computing environments like MATLAB and R. They offer a library of statistical functions, a rich programming language, and powerful visualization tools. They are both excellent tools for teaching, and even if they are not best suited in terms of efficiency, they can be used to validate a model and to prototype code; the code can be then translated to a more efficient compiled language. In this book, we will adopt R, which has the very welcome feature of being free.21 All the code displayed in this book is available on my website:

http://staff.polito.it/paolo.brandimarte

# 1.7 Prerequisites

The prerequisites for reading this book involve:

  Mathematical background

  Financial/economical background

  Technical/programming background

Generally speaking, the requirements are rather mild and any professional or advanced undergraduate student in mathematics, statistics, economics, finance, and engineering should be able to take advantage of this book.

## 1.7.1 MATHEMATICAL BACKGROUND

An obvious prerequisite for a book on Monte Carlo methods is represented by the basics of probability theory and inferential statistics, including:

  Events and conditional probabilities

  Discrete and continuous random variables (including distribution functions, quantiles, moments)

  A bit of stochastic processes

  A modicum of multivariate distributions, with emphasis on concepts related to covariance and correlation

  Statistical inference (confidence intervals and linear regression)

All of the rest will be explained when needed, including some basic concepts about stochastic integration and stochastic differential equations, as well as copula theory. Note that we will never use measure-theoretic probability. In the chapters involving optimization, we also assume some familiarity with linear and nonlinear programming. The required background is typically covered in undergraduate courses, and any basic book could be used for a brush-up. In my book on quantitative methods [3], all of the required background is given, both for probability/statistics and optimization applications.

## 1.7.2 FINANCIAL BACKGROUND

This book does touch some topics that may be useful for an economist, but the main focus is on finance. To keep this book to a reasonable size, I will not provide much background on, say, portfolio management and option pricing theory. When illustrating examples, we will just state which kind of mathematical problem we have to face, then we will proceed and solve it. To be specific, when illustrating Monte Carlo methods for option pricing, I will just recapitulate what options are, but I will not provide any deep explanation of risk-neutral pricing principles; a quick outline is provided in Section 3.9. The reader interested in that background could consult, e.g., [1] or [10].

## 1.7.3 TECHNICAL BACKGROUND

I assume that the reader has a working R installation and knows how to run a simple session, and how to source functions and scripts. There are plenty of online tutorials addressing the needs of a newcomer. Therefore, rather than adding a dry appendix quickly reviewing R programming, I will provide R notes along the way, when discussing concrete examples. Certainly, some familiarity with the basic concepts of any programming language is essential. However, we will not need complicated data structures, and we will also do without sophisticated object-oriented principles. The code in this book is provided only to help the reader in understanding Monte Carlo methods, but it is far from being industrial strength, reusable, and efficient code.

# For further reading

  In this book, we will deal with Monte Carlo methods at an intermediate level. Complementary references, including some more advanced material, are [7, 13, 15]. In particular, [13] is a comprehensive treatment, offering pieces of MATLAB code. See also [18] for a treatment with emphasis on Bayesian statistics. By the way, a quite readable book on the important topic of Bayesian statistics, including Monte Carlo methods in R, is [14].

  The role of Monte Carlo methods for optimization is discussed in [8] and [19]; see [17] for an extensive treatment of dynamic stochastic optimization based on Monte Carlo simulation.

  Stochastic programming theory and models are treated, e.g., in [11].

  Probably, most readers of this book are interested in financial engineering. An excellent treatment of Monte Carlo methods for financial engineering is given in [9]; see also [12] or, at a more elementary level, [2], which is MATLAB based.

  On the technical side, good readings on R programming are [6] and [16].

# References

1 Z. Bodie, A. Kane, and A. Marcus. _Investments_ (9th ed.). McGraw-Hill, New York, 2010.

2 P. Brandimarte. _Numerical Methods in Finance and Economics: A MATLAB-Based Introduction_ (2nd ed.). Wiley, Hoboken, NJ, 2006.

3 P. Brandimarte. _Quantitative Methods: An Introduction for Business Management._ Wiley, Hoboken, NJ, 2011.

4 P. Brandimarte and G. Zotteri. _Introduction to Distribution Logistics._ Wiley, Hoboken, NJ, 2007.

5 J.Y. Campbell and L.M. Viceira. _Strategic Asset Allocation._ Oxford University Press, Oxford, 2002.

6 J.M. Chambers. _Software for Data Analysis: Programming with R._ Springer, New York, 2008.

7 G.S. Fishman. _Monte Carlo: Concepts, Algorithms, and Applications._ Springer, Berlin, 1996.

8 M.C. Fu. Optimization by simulation: A review. _Annals of Operations Research_ , 53:199–247, 1994.

9 P. Glasserman. _Monte Carlo Methods in Financial Engineering._ Springer, New York, 2004.

10 J.C. Hull. _Options, Futures, and Other Derivatives_ (8th ed.). Prentice Hall, Upper Saddle River, NJ, 2011.

11 P. Kall and S.W. Wallace. _Stochastic Programming._ Wiley, Chichester, 1994.

12 R. Korn, E. Korn, and G. Kroisandt. _Monte Carlo Methods and Models in Finance and Insurance._ CRC Press, Boca Raton, FL, 2010.

13 D.P. Kroese, T. Taimre, and Z.I. Botev. _Handbook of Monte Carlo Methods._ Wiley, Hoboken, NJ, 2011.

14 J.K. Kruschke. _Doing Bayesian Data Analysis: A Tutorial with R and BUGS._ Academic Press, Burlington, MA, 2011.

15 C. Lemieux. _Monte Carlo and Quasi–Monte Carlo Sampling._ Springer, New York, 2009.

16 N.S. Matloff. _The Art of R Programming: A Tour of Statistical Software Design._ No Starch Press, San Francisco, 2011.

17 W.B. Powell. _Approximate Dynamic Programming: Solving the Curses of Dimensionality_ (2nd ed.). Wiley, Hoboken, NJ, 2011.

18 C.P. Robert and G. Casella. _Introducing Monte Carlo Methods with R._ Springer, New York, 2011.

19 J.C. Spall. _Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control._ Wiley, Hoboken, NJ, 2003.

1The estimation of π by Monte Carlo methods will also be used as an illustration of variance reduction strategies in Chapter 8.

2In Chapter 3 we offer a refresher on common probability distributions, among other things.

3Later, we will understand in more detail what is accomplished by the command set.seed (55555). For now, it is enough to say that we always use this function to set the state of pseudorandom number generators to a given value, so that the reader can replicate the experiments and obtain the same results. There is nothing special in the argument 55555, apart from the fact that 5 is the author's favorite number.

4Stochastic differential equations are introduced in Section 3.7, and approaches for sample path generation are described in Chapter 6.

5See Section 3.9 and Chapter 11.

6The Markov property is essential in dynamic programming, as we shall see in Chapter 10; we will cover some material on Markov chains in Chapter 14.

7See Section 3.1.3 for further details and a motivation of such an interest rate.

8The Wiener process is introduced in Chapter 3, and its sample path generation is discussed in Chapter 6.

9In Section 3.2.2 we shall see that 1/λ is the corresponding expected value of the exponential random variable.

10The curious reader might refer, e.g., to [4].

11See Section 3.2.2.1.

12See Section 4.2.3.

13See Example 1.8. We discuss how to price American-style options with Monte Carlo methods in Chapter 11.

14For an extensive discussion of such models see, e.g., [5].

15The quantile _z_ 1−α/2 is a number such that P{ _Z_ ≤ _z_ 1−α/2} = 1 − α/2 where _Z_ ~ N(0, 1) is standard normal. Output analysis is treated in Chapter 7.

16Well, we should say that there is no _explicit_ dependence on dimensionality, even though this may affect variability, not to mention computational effort.

17We shall deal with risk measurement and management in Chapter 13.

18See http://www.palisade.com and the copyright notices there.

19See http://www.mathworks.com and the copyright notices there.

20See http://www.arenasimulation.com and the copyright notices there.

21I have made my point about my personal preference between MATLAB and R in the Preface, and I will not repeat myself here. For more information on the use of MATLAB for financial applications, see http://www.mathworks.com/financial-services/, and for an illustration of using MATLAB in Monte Carlo financial applications see [2].

# Chapter Two

# Numerical Integration Methods

Numerical integration is a standard topic in numerical analysis, and in the previous chapter we have hinted at the link between integration and Monte Carlo methods. In this chapter we have a twofold objective:

  On the one hand, we want to insist on the link between numerical integration and Monte Carlo methods, as this provides us with the correct framework to understand some variance reduction methods, such as importance sampling, as well as alternative approaches based on low-discrepancy sequences.

  On the other hand, we want to outline classical and less classical approaches to numerical integration, which are deterministic in nature, to stress the fact that there are sometimes valuable alternatives to crude Monte Carlo; actually, stochastic and deterministic approaches to numerical integration should both be included in our bag of tricks and can sometimes be integrated (no pun intended).

In many financial problems we are interested in the expected value of a function of random variables. For instance, the fair price of a European-style option may be evaluated as the discounted expected value of its payoff under a risk-neutral probability measure. In the one-dimensional case, the expected value of a function _g_ (·) of a single random variable _X_ with probability density _f_ _x_ ( _x_ ) is given by the following integral:

This is just an ordinary and quite familiar integral. We also recall that when we need to estimate the probability of an event _A_ , which may occur or not depending on the realization of a random variable _X_ , we are back to the case above:

where 1 _A_ ( _x_ ) is the indicator function for event _A_ (taking the value 1 if event _A_ occurs when _X_ = _x_ , 0 otherwise).

In lucky cases, an integral may be evaluated in analytical or semianalytical form. To illustrate the latter case, consider the Black–Scholes–Merton (BSM) formula for the price of a call option written on a non-dividend-paying stock share at time _t_ :

In the formula, _S_ ( _t_ ) is the current spot price of the underlying asset, _K_ is the strike price, _T_ is the option maturity, i.e., when it can be exercised, and

where _r_ is the continuously compounded risk-free rate, and σ is the annualized volatility of the underlying asset. Both _r_ and σ are assumed constant in this model. Finally, Φ( _z_ ) is the cumulative distribution function of a standard normal variable:

The BSM formula is typically labeled as analytical, even though, strictly speaking, it is semianalytical, since there is no analytical expression for φ( _z_ ); nevertheless, there are quite efficient and accurate ways to approximate this function, and we need not resort to general-purpose numerical integration. In other cases, we do need a numerical approach, typically based on quadrature formulas. In this chapter we will only give the basics of such numerical strategies, but quite sophisticated methods are available. Indeed, there is no need for Monte Carlo methods in low-dimensional problems. The difficulty arises when we deal with a vector random variable **X** , with support  , and the corresponding multidimensional integral

Product quadrature formulas quickly grow out of hand, in the sense that they require a huge number of points, and this is where Monte Carlo comes into play. Indeed, even sophisticated numerical computing environments only offer limited support for multidimensional integration via classical quadrature formulas.

However, we cannot afford to dismiss deterministic approaches to numerical integration. To see why, consider a stochastic optimization problem, whereby we need the expected value of a function _h_ (·, ·) depending on both a vector of control variables x, modeling our decisions, and a vector of random variables ξ with joint density _f_ ξ(·), modeling what we cannot control:

In such a case, we do need just a single number, i.e., an expected value estimated by a sample mean, but a way to approximate the function _H_ (x), possibly obtained by selecting _S_ scenarios characterized by realizations ξ _s_ and probabilities π _s_ , _s_ = 1,..., _S_ :

As we have already pointed out, Monte Carlo methods may need a large sample size to achieve an acceptable degree of accuracy, and we may not afford that when solving a difficult optimization problem. In this setting, alternative approaches may be valuable. Some of them are deterministic in nature and are best understood with reference to a view integrating numerical integration and random sampling.

We start in Section 2.1 with a brief overview of quadrature formulas, emphasizing the role of interpolation based on polynomials. Then, we outline more sophisticated Gaussian quadrature formulas in Section 2.2. These approaches are developed for one-dimensional integration but, at least in principle, they can be extended to the multidimensional case by product formulas; this idea and its limitations are discussed in Section 2.3. Alternative approaches for high-dimensional problems, such as good lattices, Monte Carlo, and quasi–Monte Carlo, are introduced in Section 2.4. In Section 2.5 we pause to illustrate some important links between statistics and numerical integration, by considering moment matching methods. We close the chapter by outlining some R functions for numerical integration in Section 2.6.

# 2.1 Classical quadrature formulas

Consider the problem of approximating the value of the one-dimensional definite integral

involving a function _f_ (·) of a single variable, over a bounded interval [ _a, b_ ]. Since the integration is a linear operator, i.e.,

for all functions _f, g_ and real numbers α, β, it is natural to look for an approximation preserving linearity. A quadrature formula for the interval [ _a, b_ ] is defined by two ingredients:

**1.** A set of _n_ \+ 1 _nodes x_ _j_ , _j_ = 0, 1,..., _n_ , such that

**2.** A corresponding set of _weights w_ _j_ , _j_ = 0, 1,..., _n_.

Then, we approximate _I_ [ _f_ ] by

To be precise, a quadrature formula like the one we are describing is called a _closed_ formula, since the set of nodes includes the extreme points _a_ and _b_ of the interval. Open formulas are used when the function is not well-behaved near _a_ or _b_ , or when we are integrating on an infinite interval. In this book we will only consider closed formulas, as in most cases of interest for Monte Carlo simulation, as we shall see later, the integration domain is the unit interval [0, 1] or the unit hypercube [0, 1] _p_.

## 2.1.1 THE RECTANGLE RULE

One intuitive way to choose nodes and weights is the rectangle rule. The idea is illustrated in Fig. 2.1, where the underlying idea is clearly to regard the integral as an area and to approximate it by the union of rectangles. If we are integrating over the unit interval [0, 1], the rectangle rule may be expressed as

**FIGURE 2.1** **Illustration of the rectangle rule.**

(2.1)

Note that, in such a case, we divide the unit interval in _n_ slices, and we take the function value at the left end of each subinterval as the representative value for that slice; if we sum for _j_ in the range from 1 to _n_ , we are considering right end points.

For the case in the figure, i.e., _f_ ( _x_ ) = _x_ , it is fairly easy to see that

Hence, the absolute integration error is

As expected, the error goes to zero when _n_ goes to infinity, but perhaps we may do better. By "better" we do not only mean faster convergence. One would expect that, at least for such an easy function like _f_ ( _x_ ) = _x_ , a quadrature formula would be able to hand us the exact result. We shall see next how to accomplish this, but we should mention that the simple rectangle formula can be generalized to much more sophisticated rules based on good lattices, as we shall see later.

## 2.1.2 INTERPOLATORY QUADRATURE FORMULAS

Polynomials are arguably the easiest function to integrate. We also know, from one of the many Weierstrass theorems, that a continuous function can be approximated on an interval by a suitably high-order polynomial with arbitrary precision. So, it is natural to look for quadrature formulas such that the integration error

is zero for a wide class of polynomials.

**DEFINITION 2.1 (Order of a quadrature formula)** _We say that a quadrature formula Q is of order m if the integration error is zero for all the polynomials of degree m or less, but there is a polynomial of degree m_ \+ 1 _such that the error is not zero_.

Now, how can we approximate a function by a polynomial? One possibility is represented by Lagrange polynomials, which we outline very briefly. Given a function _f_ (·), say that we want to interpolate it at a given set of points ( _x_ _j_ , _y_ _j_ ), _j_ = 0, 1,..., _n_ , where _y_ _j_ = _f_ ( _x_ _j_ ), and _x_ _j_ ≠ _x_ _k_ for _j_ ≠ _k_. It is easy to find a polynomial _P_ _n_ of degree (at most) _n_ such that _P_ _n_ ( _x_ _j_ ) = _y_ _j_ for all _j_. The Lagrange polynomial _L_ _j_ ( _x_ ) is defined as

(2.2)

Note that this is a polynomial of degree _n_ and that

Then, an interpolating polynomial can be easily written as

Now, in order to define a quadrature formula, we have to define its nodes. The seemingly obvious way is to consider equally spaced nodes:

where _h_ = ( _b − a_ )/ _n_ ; also let _f_ _j_ = _f_ ( _x_ _j_ ). As it turns out, this choice need not be the best one, but it is a natural starting point. Selecting equally spaced nodes yields the set of Newton–Cotes quadrature formulas. Given these _n_ \+ 1 nodes and the corresponding Lagrange polynomial _P_ _n_ ( _x_ ) of degree _n_ , we compute the quadrature weights as follows:

which yields

(2.3)

Consider the case of two nodes only, _x_ 0 = _a_ and _x_ 1 = _b_ , with step _h_ = _x_ 1 − _x_ 0. Here we are just interpolating _f_ by a straight line:

A straightforward calculation yields

A closer look shows that this formula approximates the area below the function using a trapeze. We may improve the integration error by applying the formula on subintervals, thus approximating the overall area by trapezoidal elements, as depicted in Fig. 2.2. Thus, we find the trapezoidal rule:

**FIGURE 2.2** **Illustrating the trapezoidal quadrature formula.**

This idea is general: Given any quadrature formula for an interval, we can build a _composite_ formula by applying the same pattern to small subintervals of the integration domain.

## 2.1.3 AN ALTERNATIVE DERIVATION

A quadrature formula based on _n_ \+ 1 nodes and the corresponding weights obtained by Lagrange polynomials is by construction exact for polynomials of degree ≤ _n_. We may go the other way around, and find weights of a formula by requiring that it features a given order. For the sake of simplicity, let us consider an integral on the unit interval, and the three quadrature nodes 0, 0.5, 1:

Since we have three nodes, we should be able to find a formula that is exact for polynomials of degree ≤ 2. Given linearity of integration, we may enforce this requirement by making sure that the formula is exact for the following monomials:

The corresponding weights can be found by solving the following system of linear equations:

(2.4)

which yields  . Applying the same idea on the more general interval [ _a, b_ ], we obtain the well-known Simpson's rule:

It is fairly easy to see that, somewhat surprisingly, this formula is actually exact for polynomials of degree ≤ 3. In fact, we have

Applying Simpson's rule we have, by some straightforward algebra:

In order to improve precision, Simpson's rule may be applied to subintervals of ( _a, b_ ), resulting in a composite formula.

What we have discussed only scratches the surface of numerical integration. We would also need:

  A way to estimate the integration error in order to decide if a more refined partition of the interval is needed.

  A way to adapt partitions to the variability of the function in different integration subintervals.

  A non-naive way to choose nodes.

In the next section we give a partial answer to the last question. What is fundamental, however, is to start seeing the connection between the system of equations (2.4), which is motivated by a classical problem in numerical analysis, and the statistical concept of moment matching, to be discussed in Section 2.5.

# 2.2 Gaussian quadrature

In Newton–Cotes formulas, nodes are fixed and suitable weights are sought in order to squeeze the most in terms of quadrature order. With _n_ \+ 1 nodes, we may find a quadrature formula with order _n_. However, we might improve the order by finding nodes and weights jointly, as this essentially means doubling the degrees of freedom. In fact, this is the rationale behind Gaussian quadrature. Gaussian quadrature may yield high-quality numerical integration, and it is particularly relevant for some problems in finance, statistics, and stochastic optimization, as it may be interpreted in terms of discretization of a continuous probability distribution. There are different Gaussian quadrature formulas, associated with a non-negative weight function _w_ ( _x_ ) and an integration interval:

(2.5)

for a weight function _w_ ( _x_ ) and an integration interval [ _a, b_ ]. In this section, for the sake of convenience, we consider _n_ nodes _x_ _i_ , _i_ = 1,..., _n_. The integration interval can also be (−∞, +∞), in which case we need a weight function _w_ ( _x_ ) that goes to zero quickly enough to guarantee, as far as possible, the existence of the integral. This is quite relevant for our purposes, as we will relate _w_ ( _x_ ) to a probability density, such as the density of a normal random variable.1 Indeed, the integration interval is (−∞, +∞), and a weight function doing the job is _w_ ( _x_ ) = _e_ − _x_ 2. With such a choice of the weight function, we talk of Gauss–Hermite quadrature formulas. Actually, this weight function is not exactly the density of a normal variable, but it is easy to adjust the formula by a change of variable. Let _Y_ be a random variable with normal distribution N(μ, σ2). Then, recalling the familiar density of a normal variable:

In order to use weights and nodes from a Gauss–Hermite formula, we need the following change of variable:

Hence, we find

(2.6)

In practice, we may regard Gauss–Hermite formulas as a way to approximate a normal distribution by a discrete random variable with realizations _x_ _i_ and probabilities _w_ _i_ , _i_ = 1,..., _n_. In the next two sections we provide the interested reader with the theoretical background behind Gaussian quadrature, showing the role of orthogonal polynomials, and some hints about computational issues. However, the practically oriented reader may safely skip these sections and proceed directly to Section 2.2.2, where we show how to use the corresponding R package.

## 2.2.1 THEORY OF GAUSSIAN QUADRATURE: THE ROLE OF ORTHOGONAL POLYNOMIALS

There is a rich theory behind Gaussian quadrature formulas, and the key concept is related to families of orthogonal polynomials. In fact, a quadrature formula with maximum order is found by choosing nodes nodes as the _n_ roots of a polynomial of order _n_ , selected within a family of orthogonal polynomials with respect to the inner product

where _w_ (·) is a weight function and the integration interval can be bounded or, with a suitable choice of the weight function, unbounded. The following properties, among many others, can be proved:2

  A polynomial of degree _n_ within such a family has _n_ distinct real roots, all located within the interval ( _a, b_ ).

  These roots are interleaved, in the sense that each of the _n_ − 1 roots of the polynomial of degree _n_ − 1 lies in an interval defined by a pair of consecutive roots of the polynomial of degree _n_.

Using this choice of nodes, along with a proper choice of weights, yields a quadrature formula with order 2 _n_ − 1. To see this, let Π _n_ denote the set of polynomials of degree _n_ , and let us consider a polynomial _q_   Π _n_ that is orthogonal to all polynomials in Π _n_ −1, i.e.,

One might wonder whether such a polynomial can always be found. The answer lies in how families of orthogonal polynomials can be built, on the basis of recursive formulas that we hint at later. We obtain a sequence of polynomials of increasing degree and, since orthogonal polynomials of degree up to _n_ − 1 form a basis for Π _n_ −1, the next polynomial in the sequence is of degree _n_ and orthogonal to all polynomials in Π _n_ −1.

Now, let us consider a function _f_   Π2 _n_ −1, i.e., a polynomial of order 2 _n_ − 1. Any such polynomial can be divided by the aforementioned _q_ , obtaining a quotient polynomial _p_ and a remainder polynomial _r_ :

where _p, r_   Π _n_ −1. Now let us integrate the (ordinary) function product _wf_ by a quadrature formula based on _n_ nodes _x_ _i_ , _i_ = 1,..., _n_ , chosen as the zeros of _q_ :

This shows that indeed the formula is exact for polynomials of degree 2 _n_ − 1. Using this property, it is possible to prove the following essential facts:

  The sum of weights is 1.

  Weights are positive.

These are essential facts in light of our probabilistic applications. In fact, different formulas can be adapted to different probability distributions, like we have shown for Gauss–Hermite formulas. The specific family of orthogonal polynomials to be used depends on the weight function and the integration interval. Other examples are:

  Gauss–Legendre formulas, where the integration domain is (−1, 1) and the weight function is _w_ ( _x_ ) = 1.

  Gauss–Laguerre formulas, where the integration domain is (0, +∞) and the weight function is _w_ ( _x_ ) = _x_ α _e_ − _x_.

After clarifying the link between orthogonal polynomials and Gaussian quadrature, how can we actually find nodes and weights? There are different approaches.

**1.** We may build the relevant sequence of orthogonal polynomials by a recursion of the form

We omit further details, such as the form of the coefficients _a_ _j_ and _b_ _j_ , referring the reader to the references. We just stress two facts:

  Since roots of successive polynomials in the sequence are interleaved, the roots of the previous polynomials are useful to initialize fast root-finding algorithms.

  After determining the nodes, we may find the corresponding weights, e.g., by solving a system of linear equations or by using a set of formulas involving the underlying orthogonal polynomials and their derivatives; see, e.g., [10, Chapter 4].

**2.** Another kind of approach has the same starting point as the previous one, but the recursive formulas are used to build a matrix (tridiagonal and sym metric), whose eigenvalues are, in fact, the roots we need. Indeed, the R functions we describe in the next section follow an approach in this vein; see [4].

## 2.2.2 GAUSSIAN QUADRATURE IN R

Gaussian quadrature in R is accomplished by the package statmod, which offers the following two functions:

  gauss.quad(n,kind="legendre"), whose main input arguments are the number n of nodes and weights, and the kind of quadrature, which can be of many types, including "hermite"; "legendre" is the default setting of the parameter. If we choose this function, we have to use the transformations of Eq. (2.6) to deal with the normal distribution.

  If we want a more direct approach, we may use

    gauss.quad.prob(n,dist="uniform",1=0,u=1,mu=0,sigma=1, alpha=1,beta=1),

which allows to select a distribution "uniform", "normal", "beta", or "gamma", along with the relevant parameters.3

In both cases, the output is a list including weights and nodes. To illustrate, let us check a Gauss–Hermite quadrature with 5 nodes, for a normal random variable with parameters μ = 50 and σ = 20:

    > mu <- 50
    > sigma <- 20
    > N <- 5
    > out <- gauss.quad(N, kind="hermite")
    > w1 <- out$weights/sqrt(pi)
    > x1 <- sqrt(2)*sigma*out$nodes + mu
    > out <- gauss.quad.prob(N,dist="normal",mu=mu,sigma=sigma)
    > w2 <- out$weights
    > x2 <- out$nodes
    > rbind(w1,w2)
    [,1] [,2] [,3] [,4] [,5]
    w1 0.01125741 0.2220759 0.5333333 0.2220759 0.01125741
    w2 0.01125741 0.2220759 0.5333333 0.2220759 0.01125741
    > rbind(x1,x2)
    [,1] [,2] [,3] [,4] [,5]
    x1 −7.1394 22.88748 50 77.11252 107.1394
    x2 −7.1394 22.88748 50 77.11252 107.1394

We immediately verify that the same results are obtained by using the two functions, provided that we apply the correct variable transformations. The nodes, as expected, are symmetrically centered around the expected value. Furthermore, we may also check that probabilities add up to one and we get the correct first- and second-order moments:

    > sum(w1)
    [1] 1
    > sum(w1*x1)
    [1] 50
    > sqrt(sum(w1*x1^2)-sum(w1*x1) ^2)
    [1] 20

Readers familiar with the concepts of skewness and kurtosis, will also appreciate the following results:4

    > sum(w1*(x1-50)^3)/20^3
    [1] −8.526513e-16
    > sum(w1*(x1-50)^4)/20^4
    [1] 3

Within numerical accuracy, the above results are correct for a normal distribution: We are able to match moments of order 3 and 4 as well, with only five nodes. We will come back to this in Section 2.5. All of the results above are related to the quadrature order of Gauss–Hermite formulas when applied to polynomials. With a more complicated function, say, an exponential, we cannot expect exact results; so, let us compute the expected value of an exponential function of a normal random variable. In this particular case we can even afford an instructive comparison with the exact result. Indeed, given the properties of the lognormal distribution,5 we know that if _X_ ~ N(μ, σ2), then

The script displayed in Fig. 2.3 computes the percentage error for μ = 4 and σ = 2, when 5, 10, 15, and 20 nodes are used. Running the script, we observe that a remarkable precision is achieved with a fairly modest number of nodes:

**FIGURE 2.3** **Script to check Gauss–Hermite quadrature.**

    N= 5 True= 403.4288 Approx= 398.6568 %error 1.182865
    N= 10 True= 403.4288 Approx= 403.4286 %error 5.537709e-05
    N= 15 True= 403.4288 Approx= 403.4288 %error 1.893283e-10
    N= 20 True= 403.4288 Approx= 403.4288 %error 9.863052e-14

These results explain why Gaussian quadrature is used, e.g., when solving stochastic dynamic optimization problems by dynamic programming. However, there remains an issue: How does all of this scale to higher dimensions? We investigate the issue next; unfortunately, the answer is not quite positive.

# 2.3 Extension to higher dimensions: Product rules

Let us consider a function of _p_ variables

How can we integrate this function over a domain Ω? We know from Fubini theorem that a multidimensional integral can be transformed into a sequence of nested one-dimensional integrals. If we consider the case in which the integration region is just the Cartesian product of subsets,

we have

It is worth mentioning that this is indeed the relevant case for Monte Carlo simulation, where Ω = [0, 1] _p_ = [0, 1] × [0, 1] × ··· × [0, 1].

Therefore, it seems quite natural to extend quadrature formulas to multidimensional integration by using a product rule reflecting the above structure. More precisely, let us assume that we have weights and nodes for a Newton–Cotes quadrature formula along each dimension: For dimension _k, k_ = 1,..., _p_ , we have weights _w_ _i_ _k_ and nodes _x_ _i_ _k_ , _i_ = 1,..., _m_ _k_. A product rule approximates the integral as

The idea behind a product rule is quite intuitive, and it is illustrated in Fig. 2.4. From the figure, we may immediately appreciate the drawbacks of the approach:

**FIGURE 2.4** **An illustration of product rules.**

  If _N_ nodes per dimension are required to achieve a satisfactory precision, a total of _N_ _p_ nodes are required; this exponential growth makes the approach impractical for high-dimensional integrals.

  A subtler issue is that if we want to add a few points to improve precision, there is no obvious way to do it; adding a single point would disrupt the overall structure.

  Furthermore, we also observe that the same coordinates are used repeatedly for each dimension: The grid of Fig. 2.4 consists of 11 × 11 = 121 points in the plane, but if we project them on each coordinate axis, we only find 11 points. One might wonder if there is a better way to gather information about the function.

Since such regular grid is going to be impractical for large _p_ , we have to find alternative approaches, either deterministic or stochastic.

# 2.4 Alternative approaches for high-dimensional integration

In order to overcome the limitations of classical quadrature, alternative approaches have been developed. Needless to say, random sampling strategies collectively known as Monte Carlo methods play a prominent role in this respect, but we should not take for granted that random methods are the only alternative. In fact, there are deterministic strategies that work well in medium-sized problems:

  Lattice methods

  Low-discrepancy sequences

Here we just introduce Monte Carlo methods within the framework of numerical integration, leaving further development to later chapters. Low-discrepancy sequences are also just outlined here, as they are the subject of Chapter 9. We discuss lattice methods in some more detail, as they will not be dealt with elsewhere in this book.

## 2.4.1 MONTE CARLO INTEGRATION

To see how random sampling may be used to compute a definite integral, which is a deterministic quantity, let us consider a one-dimensional integral on the unit interval [0, 1]:

The last rewriting, where we multiply the function _g_ ( _u_ ) by 1, may not look much of an achievement. However, let us recall that the density of a uniform random variable _U_ on the unit interval [0, 1] is, in fact,

Therefore, we may think of this integral as the expected value

for _U_ ~ U(0, 1). Now it is natural to estimate the expected value by a sample mean. What we have to do is to generate a sequence { _U_ _i_ } of _independent_ random realizations from the uniform distribution, and then to estimate _I_ as

The strong law of large numbers implies that, with probability 1,

Genuine random sampling is not really feasible with a computer, but we can produce a sequence of _pseudorandom_ numbers using generators provided by most programming languages and environments, including R, as we shall see in Chapter 5.

###   Example 2.1 A first Monte Carlo integration

Consider the trivial case

To sample the uniform distribution, we rely on the run if function, whose first argument is the sample size, setting the seed of the generator in order to allow the replication of the experiment:

    > set.seed(55555)
    > mean(exp(runif(10)))
    [1] 1.657509
    > mean(exp(runif(10)))
    [1] 1.860796
    > mean(exp(runif(10)))
    [1] 1.468096
    > mean(exp(runif(100)))
    [1] 1.701009
    > mean(exp(runif(100)))
    [1] 1.762114
    > mean(exp(runif(100)))
    [1] 1.706034
    > mean(exp(runif(1000000)))
    [1] 1.718733

Clearly, a sample size _m_ = 10 does not yield a reliable estimate, and _m_ = 100 is just a bit better. A large sample size is needed to get closer to the actual result. Monte Carlo looks extremely straightforward and flexible, but not necessarily quite efficient. It is useful to compare the above results with those obtained by Gaussian quadrature:

    > require(statmod)
    Loading required package: statmod
    > out=gauss.quad.prob(10,dist="uniform", 1=0, u=1)
    > sum(out$weight*exp(out$nodes))
    [1] 1.718282

Needless to say, we do not know the exact result in practice, and we should wonder how to qualify the reliability of the estimate. This can be done by computing a confidence interval as follows:

    > set.seed(55555)
    > out <- t.test(exp(runif (100)))
    > out$estimate
    mean of x
    1.717044
    > out$conf.int
    [1] 1.621329 1.812759
    attr(,"conf.level")
    [1] 0.95

Note how a confidence interval for a mean is obtained in R by what may seem a weird route, using the t.test function, which returns a data structure including the estimate and, by default, a 95% confidence interval, among other things. This is due to the link between testing hypotheses about the mean of a normal population and confidence intervals for the mean. Even though the result may not look bad, the 95% confidence interval points out the issues with Monte Carlo. A large sample size is needed, unless suitable variance reduction strategies are employed, as we discuss at length in Chapter 8.

We will learn later why integrating on the unit interval is so relevant, but we may estimate the expected value with respect to other distributions. For instance, we have already considered the expected value of the exponential function of a normal random variable. In Section 2.2.2 we have seen that E[ _e_ _X_ ] = 403.4288 for _X_ ~ N(4, 22). Here we sample the normal distribution using rnorm:6

    > set.seed(55555)
    > X=exp(rnorm(100,mean=4,sd=2))
    > t.test(X)
                One Sample t-test
    data: X
    t = 4.1016, df = 99, p-value = 8.426e-05
    alternative hypothesis: true mean is not equal to 0
    95 percent confidence interval:
    153.2929 440.5990
    sample estimates:
    mean of x
    296.9459

Once again, the results are disappointing. Indeed, for one-dimensional integration, Monte Carlo is hardly competitive with deterministic quadrature, but when computing a multidimensional integral it may be the only viable option. In general, if we have an integral like

(2.7)

where Ω ⊂  _p_ , we may estimate _I_ [ _g_ ] by randomly sampling a sequence of points **x** _i_   Ω, _i_ = 1,..., _m_ , and building the estimator

(2.8)

where vol(Ω) denotes the volume of the region Ω. To understand the formula, we should think that the ratio   estimates the average value of the function, which must be multiplied by the volume of the integration region in order to find the integral.

We will see that in practice we need only to integrate over the unit hypercube, i.e.,

hence vol(Ω) = 1. More generally, if we consider a vector random variable

with joint density function _f_ **x** ( _x_ 1, ···, _x_ _p_ ), we have to sample **X** to estimate:

The good news is that, as we know from elementary inferential statistics, the half-length of a standard confidence interval with confidence level 1 − α is

assuming that the sample size _n_ is large enough to warrant use of quantiles _z_ 1−α/2 from the standard distribution. Here σ is the standard deviation of the estimator, and we see that _H_ does not depend on the problem dimensionality7 _p_ , whereas product rules based on classical quadrature are highly impractical for large _p_. The bad news is in the denominator,  , which does not grow quickly enough. In fact, to reduce _H_ by a factor of 10, we have to increase _n_ by a factor of 100, which shows that Monte Carlo estimates may converge slowly in practice.

## 2.4.2 LOW-DISCREPANCY SEQUENCES

As we have seen in the previous sections, the performance of naive Monte Carlo is far from impressive for small sample sizes. We may also get a further intuition by plotting a bidimensional random sample on the unit square. The following R commands

    > set.seed(55555)
    > plot(runif(100),runif(100),pch=20)
    > grid(nx=10)

produce the plot in Fig. 2.5. Here we are sampling a bidimensional uniform distribution, with independent marginals. We observe that the coverage is far from accurate and uniform. Using the grid(nx=10) function, we plot a grid of 10 × 10 small squares, many of which do not contain any point (ideally, each subsquare should contain exactly one). One possibility, of course, is to increase the sample size. In Chapter 9 we will see that we may define an objective measure of lack of uniform coverage of the unit hypercube, namely, the star discrepancy. Good set of points should have low discrepancy, and this leads to the concept of low-discrepancy sequences. They are deterministic sequences but, unlike grids produced by product quadrature rules, they do not display any apparent regularity. The Halton sequence is arguably the simplest low-discrepancy sequence, which is included in the library randtoolbox along with other sequences, such as the Sobol sequence. To illustrate the idea, let us generate and plot a two-dimensional sequence of 100 points using the Halton sequence:

**FIGURE 2.5** **A uniform random sample within the unit square.**

    > library(randtoolbox)
    > × <- halton(100,dim=2)
    > plot(x[,1], x[,2], pch=20)
    > grid(nx=10)

The resulting sequence, displayed in Fig. 2.6, looks more evenly distributed than the random one of Fig. 2.5. More formally, while Monte Carlo has an error of order  , low-discrepancy sequences have an error of order _O_ (log _n_ _p_ / _n_ ). This is bad news on the one hand, since the problem dimensionality _p_ does play a role; on the other hand, we get rid of the square root, which makes Monte Carlo not quite efficient. In practice, it turns out that low-discrepancy sequences are efficient for problems with a moderate dimensionality. Another point worth mentioning is that if we want to add one point to a low-discrepancy, we can do so without disrupting its structure, just like with random sequences; this is not true for regular grids. We should also note that deterministic and stochastic methods are not mutually exclusive, as scrambled low-discrepancy sequences can also be used, as we shall see later in Chapter 9.

**FIGURE 2.6** **A bidimensional low-discrepancy sequence of points.**

## 2.4.3 LATTICE METHODS

Another class of deterministic integration methods that is often discussed in conjunction with low-discrepancy sequences is based on lattices. Since the term "lattice" is used in diverse ways with a different meaning, it is best to state clearly what is meant in this context.

**DEFINITION 2.2 (Lattice)** _A p-dimensional lattice is a discrete subset of_  _p_ _that is closed under addition and subtraction_.

A familiar lattice is depicted in Fig. 2.4. Indeed, a regular grid associated with product rules is a lattice or, maybe more precisely, a lattice point set. Lattices may be associated with a _basis_ , i.e., a linearly independent set of vectors { **w** 1,..., **w** _p_ }, called generators, such that the whole lattice is obtained by taking suitable linear combinations of the generators:

(2.9)

where  _p_ is the set of _p_ -dimensional vectors with integer coordinates. Actually, different bases may generate the same lattice.

###   Example 2.2 A regular grid as a lattice

Consider the following basis:

where the superscrip  denotes vector and matrix transposition. This basis generates a grid of points with vertical and horizontal spacing given by 0.2. If we take the intersection of this grid with the half-open square [0, 1)2, we find points with coordinates in the set

Clearly, there are _N_ = 25 points in the set. Now, let us consider the matrix formed by lining up the vectors of the basis in columns:

Clearly, 1/det( **W** ) = 25, i.e., the determinant of **W** is related to the number _N_ of points in the intersection between the lattice and the half-open unit square. As we shall see, this is no coincidence. More generally, if we consider a basis formed by vectors

where **e** _j_ is the _j_ th unit vector in  _p_ and _n_ _j_ is an integer number associated with dimension _j_ , we obtain a grid with

points. If we adopt the same spacing along each dimension, i.e., we consider a single value _n_ , we obtain a grid with _N_ = _n_ _p_ points.

In the following, we will be interested in integrating a function on the half-open unit hypercube [0, 1) _p_. One might wonder why we should consider [0, 1) _p_ rather than [0, 1] _p_. From a theoretical point of view, lattice rules are designed for fairly smooth and periodic functions,8 such that _f_ ( **x** ) = _f_ ( **x** \+ **z** ) for any **x**    _p_ and **z**    _p_. Thus, in lattice rules, the North–East border is ruled out as there the function is supposed to take the same values as in the opposite border, and we do not want to double-count them. When applying lattice rules to more general functions, in principle, nothing changes since we are ruling out a set of measure zero. From a numerical point of view, we should also note that many pseudorandom number generators employed in Monte Carlo integration actually produce numbers in the interval [0, 1).

Using Eq. (2.9), an integration lattice _L_ _p_ is generated; the lattice point set is just the intersection

which includes _N_ points u _i_ , _i_ = 1,..., _N_. The integral of a function _f_ (x) on the unit hypercube is approximated as

Note that weights are not related to quadrature formulas, and that this rule can be associated with sampling. We may also consider lattice rules as a generalization of the rectangle rule of Section 2.1.1. The key feature of lattices is that they can be generated in a more general way than a rectangular grid, improving the accuracy of numerical integration. To this aim, a proper choice of the basis is in order.

To get a better understanding of lattices, it is useful to associate a "unit cell" with each choice of generators. The unit cell is generated by taking linear combinations as follows:

It is easy to see that this is parallelepiped. To illustrate, let us consider the basis

(2.10)

The corresponding lattice is displayed in Fig. 2.7, along with an arbitrarily selected shaded unit cell. We note the following:

**FIGURE 2.7** **Lattice generated by the basis inEq. (2.10).**

  The unit shaded cell can be shifted repeatedly to generate the lattice.

  The area of the parallelepiped is just the determinant of the matrix **W** whose columns are the generators.

  This area is easily seen to be det( **W** ) = 1/5.

  If we imagine tiling the space by shifting the unit cell, we see that the average number of lattice points per unit volume is just 1/ det( **W** ) = 5.

  Indeed, in the unit square [0, 1)2, we have five points, the vertices of the shaded cell plus the origin.

When choosing the basis, we have to fix the number _N_ of points that we want to include in the point set, and this in turn is influenced by the choice of generators. Actually, we need not select _p_ generators. We may simplify the choice and the related analysis by selecting a number _r_ < _p_ of generators. This results in rank- _r_ lattice rules, which may be rewritten as

Note that by taking a number _x_ modulo 1 we obtain its fractional part. The simplest rank- _r_ rule is obtained for _r_ = 1, and an example of rank-1 rule is the Korobov point set, which is generated as follows:

This point set can be also generated by the basis

The determinant is clearly 1/ _N_ , and in fact the point set, with a proper choice of parameters, includes _N_ points. An R function to generate a Korobov point set is illustrated in Fig. 2.8. In the code we take the increasing powers of _a_ immediately modulo _N_ , rather than computing possibly large powers of _a_ and then applying the modulo operator; this may avoid issues with overflow or loss of precision. The following snapshot generates the point set displayed in Fig. 2.9.

**FIGURE 2.8** **Function to generate a Korobov point set.**

**FIGURE 2.9** **Korobov point set for** _a_ = 3, _d_ = 2, _N_ = 10.

    > a <- 3
    > d <- 2
    > N <- 10
    > X <- korobov (a, d,N)
    > X
      [1,] [,2]
      [1,] 0.1 0.3
      [2,] 0.2 0.6
      [3,] 0.3 0.9
      [4,] 0.4 0.2
      [5,] 0.5 O.5
      [6,] 0.6 0.8
      [7,] 0.7 0.1
      [8,] 0.8 0.4
      [9,] 0.9 0.7
      [10,] 0.0 0.0
    > plot(X[,1],X[,2],pch=20)

This choice of parameters is good in the sense that it is _projection regular._ A point set consisting of _N_ points is projection regular if the projection of its points on each coordinate axis consists of _N_ distinct points:

    > sort(X[,1])
    [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
    > sort(X[,2])
    [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

We observe that a regular grid does not enjoy such a property, as it is clear in Fig. 2.4. For instance, if we build a 5 × 5 grid, we have 25 points on the grid, but only 5 along each projection. This "shadowing" effect may have the effect that we do not collect rich enough information about the function.9 Full projection is not to be taken for granted. For instance, if we take _N_ = 12 points in the above sequence, the point set is less satisfactory:

    > N <- 12
    > X <- korobov(a,d,N)
    > sort(X[,1])
    [1] 0.00000000 0.08333333 0.16666667 0.25000000
    0.33333333 0.41666667 0.50000000 0.58333333
    [9] 0.66666667 0.75000000 0.83333333 0.91666667
    > sort (X[,2])
    [1] 0.00 0.00 0.00 0.25 0.25 0.25 0.50 0.50 0.50 0.75 0.75 0.75

In this case, we do not have 12 distinct points along the second dimension. There are theoretical conditions ensuring projection regularity, as well as tables suggesting parameters in order to generate good lattices.

Another important feature of lattice rules is that they are associated with a fixed number _N_ of points. Unlike pseudorandom or low-discrepancy sequences, there is no obvious way to add a single point. However, extensible lattice rules have been proposed in the literature.

# 2.5 Relationship with moment matching

In Section 2.1.3, when developing quadrature formulas, we have considered matching the integral of a monomial exactly:

If we interpret integrals as expectations and quadrature formulas as discretizations of continuous probability distributions, with realizations _x_ _j_ and probabilities _w_ _j_ , we may wonder if there is a link with matching moments E[ _X_ _k_ ] of the continuous random variable with the corresponding moments of the discrete random variable approximating _X_. Indeed, the link is pretty evident for Gaussian quadrature, as we have observed in Section 2.2.2. In the following two sections we further illustrate the idea with reference to scenario generation. Again, it is important to understand that plain Monte Carlo scenario generation need not be the best option.

## 2.5.1 BINOMIAL LATTICES

A widely used approach to price financial options relies on a binomial lattice10 discretizing the continuous-time, continuous-state stochastic process followed by the underlying variable. The underlying variable could be the price _S_ ( _t_ ) of a stock share. Let _S_ 0 be the price of the asset at time _t_ = 0, and consider the new price _S_ δ _t_ after a time period δ _t_. We discretize the process with respect to both time and price, assuming that the new price can only take two values:

Here _u_ is a multiplicative shock corresponding to an "up" move, whereas _d_ corresponds to a "down" move. The advantage of using multiplicative shocks is that the lattice recombines: Since _S_ 0 _ud_ = _S_ 0 _du_ , an up-down sequence leads to the same price as a down-up sequence. The resulting lattice is illustrated in Fig. 2.10. Clearly, the size of a recombining lattice grows linearly in time, whereas a non-recombining tree explodes exponentially. If we assume that the parameters _u, d_ , and _p_ apply to the whole lattice, how can we choose them? Since we have three degrees of freedom, we may match two moments of the random variable _S_ δ _t_. Let

**FIGURE 2.10** **A recombining binomial lattice.**

We will see in Chapter 3 that if we assume a geometric Brownian motion, the resulting distribution is lognormal, and its expected value and variance are related to the drift and the volatility of the stochastic process; for now, let us just take the above values as given. The moment matching conditions are

We are left with a residual degree of freedom that can be used to set additional conditions. One possible choice is _p_ = 0.5; another widely used choice is _u_ = 1/ _d._ In general, we find a system of possibly nonlinear equations that can be solved exactly or approximately, depending on its complexity. For instance, if _S_ ( _t_ ) follows a geometric Brownian motion with drift μ and volatility σ, a common calibration of the lattice yields11

For small values of δ _t_ , this discretized lattice does approximate very well the continuous-time, continuous-state stochastic process, allowing for efficient pricing of options, including those featuring early exercise opportunities. However, the approach can only be applied to low-dimensional problems. A subtler issue is path dependency: Some options, like Asian-style options, have a payoff depending on the whole path of the underlying variables. The recombining binomial lattice has a Markovian feature preventing its direct use in such a case. For high-dimensional and path-dependent options, Monte Carlo is the method of choice.

## 2.5.2 SCENARIO GENERATION IN STOCHASTIC PROGRAMMING

We have introduced stochastic programming models in Section 1.4.2. One of the main issues when formulating such problems is scenario generation. Given, e.g., a multivariate normal variable **Y** , modeling the random data, with expected value μ and covariance matrix Σ, we might use Monte Carlo sampling to generate a discrete set of equiprobable scenarios. As an alternative, we may look for a set of scenarios matching moments and, possibly, a few more desirable properties. Since we are dealing with a multivariate normal, we know that for each marginal _Y_ _i_ skewness should be zero and kurtosis should be 3:

Say that we want to generate a scenario fan (the single stage of a tree) of size _S_ and that each realization has probability 1/ _S_ , for the sake of simplicity. Let us denote by _y_ _i_ _s_ the value of the random variable _Y_ _i_ in scenario _s_. Then, we should have

Approximate moment matching is obtained by solving the problem:

The objective function includes four weights _w_ _k_ , which may be used to fine tune performance. Again, this is a deterministic approach that should be compared against random sampling. We will consider scenario generation for stochastic programming models in Section 10.5.3.

# 2.6 Numerical integration in R

Available functions to compute integrals in R are, e.g., integrate for one-dimensional integrals and adaptIntegrate for multidimensional integrals on hypercubes. Consider the integral

Integration by parts yields the exact result, which we may use to check the accuracy of numerical solutions:

The following R snapshot illustrates how to do the job:

    > f <- function(x) exp(−x)*sin(10*x)
    > integrate (f, lower=0, upper=2*pi)
    0.09882501 with absolute error < 5.9e-08

It is worth noting that, because of its internal working, integrate needs a vectorized function, i.e., a function that can receive a vector input and return the corresponding vector output. The following example, involving a constant function, illustrates the issue:

    > g <- function (x) return (1)
    > integrate(g,0,10)
    Error in integrate(g, 0, 10) :
    evaluation of function gave a result of wrong length

One possibility to overcome the obstacle is to vectorize the function manually:

    > g1 <- function(x) rep(1,length(x))
    > g(1:5) [1] 1
    > gl(1:5)
    [1] 1 1 1 1 1
    > integrate(g1,0,10)
    10 with absolute error < 1.1e-13

Alternatively, we may use the higher level function Vectorize:

    > g2 <- Vectorize(g)
    > g2(1:5)
    [1] 1 1 1 1 1
    > integrate(g2,0,10)
    10 with absolute error < 1.1e-13

The integrate function can deal with integrals on unbounded domains, and it is always suggested to take advantage of that feature, rather than using very small or very large values for lower or upper integration limits, respectively. Let us try integrating the density of a standard normal distribution to see the point:

    > integrate(dnorm,0,2)
    0.4772499 with absolute error < 5.3e-15
    > integrate(dnorm,0,2000)
    0.5 with absolute error < 4.4e-06
    > integrate(dnorm,0,20000000)
    0 with absolute error < 0
    > integrate(dnorm,0,Inf)
    0.5 with absolute error < 4.7e-05

To deal with multidimensional integrals, we may use adaptIntegrate from the package cubature. As an example, let us consider the bidimensional integral:

    > library(cubature)
    > h <- function(x) exp(−x[1]*x[2])*(sin(6*pi*x[1])+
    cos(8*pi*x[2]))
    > adaptIntegrate(h,lower=c(0,0),upper=c(1,1))$integral
    [1] 0.01986377

# For further reading

  Numerical integration methods are covered in any textbook on numerical analysis. One such general reference is [7]; for numerical methods in economics see [6].

  A more specific reference for numerical integration, with emphasis on Gaussian quadrature, is [2].

  Computational approaches for Gaussian quadrature are discussed, e.g., in [10], where approaches based on finding roots of orthogonal polynomials are illustrated, and in [4], which deals with approaches based on matrix eigenvalues. A related interesting reading is [3], where the role of matrix computations is emphasized, as well as the relationship with moments, which is also interesting from a probabilistic viewpoint.

  A deep treatment on lattice methods for numerical integration is [11]. You may also see [9] for a treatment related to Monte Carlo and quasi–Monte Carlo sampling, including randomized versions of these methods.

  Scenario generation for stochastic programming is discussed, e.g., in [5] and [8].

# References

1 P. Brandimarte. _Numerical Methods in Finance and Economics: A MATLAB-Based Introduction_ (2nd ed.). Wiley, Hoboken, NJ, 2006.

2 P.J. Davis and P. Rabinowitz. _Methods of Numerical Integration_ (2nd ed.). Academic Press, Orlando, FL, 1984.

3 G.H. Golub and G. Meurant. _Matrices, Moments, and Quadrature with Applications._ Princeton University Press, Princeton, NJ, 2010.

4 G.H. Golub and J.H. Welsch. Calculation of Gaussian quadrature rules. _Mathematics of Computation_ , 23:221–230, 1969.

5 K. Hoyland and S.W. Wallace. Generating scenario trees for multistage decision problems. _Management Science_ , 47:296–307, 2001.

6 K.L. Judd. _Numerical Methods in Economics._ MIT Press, Cambridge, MA, 1998.

7 D. Kincaid and W. Cheney. _Numerical Analysis: Mathematics of Scientific Computing._ Brooks/Cole Publishing Company, Pacific Grove, CA, 1991.

8 A.J. King and S.W. Wallace. _Modeling with Stochastic Programming._ Springer, Berlin, 2012.

9 C. Lemieux. _Monte Carlo and Quasi–Monte Carlo Sampling._ Springer, New York, 2009.

10 W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery. _Numerical Recipes in C_ (2nd ed.). Cambridge University Press, Cambridge, 1992.

11 I.H. Sloan and S. Joe. _Lattice Methods for Multiple Integration._ Oxford University Press, New York, 1994.

1 In case of need, the reader may find a refresher on the normal distribution in Section 3.2.3.

2 See. e.g., [2, Chapter 2].

3 The reader who needs a refresher on such probability distributions may have a look at Chapter 3.

4 Skewness and kurtosis for a random variable _X_ with expected value μ and standard deviation σ are defined as E[( _X_ − μ)3]/σ3 and E[( _X_ − μ)4]/σ4, respectively. Skewness measures the lack of symmetry and is zero for a normal distribution, which is symmetric; kurtosis is related to the tails of the distribution and is 3 for any normal distribution. Distributions with fatter tails, which play a key role in risk management, have a larger kurtosis.

5 See Section 3.2.3.

6 Unlike the last example, we let R print the whole output from t.test; clearly, we are only interested in the estimate, and the test _p_ -value is irrelevant here.

7 As we shall also remark later, this statement should be somewhat tempered. It is true that _p_ does not play an explicit role, but it may be expected that a large _p_ will influence σ implicitly, not to mention the computational effort.

8 Periodicity is required for the validity of certain arguments showing the precision of lattice rules, which rely on Fourier series. See [11] for details.

9This is not only relevant in numerical integration but also in global optimization.

10Needless to say, this kind of lattice has nothing to do with lattices discussed in Section 2.4.3.

11See, e.g., [1] for a derivation and an illustration of possible computational advantages of this choice.

# Part Two

# Input Analysis: Modeling and Estimation

# Chapter Three

# Stochastic Modeling in Finance and Economics

This chapter is included for the sake of readers who are not familiar with econometrics and the modeling techniques that are commonly used in finance and financial economics. Needless to say, the following treatment is not meant as a substitute of one of the several excellent books on the subject. Our limited aims are:

**1.** To offer a concise introduction for readers with a background, e.g., in mathematics or engineering.

**2.** To provide readers with a short refresher, and possibly an introduction to some specific topics with which they might be not quite familiar.

**3.** At the very least, we use the subject of this chapter as an excuse to introduce some useful R functions to deal with probability distributions and some multivariate analysis methods.

Here we only deal with modeling; issues related to estimation are deferred to Chapter 4, which has similar objectives and limitations. In modeling, one typically works with parameters, expected values, variances, etc. In estimation, one typically works with their sample counterparts, like sample mean and sample variance. However, since it is often useful to illustrate ideas by simple but concrete examples, we will occasionally use sample counterparts of probabilistic concepts. In doing so, we will rely on readers' intuition and, possibly, some background in elementary inferential statistics, as well as the availability of easy to use R functions; more advanced concepts and issues are dealt with in the next chapter.

Within this framework, the choice of topics is admittedly open to criticism. We cover some rather basic issues related to probability distributions, together with some more advanced ones, such as copulas and stochastic differential equations. However, the coverage of some standard tools in econometrics, such as multiple linear regression models, will leave much to be desired. Our choice is deliberate and related to what one needs in order to appreciate Monte Carlo techniques.

In Section 3.1 we use elementary examples to motivate our study and to further illustrate the differences between static, discrete-time, and continuous-time models. Even these deceptively simple examples raise a number of issues that may be addressed by rather sophisticated tools. Section 3.2 deals with rather basic topics, as we outline a few essential (univariate) probability distributions that are commonly used in finance. Probably, many readers are quite familiar with the content of this section, which may be skipped and referred back when needed. In order to make the section as useful as possible, we take advantage of it by introducing some R functions to deal with probability distributions, including random sample generation; the technicalities involved in the generation of pseudorandom samples from these distributions is deferred to Chapter 5. We move on to deal with multivariate distributions and dependence in Sections 3.3 and 3.4. In the former section we outline multivariate distributions, including a brief review of standard concepts related to covariance and Pearson's correlation. We also cover multivariate extensions of normal and _t_ distributions, but the main aim of the section is to point out the limitations of these standard approaches, paving the way for more advanced ideas revolving around copula theory, which is the subject of Section 3.4. This section is a bit more advanced and also covers alternative correlation measures, such as Spearman's rho and Kendall's tau; we also outline such concepts as tail dependence and independence, which are relevant for risk management. Section 3.5 is a bit of an interlude, where we cover linear regression models within a probabilistic framework. Usually, linear regression is associated with the statistical estimation of linear models; here we insist on the probabilistic view of regression as an orthogonal projection, laying down the foundations for variance reduction by control variates and model complexity reduction by factor models. Regression models are related and contrasted with time series models, which are described in Section 3.6, within the framework of discrete-time models. We should note that a great deal of work has been carried out on representing, estimating, and simulating time series models in R, which also includes a specific time series object. Continuous-time models, based on stochastic differential equations, are introduced in Section 3.7. These models are most useful in financial engineering applications, but they require some background in stochastic calculus and stochastic integration. Here we introduce the basic concepts, most notably Itô's lemma, without paying too much attention to mathematical rigor. In Section 3.8 we outline two approaches that can be used to streamline a model, namely, principal component analysis and factor models. These multivariate analysis tools may be useful from both a model estimation and a simulation perspective, since they reduce data and computational requirements. Finally, in Section 3.9 we rely on Itô's lemma to develop the powerful tools of risk-neutral option pricing. This section may be obviously skipped by the familiar reader, but also by readers who are not interested in option pricing.

# 3.1 Introductory examples

In Chapter 1 we considered the difference between Monte Carlo sampling and simulation. In the former case, we are essentially approximating an integral, which is related to an expectation; in the latter case, we have to simulate dynamics over time. We have also seen that, in principle, any dynamic simulation can be considered as the estimation of the integral of a possibly quite complicated function. Hence, the line between sampling and simulation is not so sharp at all, from a Monte Carlo viewpoint. From a modeling viewpoint, however, the issues involved may be quite different. In this section we outline three related financial modeling problems:

**1.** A static model for portfolio optimization

**2.** A discrete-time consumption/saving model

**3.** A continuous-time model to describe asset price dynamics

The best known static portfolio optimization model is certainly the mean–variance model of Markowitz. In the basic version of this model, time is disregarded and issues related to rebalancing are neglected. To build the model, we need to characterize the joint distribution of asset returns over the investment time horizon, which involves subtle issues, particularly from a risk management perspective.

More realistic models, possibly involving asset and liabilities, transaction costs, etc., require the introduction of time. We may accomplish the task within a discrete- or a continuous-time framework. The former approach typically requires the machinery of time series analysis, whereas the latter one relies on stochastic calculus concepts. When time enters the stage, we may face the task of representing intertemporal dependence. Actually, if we believe the efficient market hypothesis (EMH), we may avoid the task. EMH can be stated in different, more or less stringent forms. The bottom line is that it is impossible to beat the market by stock picking or buy/sell timing. From our point of view, the essential implication is that there is no intertemporal dependence in returns over different time periods. On the contrary, if we have a different view or we are modeling something else and the EMH is not relevant, we are faced with the task of modeling some form of dependence over time.

In econometrics, to categorize the issues a modeler faces when dealing with dependence, the following model classification is used:

  We have a cross-sectional model, when we are representing different variables at the same time.

  We have a longitudinal model, when we observe a single variable at several time instants.

  Finally, we have a panel model, when both dimensions are relevant.

One of the main tasks of this chapter is to illustrate a few of these models, pointing out the related issues.

## 3.1.1 SINGLE-PERIOD PORTFOLIO OPTIMIZATION AND MODELING RETURNS

To build a single-period portfolio optimization problem, we need the following ingredients:

  A universe of _n_ assets, indexed by _i_ = 1,..., _n_.

  An initial wealth _W_ 0 that we have to allocate among the assets, taking into account some representation of investor's preferences in terms of risk aversion.

  A model to represent the uncertainty in asset prices or returns.

In principle, the microeconomics of uncertainty offers a way to represent investor's preferences, in the form of a utility function _u_ (·) mapping a monetary amount into a real number. Utility theory posits that we should maximize the expected utility of wealth _W_ _T_ , a random variable, at the end of the holding period _T._ As we did in Example 1.1, we denote the initial asset price by _P_ _i_ 0 and the amount of shares of asset _i_ held in the portfolio by _h_ _i_ ; for the sake of simplicity, here we assume that the number of shares is large enough to be approximated by a continuous variable and avoid integrality restrictions. If we neglect transaction costs and tax issues, the problem can be written as

(3.1)

where _P_ _i_ _T_ is the asset price at the end of the holding period. This models looks easy enough, but there are two serious difficulties:

  How can we find a suitable utility function?

  How can we specify a joint distribution of prices and then compute the expectation?

Computing the expectation is a technical problem, which may be tackled by the methods described in the rest of this book. However, defining a utility function is a much more serious issue. In fact, despite their popularity in most economic literature, utility functions are not easily applicable to real-life financial optimization. On the one hand, even a sophisticated investor would find it hard to assess her utility function; on the other hand, which utility function should a professional fund manager use, when reporting to her many clients? Should she consider her own attitude toward risk or that of her wealthiest client? A common approach is to resort to a risk measure, related to uncertainty in future wealth. We will have more to say about risk measures later, but since they should capture uncertainty, a seemingly natural measure is standard deviation. Furthermore, it may also be convenient to get rid of wealth _W_ 0 and deal with a sort of adimensional model in terms of portfolio weights, i.e., fraction of total wealth allocated to each asset, and asset returns. To be more specific, if the assets we are considering are stock shares, we should also consider any dividend that is paid within the planning horizon. If asset _i_ pays a dividend _D_ _i_ during the holding period, its (rate of) return can be written as

Note that the return can be negative, but it cannot be less than −1, since stock shares are limited liability assets and the worst price that we may observe in the future is _P_ _i_ _T_ = 0. To be precise, this definition does not account for the time value of money; when the dividend is paid exactly does make a difference if the time period is long enough and we consider the possibility of reinvesting the dividend. For the sake of simplicity, let us neglect this issue.

Now, what about the joint distribution of returns? If we assume that the investor only cares about expected value and standard deviation of portfolio return, we may streamline our task considerably. Let:

  μ    _n_ be the vector of expected returns, with components μ _i_ = E[ _R_ _i_ ].

  Σ    _n,n_ be the return covariance matrix, collecting entries σ _ij_ = Cov( _R_ _i_ , _R_ _j_ ).

If we denote the wealth fraction allocated to asset _i_ by _w_ _i_ , so that Σ _i_ _w_ _i_ = 1, we immediately find that the portfolio return _R_ _p_ is

with expected value

where w is the column vector collecting portfolio weights, which is transposed by the operator . The variance of portfolio return is given by

In the classical Markowitz portfolio optimization model, a mean–variance efficient portfolio is found by minimizing risk subject to a constraint on expected portfolio return:

where α _T_ is a minimal target for expected return and e = [1, 1,..., 1] ; non-negativity on portfolio weights may be added if short sales are forbidden. Note that we minimize variance, rather than standard deviation, for the sake of computational convenience. Doing so, the solution of the optimization problem does not change, but we have to solve a (convex) quadratic programming problem, for which extremely efficient algorithms are widely available.

In fact this new model looks remarkably simple, and there is no need for Monte Carlo methods for its solution. Furthermore, there is no need to specify a full-fledged joint return distribution, as first- and second-order moments are all we need. However, severe difficulties have just been swept under the rug. To begin with, how can we estimate the problem data? The covariance matrix consists of many entries, and there is no hope to estimate them all in a reliable way. To see why, assume that we have to deal with _n_ = 1000 assets. Then, the covariance matrix consists of one million entries. Actually, given symmetry, "only"

entries need be specified. This is a formidable task, as we would need a very long history of returns to obtain reliable estimates. Unfortunately, even if we collect many years of data, assuming that they are available for all of the assets that we are considering for inclusion in our portfolio, few of them are actually relevant since market conditions have changed over the years. This is why we need to resort to some data reduction strategies, as described in Section 3.8.2.

Even if we assume that we have perfect knowledge of the involved parameters, tough issues are to be considered:

  Measuring risk by variance or standard deviation is appropriate for symmetric distributions like the multivariate normal. However, returns are not symmetric, especially when financial derivatives are involved.

  We may need alternative risk measures, such as value-at-risk or conditional value-at-risk, which call for a fuller picture of the joint distribution; see Chapter 13. The behavior on the bad tails of the distribution, where we lose money, is relevant, but capturing this is not easy.

  Covariances and correlations may fail to capture true dependency. We stress this point in Section 3.4, where we motivate the introduction of copulas. In particular, can we believe that correlations stay the same in good and bad times, when extreme events occur?

  Last but not least, standard statistical methods are by their very nature backward looking. However, in finance we should look forward. For instance, assume that yesterday was very lucky for a certain stock share that earned a stellar 150% daily return, which may have a sensible impact on return statistics. Is this a good reason to invest all of our wealth in that stock?

We see that even specifying a single-period, static model is not trivial at all. No approach based on Monte Carlo sampling will yield useful results, if the underlying uncertainty model is not credible.

## 3.1.2 CONSUMPTION-SAVING WITH UNCERTAIN LABOR INCOME

In many applications a static and single-period model is inappropriate, as we need to represent a sequence of decisions to be made in the face of possibly complex dynamics of stochastic risk factors. Here we illustrate a discrete-time framework for a stylized model extending the consumption–saving problem of Section 1.4.3. The task is to choose the fraction of current wealth that is consumed, and how the remaining amount saved is allocated between a risky and a risk-free asset. For strategic decision problems like this one, an appropriate time bucket could be one year; in other cases much higher frequencies are needed. Whatever time period we choose, in any discrete-time model it is essential to specify the sequence of events very clearly, as this defines the information that is actually available when making a decision. We consider _time instants_ and _time buckets_ (or periods):

  The system state is observed and decisions are made only at discrete time instants _t_ = 0, 1, 2,....

  However, the state evolves during time buckets. To avoid excessive notation, we associate time bucket _t_ = 1, 2,... with the pair of time instants ( _t_ − 1, _t_ ). It is important to clarify if we refer to something happening at the beginning, at the end, or during a time bucket.

The rules of the game in our problem are:

  At time instant _t_ = 0, 1, 2,..., _T_ , i.e., at the beginning of the corresponding time bucket, the agent owns a current wealth denoted by _W_ _t_ , resulting from the previous saving and investment decisions.

  Labor income _L_ _t_ is collected; actually, this is income earned during the previous time bucket _t_ − 1.

  The total cash on hand _W_ _t_ \+ _L_ _t_ is split between saving _S_ _t_ and consumption _C_ _t_ during the next time bucket.

  The saved amount is allocated between a risk-free asset, with deterministic rate of return _r_ _f_ , and a risky asset, with random rate of return _R_ _t_ +1; note how the subscript _t_ \+ 1 emphasizes the fact that this return will be known only at the next time instant _t_ \+ 1, or, if you prefer, at the _end_ of the next time bucket, whereas the allocation decision must be made now, at the _beginning_ of the time bucket; let us denote by the fraction of saving that is allocated to the risky asset by α _t_   [0, 1].

Note that we are considering two sources of uncertainty here:

**1.** The uncertain return of the risky asset

**2.** The uncertainty in the labor income stream

The first source of uncertainty can be represented using the concepts from the previous section, if we do not want to consider any predictability in asset returns. However, it is likely that the labor income _L_ _t_ is not quite independent from _L_ _t_ −1; therefore, we must choose some way to represent both predictable and unpredictable (shock) components, as we show below.

Given a decision α _t_ , the rate of return of the investment portfolio over the next time bucket is

where _R_ _t_ +1 − _r_ _f_ is typically referred to as _excess return._ Therefore, wealth at time instant _t_ \+ 1 is

(3.2)

We emphasize again that the risky asset return _R_ _t_ +1 is not known when making the asset allocation decision, whereas the risk-free rate is known. Actually the risk-free return should also be modeled as a stochastic process, as the level of interest rates does change over time because of changing economic conditions and the corresponding monetary policy of central banks. Hence, we should denote this rate as _r_ _ft_. The main difference with respect to the risky return _R_ _t_ +1 is that, at the beginning of a time bucket, we do know the risk-free rate for that time bucket; however, we do not know the interest rates for the subsequent time buckets. So, _r_ _ft_ is a predictable stochastic process. For the sake of simplicity, let us assume a constant interest rate _r_ _f_. We might also wish to model correlations between the two returns. After all, we know that when central banks change the level of interest rates, stock exchanges are indeed affected.

To model labor income, we really must be more specific and decompose _L_ _t_ in components. A model proposed, e.g., in [4, Chapter 7], is

(3.3)

where

  _f_ ( _t, Z_ _t_ ) is a deterministic component, depending on time/age and other individual characteristics like the level of education (the more you invest in education, the more, hopefully, your wage should increase over time).

  ν _t_ is a random _permanent_ shock.

   _t_ is a random _transitory_ shock.

The difference between the two random components is that the transitory random term should have an impact in one year, and then it should leave no effect, whereas permanent shocks have to be cumulated, or integrated, if you prefer, over time. This can be represented as a random walk:

where, for instance,   is a normal random variable with variance ση2 and expected value zero. By a similar token, one could assume  . Note that the expected value of a shock is zero, since any predictable component should be included in the deterministic component. In simple models, η _t_ and  _t_ are assumed independent over time.

Now, we should see a problem with an additive decomposition like the one in Eq. (3.3). If shocks are normally distributed, in principle, income could get negative. Furthermore, using the same volatilities (standard deviations) for all levels of income is questionable. One way to overcome these issues is to resort to a multiplicative decomposition. We might just reinterpret equation Eq. (3.3) in terms of log-income, defined as _l_ _t_ ≡ log _L_ _t_. In other words, we posit

(3.4)

Since we take an exponential of a normal, a negative shock will be transformed into a multiplicative shock less than 1, i.e., a reduction in income. We will also see, in Section 3.2.3, that when we take the exponential of a normal random variable, we obtain a lognormal random variable.

Finally, if we associate a utility function _u_ (·) with consumption, the overall objective could be

where β   (0, 1) is a subjective discount factor. Thus, we face a dynamic, stochastic optimization problem, which cannot be solved analytically in general; Monte Carlo and related methods play a key role in the solution of these models, as well as in the simulation of decisions over time to analyze the resulting policy. In Section 3.6 we will analyze a few more discrete-time models that can be used to represent dependency over time, changes in volatility, etc.

## 3.1.3 CONTINUOUS-TIME MODELS FOR ASSET PRICES AND INTEREST RATES

Time discretization may look like an approximation of an intrinsically continuous time. This need not be the case, actually: If decisions are made periodically, then a discretized time bucket corresponding to decision periods is all we need. In other cases, higher frequencies must be accounted for. We may then take smaller and smaller time periods in order to approximate continuous time. Needless to say, this increases the computational burden considerably and a trade-off must be assessed between modeling needs and computational tractability. A possibly surprising fact is that if we take the process to the limit and consider continuous-time models, we may actually ease many issues and come up with tractable models. Furthermore, when we consider trading on exchanges and the actual flow of information, we have to accept the fact that a more faithful representation does require continuous time. This can help in building a theoretically sound framework, even if numerical methods, most notably Monte Carlo methods, are to be employed when models cannot be solved analytically. Indeed, continuous-time models rely on differential equations, which may look tougher than discrete-time difference equations, especially since we have to cope with _stochastic_ differential equations. Stochastic models are in fact needed to represent uncertainty in stock prices, interest rates, foreign exchange rates, etc. Nevertheless, it seems wise to introduce the concept in the simplest deterministic case of some practical relevance, the investment in a safe bank account. We follow here the same path that we have followed in Section 1.3.2, providing some more motivation and details.

Consider a risk-free bank account. If we invest an initial amount _B_ (0) at time _t_ = 0, the annual interest rate is _r_ _f_ , and interest is earned once per year, our wealth after one year will be

(3.5)

However, if semiannual interest _r_ _f_ /2 is paid twice a year and is immediately reinvested, we get

This new amount, if the rate _r_ _f_ is the same, is a bit larger than the value in Eq. (3.5), since we earn interest on interest. In such a case, we speak of semiannual compounding. More generally, if interest is paid _m_ times per year, we have

and it is easy to see that if _m_ goes to infinity, we end up with the limit

(3.6)

Such a continuously compounded interest does not really exist in financial markets, even though daily compounding is essentially the same as continuous-time compounding. Nevertheless, it simplifies considerably several financial calculations and it paves the way for continuous-time models.

It is useful to describe all of the above using a dynamic model. So, let us assume that interest is continuously earned and instantaneously reinvested. Over a short time interval ( _t, t_ \+ δ _t_ ), we have the following change in the bank account:

If we let δ _t_ → 0, we obtain the differential equation

(3.7)

Many readers may be more familiar with the form

(3.8)

There are a couple of reasons why we might prefer the differential form of Eq. (3.7). First, a slight manipulation shows a way to solve it. If we rewrite the equation as

and we recall the form of the derivative of a composite function, we may recognize that a logarithm is involved:

Now we may integrate the differentials over the time interval [0, _T_ ]:

(3.9)

which can be easily rewritten as

which is consistent with Eq. (3.6).

There is another reason why the form (3.7) is preferred to (3.8): It can be extended to cope with stochastic factors. Suppose that we are investing in a risky asset, rather than a risk-free bank account. We need a model for the asset price, accounting for both drift (related to expected return) and volatility (related to random variability in the price). The drift component is essentially the same as in Eq. (3.7), with _r_ _f_ replaced by some coefficient, say, μ. However, we need to add a suitable source of randomness. A common building block is the Wiener process, denoted by _W_ ( _t_ ). This is a continuous-time stochastic process, which essentially means that for every value of the continuous parameter _t_ , we have a random variable _W_ ( _t_ ): In general we may define a stochastic process as a time-indexed collection of random variables _X_ ( _t_ ) or _X_ _t_ , where time can be continuous or discrete. To include this noise term in the equation, we should first consider increments in the stochastic process over a small time interval ( _t, t_ \+ δ _t_ ),

The Wiener process is a Gaussian process, in the sense that δ _W_ ( _t)_ ~ N(0, δ _t_ ). Please note that the variance of δ _W_ ( _t_ ) is δ _t_ , and thus its standard deviation is  . Then we take the limit for δ _t_ → 0, which yields the infinitesimal increment _dW_ ( _t_ ). Adding this term, multiplied by a volatility factor σ, to Eq. (3.7) yields the following stochastic differential equation:

(3.10)

Later, we will see that this defines an important process called geometric Brownian motion. Actually, we have skipped over some challenging issues, since "taking the limit" for δ _t_ → 0 is quite delicate when random variables are involved, as one must refer to some stochastic convergence concept. Furthermore, the Wiener process is a very "irregular" and noisy process, with sample paths that are not differentiable. We will deal with sample path generation later, but you may have a peek at Fig. 3.39 to see what we mean. Hence, taking the usual derivative _dW_ ( _t_ )/ _dt_ is not possible, which justifies the use of this form. In Section 3.7 we will outline the fundamental tools of stochastic calculus, and we will learn that Eq. (3.10) is actually a shorthand for an integral equation:

**FIGURE 3.39** **A sample path of the standard Wiener process.**

(3.11)

This looks superficially like the integral in Eq. (3.9), but it relies on the concept of Itô stochastic integral, to be discussed in Section 3.7. The differential form, however, is more intuitive and paves the way for the development of several interesting processes.

For instance, let us consider a model for stochastic interest rates. Can we model an interest rate _r_ ( _t_ ) by geometric Brownian motion? The answer is no, and to see why, let us wonder about the solution of Eq. (3.10) when σ = 0. When there is no volatility, the equation boils down to Eq. (3.7), whose solution is an increasing exponential. If there is a volatility component, intuition suggests that the solution is an exponentially increasing trend, with some superimposed noise. Financial common sense suggests that interest rates do not grow without bound, and tend to oscillate around a long-term average level  . To model such a mean reversion, we may use the following equation:

(3.12)

which is an example of the Ornstein–Uhlenbeck process, to be discussed later in Section 3.7.5. The intuition is that the drift can be positive or negative, pulling back _r_ ( _t_ ) toward its long-term average level. It is worth mentioning that, in practice, continuous-time models are the rule in financial engineering.

# 3.2 Some common probability distributions

Most readers are quite familiar with the distributions that we outline in the following and can safely skip the section, possibly referring back when needed. The treatment is limited and just meant to be a refresher, but in order to make the section as useful as possible to the widest readership, we also take the review as an excuse to introduce R functions to deal with probability distributions; we will also use functions to produce some simple plots, and R scripts to illustrate both R programming and some specific properties of distributions. As a general rule, four functions are associated with each standard distribution in R. Given a distribution distr, R provides us with:

**1.** pdistr to compute the cumulative distribution function (CDF), defined as _F_ _X_ ( _x_ ) ≡ P{ _X_ ≤ _x_ }

**2.** ddistr to compute the probability density function (PDF) _f_ _X_ ( _x_ ), for a continuous random variable, or the probability mass function (PMF), i.e., _p_ _X_ ( _x_ ) = P{ _X_ = _x_ }, which makes sense only for a discrete random variable

**3.** qdistr to compute a quantile, i.e., to invert the CDF

**4.** rdistr to generate a random sample

Note how the prefixes p, d, q, and r characterize each function. The p is somewhat confusing, as "probability mass" could be associated with discrete distributions, whereas in R the p only refers to cumulative probabilities; the choice of the prefix d reminds a PDF and could possibly reflect some bias toward continuous distributions.

We may illustrate these four functions with reference to a continuous uniform variable on the interval [ _a, b_ ], which features a constant density function on its support:

A commonly used notation to state that a random variable _X_ has this distribution is _X_ ~ U[ _a, b_ ], or _X_ ~ U( _a, b_ ). We also recall that

The following R snapshots illustrate the related R functions:

    > a=5
    > b=15
    > dunif(8,min=a,max=b)
    [1] 0.1
    > punif(8,min=a,max=b)
    [1] 0.3
    > dunif(3,min=a,max=b)
    [1] 0
    > punif(3,min=a, max=b)
    [1] 0
    > dunif(18,min=a,max=b)
    [1] 0
    > punif(18,min=a,max=b)
    [1] 1

Note how qunif and punif are inverse of each other:

    > punif(9,min=a,max=b)
    [1] 0.4
    > qunif(0.4,min=a, max=b)
    [1] 9

The following snapshot will probably give different results on your computer, as random samples depend on the state of the (pseudo)random number generator:

    > runif(5,min=a, max=b)
    [1] 8.434857 5.190168 10.679197 5.220115 10.036124

This is why, throughout this book, we will use the command set.seed(55555) to set the state of the generator to a specific state, thus allowing replication of results. We defer to Chapter 5 the technicalities involved in the generation of pseudorandom numbers and variates. By the way, while the uniform distribution does not look too interesting in itself, it is fundamental in Monte Carlo simulation as all distributions are sampled by suitable transformations of U(0, 1) variables.

In the following sections we illustrate a few common _univariate_ distributions, both discrete and continuous, without any claim of exhaustiveness. Some of the distributions that we describe here have a multivariate counterpart, to be discussed in Section 3.3.

## 3.2.1 BERNOULLI, BINOMIAL, AND GEOMETRIC VARIABLES

The binomial distribution is a discrete distribution built on top of a Bernoulli random variable. The mechanism behind a Bernoulli variable is carrying out a random experiment, which can result in either a success, with probability _p_ , or a failure, with probability 1 − _p_. If we assign to the random variable _X_ the value 1 in case of a success, and 0 in case of a failure, we obtain the PMF:

For a Bernoulli variable, denoted by _X_ ~ Ber( _p_ ), we can easily calculate the expected value,

and the variance,

The formula for variance makes intuitive sense: The variance is zero for _p_ = 1 and _p_ = 0 (there is no uncertainty on the outcome of the experiment), and it is maximized for _p_ =  , where we have the largest uncertainty about the outcome.

When needed, we may use any other value, rather than 0 and 1. A financial motivation for a Bernoulli variable, quite relevant in option pricing, is modeling random asset prices _S_ ( _t_ ) in simplified form, rather than using stochastic differential equations. According to this simplified model, given the current asset price _S_ (0) = _S_ 0, the price after a time step δ _t_ , denoted by _S_ (δ _t_ ), can be either _S_ _u_ = _uS_ 0 or _S_ _d_ = _dS_ 0, with probabilities _p_ _u_ and _p_ _d_ , respectively. The subscripts may be interpreted as "up" and "down," if we choose multiplicative shocks _d_ < _u_. Now, imagine that the process _S_ ( _t_ ) is observed up to time _T_ = _n_ · δ _t_ and that up and down jumps are independent over time. Since there are _n_ steps, the largest value that _S_ ( _T_ ) can attain is _S_ 0 _u_ _n_ , and the smallest one is _S_ 0 _d_ _n_. Intermediate values are of the form _S_ 0 _u_ _m_ _d_ ( _n-m_ ), where _m_ is the number of up jumps occurred, and _n − m_ is the number of down jumps. Note that the exact sequence of up and down jumps is not relevant to determine the terminal price, as multiplicative shocks commute: _S_ 0 _ud_ = _S_ 0 _du_. A simple model like this is referred to as _binomial model_ and can be depicted as the binomial lattice in Fig. 3.1. Such a model is quite convenient for simple, low-dimensional options, as recombination keeps complexity low, whereas the size of a generic tree explodes exponentially. Now, we might wonder what is the probability that _S_ ( _T_ ) = _S_ 0 _u_ _m_ _d_ ( _n-m_ ). Indeed, this is an example of a binomial variable: To build a binomial variable, we carry out a given number _n_ of independent Bernoulli experiments and count the number _m_ of successes. Hence, a binomial variable is just the sum of _n_ independent Bernoulli variables. This distribution has finite support {0, 1,..., _n_ } and depends on the two parameters _p_ and _n_ ; thus, it may be denoted by _X_ ~ Bin( _n, p_ ). By using properties of sums of independent random variables, it is easy to show that

**FIGURE 3.1** **A binomial lattice model for a stock price.**

The PMF of a binomial variable is not completely trivial:

(3.13)

where we use the binomial coefficient

To see why this coefficient is needed, consider that there are many ways in which we may have _r_ successes and _n − r_ failures; which specific experiments succeed or fail is irrelevant. In fact, we have _n_! permutations of the _n_ individual experiments, ( _n − r_ )! permutations of failures, and _r_! permutations of successes. Since the specific order of each permutation is irrelevant, we divide _n_! by ( _n − r_ )! and _r_!.

Figure 3.2 shows the PMF for _n_ = 30, and _p_ = 0.2, _p_ = 0.4. The plot has been obtained by the following commands:

**FIGURE 3.2** **Probability mass function of two binomial random variables.**

    > par(mfrow=c(2,1))
    > plot(dbinom(0:30,size=30,p=0.2),type='h',lwd=5)
    > plot(dbinom(0:30,size=30,p=0.4),type='h',lwd=5)

The first command sets mfrow in order to draw plots according to a layout consisting of two rows and one column, thus lining the plots vertically. The plot function uses the parameter lwd to set the line width. Also check the following calculations:

    > dbinom(10,size=30,p=0.4)
    [1] 0.1151854
    > factorial(30)/factorial(20)/factorial(10)*0.4^10*0.6^20
    [1] 0.1151854

The second way of computing binomial probabilities is a straightforward application of Eq. (3.13), but it is not quite smart, as by taking the factorial of integer numbers there is a danger of overflow, i.e., of producing a number so large that it cannot be represented on a computer. When in need, it is better to leave the task to an appropriately coded function:

    > choose (30,10)*0.4^10*0.6^20
    [1] 0.1151854

Another discrete distribution that can be built on top of the Bernoulli is the geometric. This is defined by repeating identical and independent Bernoulli trials with success probability _p_ , until the first success is obtained and counting the number of experiments. The PMF of the geometric distribution is

The expected value E[ _X_ ] can be easily found by conditioning on the result of the first experiment, rather than by applying the definition. If the first experiment is a success, which occurs with probability _p_ , the conditional expected value of the number of experiments is just E[ _X_ | success] = 1; otherwise, since there is no memory of past outcomes, it is E[ _X_ ] failure] = 1 + E[ _X_ ], i.e., we count the first failed experiment and then we are back to square one. Thus:

(3.14)

R offers functions for the geometric distribution, too, but there is a little peculiarity:

    > set.seed(55555)
    > rgeom(10,prob=0.5)
    [1] 1 2 3 1 0 2 0 0 0 0

We notice that the random variable can take the value 0, because in R the convention for the geometric distribution is to count failures, rather than experiments.

## 3.2.2 EXPONENTIAL AND POISSON DISTRIBUTIONS

The exponential and the Poisson distributions, despite their different nature, are tightly intertwined: The former is continuous and may be used to model the time elapsing between two events of some kind, whereas the latter is discrete and represents the corresponding number of events occurred within a given time interval. Examples of relevant events in finance are jumps in stock prices or changes in the credit rating of a bond issuer.

### 3.2.2.1 Exponential random variables

An exponential random variable owes its name to the functional form of its PDF:

where λ > 0 is a given parameter. Note that the support of the exponential distribution is [0, +∞), which makes sense since it is often used to model the length of time intervals. The corresponding CDF is

(3.15)

It is fairly easy to show that

It is worth noting that the expected value of an exponential random variable is quite different from the mode, which is zero, and that this distribution is right-skewed. If we interpret _X_ as the time elapsing between consecutive events, 1/λ is the average time between them, and λ is a rate, i.e., the average number of events occurring per unit time. The notation _X_ ~ exp(λ) is typically used to denote an exponential random variable.

A weird but remarkable property of exponential random variables is _lack of memory_. From the cumulative distribution function (3.15) we see that

Intuitively, this means that, if we are at time zero, the probability that we have to wait more than _t_ for the next event to occur is a decreasing function of _t_ , which makes sense. What is not so intuitive is that if a time period of length _t_ has already elapsed from the last event, this has no influence on the time we still have to wait for the next one. This is clearly false, e.g., for a uniform random variable U( _a, b_ ); if a time _b_ −   has elapsed from the last event, where   is small, we should get ready for the next one. On the contrary, the exponential distribution is memoryless, which can be stated formally as follows:

This characteristic is important when making modeling choices: If lack of memory is not compatible with the phenomenon we are representing, we should not use an exponential distribution.

###   Example 3.1 Checking the memoryless property

R offers the exp family of functions, which we may use to illustrate lack of memory empirically using Monte Carlo sampling as an R programming exercise. The function in Fig. 3.3 generates a sample of numRepl exponential random variables with rate lambda. Then, the probabilities P{ _X_ > _s_ }, P{ _X_ > _s_ }, and P{ _X_ > _t_ \+ _s_ | _X_ > _t_ } are estimated empirically. Note how we restrict the sample conditionally on the event { _X_ > _t_ }, when extracting the subvector times_t. Here is the result, based on one million replications:

**FIGURE 3.3** **Checking the memoryless property of exponential distribution empirically.**

    > set.seed(55555)
    > CheckMemory(1, 1, 0.5, 1000000)
    P(T>t) = 0.367936
    P(T>s) = 0.606819
    P(T>t+s|T>t) = 0.6064832

We see that, despite a small discrepancy due to sampling errors, the estimate of P{ _X_ > _t_ \+ _s_ | _X_ > _t_ } is fairly close to P{ _X_ > _s_ }. One could wonder whether one million replications are enough. The following exact evaluation is not too different from the above estimates, suggesting that the selected number of replications should be indeed large enough:

    > 1-pexp(1,rate=1)
    [1] 0.3678794
    > 1-pexp(0.5,rate=1)
    [1] 0.6065307

However, this could just be sheer luck, and in real life we cannot afford the luxury of a comparison with exact numbers (otherwise, we would not be using Monte Carlo methods in the first place). We will address similar issues in Chapter 7, when dealing with simulation output analysis.

### 3.2.2.2 Erlang random variables

The exponential distribution is the building block of other random variables. By summing _n_ independent exponentials with common rate λ we obtain the Erlang distribution, denoted by Erl( _n_ , λ). Erlang random variables may be used to model time elapsed as a sequence of exponential stages, in order to overcome the memoryless property of a single exponential. The PDF of a few Erlang variables, with a different number of stages, can be plotted using the R script in Fig. 3.4, which produces the plot of Fig. 3.5. From a mathematical point of view we immediately notice that by summing exponential variables we get a rather different distribution. In general, the sum of random variables has a distribution that is quite different from the single terms in the sum. The normal distribution is the most notable exception, as the sum of normals is still normal. We also notice that, in order to plot the PDF of an Erlang variable in R, we have to resort to the dgamma function. As we shall see in Section 3.2.5, the gamma distribution generalizes other distributions, including the Erlang.

**FIGURE 3.4** **Plotting the PDF of different Erlang distributions.**

**FIGURE 3.5** **Erlang PDFs.**

### 3.2.2.3 Poisson random variables and the Poisson stochastic process

We have already mentioned in Section 1.3.3 a peculiar stochastic process, the Poisson process, to motivate models based on discrete-event systems. Here we further investigate its properties and link it to a discrete distribution sharing the same name. The process is obtained by counting the number of events occurred in the time interval [0, _t_ ], denoted by _N_ ( _t_ ). Hence, this is a counting process, and Fig. 3.6 illustrates a sample path. Clearly, the sample path is discontinuous, but piecewise constant, as there is a jump corresponding to each event. Let _X_ _k_ , _k_ = 1, 2, 3, 4,..., be the time elapsed between jump _k_ − 1 and jump _k_ ; by convention, _X_ 1 is the time at which the first jump occurs, after starting the system at time _t_ = 0. We obtain a Poisson process if we assume that variables _X_ _k_ are mutually independent and all exponentially distributed with parameter λ. You might even stumble on exoteric jargon like a _càdlàg function_ when dealing with certain stochastic processes in finance. This is just a French acronym for "continue à droite, limitée à gauche," since the sample path is continuous from the right, and is limited (or bounded, i.e., it does not go to infinity) from the left.

**FIGURE 3.6** **Sample path of the Poisson process.**

If we count the number of events in the [0, 1] time interval, we obtain a discrete random variable, the Poisson random variable. The Poisson distribution is characterized by the same parameter λ as the exponential, and it may take values on the set {0, 1, 2, 3,...}. Its PMF is

Despite the apparent complexity of this function, it is straightforward to check that it meets the fundamental requirement of a PMF:

By a similar token, we may also find its expected value and variance:

Note that λ is, in fact, the event rate, which characterizes this distribution, denoted by _X_ ~ Pois(λ). If we want to count events occurring in the more general interval [0, _t_ ], we have just to consider _X_ ~ Pois(λ _t_ ). The R family for the Poisson distribution is pois. Figure 3.7 shows the plot of the PMF of a Poisson random variable for λ = 5, obtained by the following command:

**FIGURE 3.7** **Probability mass function of a Poisson random variable.**

    > plot(dpois(0:30,lambda=5,p=0.2),type='h',lwd=5)

We may state the connection between the Poisson random variable and the Poisson process in more general terms as follows. If we consider a time interval [ _t_ 1, _t_ 2], with _t_ 1 < _t_ 2, then the number of events occurred in this interval, i.e., _N_ ( _t_ 2) − _N_ ( _t_ 1), has Poisson distribution with parameter λ( _t_ 2 − _t_ 1). Furthermore, if we consider another time interval [ _t_ 3, _t_ 4], where _t_ 3 < _t_ 4, which is disjoint from the previous one, i.e., ( _t_ 2 < _t_ 3), then the random variables _N_ ( _t_ 2) − _N_ ( _t_ 1) and _N_ ( _t_ 4) − _N_ ( _t_ 3) are independent. We say that the Poisson process has _stationary_ and _independent_ increments.

The Poisson process model can be generalized to better fit reality, when needed. If we introduce a time-varying rate λ( _t_ ), we get the so-called **inhomogeneous** Poisson process. Furthermore, when modeling price jumps, we would like to introduce a random variable modeling the height of the jump. The cumulative sum of jump heights in the time interval [0, _t_ ] is another stochastic process, which is known as **compound** Poisson process. We will see how sample paths for such processes can be generated in Chapter 6.

## 3.2.3 NORMAL AND RELATED DISTRIBUTIONS

It is safe to say that the normal or Gaussian distribution, with its bell shape, is the best-known and most used (and misused) distribution. A normal random variable _X_ ~ N(μ, σ2) is characterized by the following PDF:

which clearly suggests symmetry with respect to the point _x_ = μ. The parameters μ and σ2 have a clear interpretation, as a few calculations show that

R offers the norm family of functions. As an illustration, in Fig. 3.8 we show the PDF for two normal distributions with μ = 0 and σ = 1, 2, respectively; the plot is obtained as follows:

**FIGURE 3.8** **PDF of normal random variables with** μ = 0 **and** σ = 1, σ = 2.

    > x <- seq(−6.5, 6.5, by=0.01)
    > plot (x, dnorm(x, mean=0, sd=1), type="l", lty="dashed")
    > lines(x, dnorm(x,mean=0,sd=2),type="l",lty="dotdash")

From the plot, we also verify that, for a normal distribution, mode, median, and expected value are the same.

In the applications, a very special role is played by the _standard_ normal distribution, characterized by parameters μ = 0 and σ = 1. The reason of its importance is that, if we are able to work with a standard normal in terms of quantiles and distribution function, then we are able to work with a generic normal variable. In fact the CDF is the integral of the bell-shaped function above, for which no analytical formula is known. Although the general CDF for a normal variable is not known analytically, efficient numerical approximations are available for the standard normal case _Z_ ~ N(0, 1):

This function is calculated by the R function pnorm. We are also able to invert the function by qnorm, which yields the quantile _z_ _q_ defined by

for a probability level _q_   (0, 1). Armed with these functions, we may easily compute the CDF and quantiles for a generic normal. If _X_ ~ N(μ, σ2) and we need the probability

we have just to standardize:

since _Z_ = ( _X_ − μ)/σ is standard normal. If we need a quantile _x_ _q_ , such that P{ _X_ ≤ _x_ _q_ } = _q_ , we find the corresponding quantile _z_ _q_ and go the other way around by destandardizing:

In R, we can avoid all of this if we specify the parameters mean and sd.

###   Example 3.2 Some R functions for the normal distribution

The following examples show how to use pnorm and qnorm:

    > pnorm(10,mean=5,sd=3)
    [1] 0.9522096
    > pnorm((10-5)/3)
    [1] 0.9522096
    > qnorm(0.95,mean=5,sd=3)
    [1] 9.934561
    > z=qnorm(0.95);z
    [1] 1.644854
    > 5+3*z
    [1] 9.934561

We may also easily check the following well-known properties:

**1.** P{μ − σ ≤ _X_ ≤ μ + σ} ≈ 68.27%

**2.** _P_ {μ − 2σ ≤ _X_ ≤ μ + 2σ} ≈ 95.45%

**3.** P{μ − 3σ ≤ _X_ ≤ μ + 3σ} ≈ 99.73%

    > pnorm(1) − pnorm(−1)
    [1] 0.6826895
    > pnorm(2) − pnorm(−2)
    [1] 0.9544997
    > pnorm(3) − pnorm(−3)
    [1] 0.9973002

The last property shows that the normal distribution has rather thin tails, since most realizations are within three standard deviations of the mean, which makes its use sometimes questionable. Indeed, the normal distribution is a sort of benchmark in terms of fat vs. thin tails, a fact that is measured by its kurtosis:

Sometimes, kurtosis is denned as κ −3, which should actually be called _excess_ kurtosis. Distributions with positive excess kurtosis allow for more frequent extreme events than the normal. Another important feature of the normal distribution is symmetry, which implies a zero skewness:

The assumption that returns on an asset are normally distributed implies their symmetry, which is contradicted by empirical facts, especially when financial derivatives are involved. Hence, a normal model is sometimes too stylized to be credible. Nevertheless, the normal distribution has also a few quite interesting and convenient properties:

  It is a limit distribution according to the central limit theorem: If we sum a large number of independent and identically distributed random variables, the result has a normal distribution in the limit. This makes the normal distribution useful in inferential statistics, for suitably large samples.

  If we add normal random variables, we get another normal random variable. Formally, if we consider _n_ independent normals _X_ _i_ ~ N(μ _i_ , σ _i_ 2), _i_ = 1,..., _n_ , and take an affine combination with coefficients _a_ and _b_ _i_ , then

The proof of this property requires more advanced concepts related to moment generating functions.

It is said that the normal distribution is a stable distribution, since a linear combination of normals is still normal. The process can also go the other way around, since stable random variables are infinitely divisible, i.e., can be expressed as sums of similar variables. We just mention that there is a stable distribution that includes the normal as a specific case.

In Section 3.3.4 we will also see some properties of the multivariate generalization of the normal distribution. The normal distribution is also the basis of other distributions obtained by transforming a normal variable or by combining independent standard normals.

### 3.2.3.1 Lognormal distribution

A random variable _X_ is said to have a lognormal distribution if log _X_ is normal. In other words, if _Y_ ~ N(μ, σ2), then _X_ = _e_ _Y_ is lognormal, denoted by _X_ ~ LogN(μ, σ2). It can be shown that the PDF density of the lognormal can be written as

(3.16)

It is important to notice that in this case the two parameters μ and σ2 cannot be interpreted as in the normal case. In fact, using the moment generating function of the normal, it is easy to show that

Note how, due to convexity of the exponential function and Jensen's inequality, if _X_ ~ Log N(μ, σ2), then

Another noteworthy fact is that

(3.17)

This suggests using a lognormal variable with unit expected value to model random errors in a multiplicative model, whereas a normal variable with zero expected value would model random errors in an additive model.

Furthermore, since by summing normal variables we still get a normal variable, a similar property holds when we multiply lognormal random variables. Formally, if we consider _n_ independent lognormals _X_ _i_ ~ LogN(μ _i_ , σ _i_ 2), then

This is useful when modeling financial returns, which should cumulate multiplicatively rather than additively. The R family for this distribution is lnorm. For instance, we may check Eq. (3.17) empirically:

    > set.seed(55555)
    > sigma <- 2
    > X=rlnorm(1000000, meanlog=-(sigma^2)/2, sd=sigma)
    > mean(X)
    [1] 0.9928838

### 3.2.3.2 Chi-square distribution

Let _Z_ _i_ , _i_ = 1,..., _n_ , be standard and independent normal variables. The random variable _X_ defined as

is certainly not normal, as it cannot take negative values. This variable is called chi-square with _n_ degrees of freedom and is often denoted by χ _n_ 2. Recalling that the expected value of a squared standard normal is

we immediately see that

It can also be shown that

Figure 3.9 shows the PDF for a chi-square variable with 4 and 8 degrees of freedom, obtained as follows:

**FIGURE 3.9** **PDF of two chi-square random variables with 4 and 8 degrees of freedom, respectively; the variable with 4 degrees of freedom is less uncertain and has a higher mode.**

    > x <- seq(0,20,by=0.01)
    > plot (x,dchisq(x,df=4),type="l",lty="dashed")
    > lines (x,dchisq(x,df=8),type="l",lty="dotdash")

The chi-square distribution is relevant in inferential statistics, and there is a well-known nonparametric test of goodness of fit owing its name to it. It is worth noting that a chi-square distribution may be regarded as a specific case of a gamma random variable, discussed later. In finance, we also meet the _noncentral_ chi-square distribution, which is obtained by the following construction. Let us consider

The corresponding variable is called noncentral chi-square with _n_ degrees of freedom and noncentrality parameter  . It is possible to generalize this distribution to a noninteger ν, resulting in a χ′ν2 (λ) variable. The following formulas for the relevant moments can be shown:

(3.18)

The R function family chisq can also be used for noncentral case. For instance:

    > set.seed(55555)
    > rchisq(3, df=4.5, ncp=3.6)
    [1] 17.775371 8.719674 15.453155

from which we see that the degrees of freedom df and the noncentrality parameter ncp may be noninteger. This distribution is relevant in the simulation of square-root diffusions.

### 3.2.3.3 Student's _t_ distribution

If _Z_ and χ _n_ 2 are a standard normal and a chi-square with _n_ degrees of freedom, respectively, and they are also mutually independent, the random variable

has a Student's _t_ distribution with _n_ degrees of freedom, denoted by _t_ _n_. This distribution is widely used in inferential statistics, for computing confidence intervals and testing hypotheses. A notable feature is that it features fatter tails than the normal. Indeed, it can be shown that

We observe that, for large _n_ , variance goes to 1, but it is not defined for _n_ ≤ 2. Since this variance is larger than the corresponding variance of the standard normal, the _t_ distribution is more dispersed. In fact, the excess kurtosis is

for _n_ > 4. We may check these fatter tails by plotting its density in Fig. 3.10:

**FIGURE 3.10** **PDF of Student's _t_ distribution, with _n_ =1 (dash-dotted line) and _n_ =5 (dashed line), compared with a standard normal (continuous line).**

    > x <- seq(−4.5,4.5,by=0.01)
    > plot(x,dnorm(x),type="l", lty="solid")
    > lines(x,dt(x,df=1),type="l", lty="dotdash")
    > lines(x,dt(x,df=5),type="l",lty="dashed")

The plot shows the densities of _t_ 1 and _t_ 5 random variables, along with a standard normal. We observe that the PDF of the _t_ distribution is bell-shaped much like a standard normal; the main differences lie in its heavier tail and in a lower mode. For increasing _n_ , the tails of _t_ _n_ get thinner and the density tends to a standard normal. In fact, for large _n_ the two distributions are virtually identical. The _t_ distribution can be generalized to the multivariate case, as shown in Section 3.3.4.

### 3.2.3.4 _F_ distribution

Another relevant distribution can be derived from the normal and plays a fundamental role in analysis of variance and in testing regression models. If we combine two independent chi-square variables with _n_ and _m_ degrees of freedom, respectively, as follows:

we obtain the _F_ distribution with _n_ and _m_ degrees of freedom. R offers functions like df (x, df1, df2); there is a noncentral version of the _F_ distribution as well.

## 3.2.4 BETA DISTRIBUTION

The beta distribution is continuous and has support on the unit interval [0, 1]. It depends on two parameters α1, α2 ≥ 0, defining its shape through the PDF

(3.19)

The denominator of the above ratio is just needed in order to normalize the PDF. A straightforward approach is defining the beta function:

Alternatively, we may use the following gamma function:

(3.20)

Using integration by parts, it is easy to see that the gamma function satisfies the following recursion:

(3.21)

This shows that the gamma function is an extension of the factorial function outside the domain of integer numbers. In fact, using Eq. (3.21) for an integer argument and unfolding the recursion, we find

The gamma function appears in the definition of different PDFs, including, needless to say, the gamma distribution to be discussed next. It also turns out that

The PDFs displayed in Fig. 3.11 are obtained by the following R snapshot; this shows that the uniform and certain triangular distributions are special cases of the beta distribution.

**FIGURE 3.11** **PDFs of several beta distributions.**

    > x <- seq(0.001,0.999,by=0.001)
    > par(mfrow = c(3,3))
    > plot (x,dbeta (x,2,1),type=′1′)
    > plot (x,dbeta (x,1,1),type=′1′)
    > plot (x, dbeta (x, 1, 2), type=′1′)
    > plot (x,dbeta (x,1,0.8),type=′1′)
    > plot (x,dbeta (x,0.8,0.8),type=′1′)
    > plot (x,dbeta (x,0.8,1),type=′1′)
    > plot (x, dbeta (x, 8, 3), type=′1′)
    > plot (x, dbeta (x, 3, 3), type=′1′)
    > plot (x, dbeta (x, 3, 8), type=′1′)

Possible uses of a beta random variable are related to random percentages, like recovery rates after a default. If needed, the support of the beta distribution can be easily mapped to the interval ( _a, b_ ). It is also interesting to compare the PDF of the beta distribution in Eq. (3.19) with the PMF of a binomial distribution in Eq. (3.13). Apart from the normalization factors, they look quite similar, at least if we choose integer parameters α1 and α2 in the PDF of a beta variable. In fact, these distributions are quite intertwined within the framework of Bayesian statistics, when we consider uncertainty in the parameter _p_ of a Bernoulli random variable; the beta distribution can be used to represent such an uncertainty, both in the prior and the posterior.

## 3.2.5 GAMMA DISTRIBUTION

The gamma distribution is characterized by the PDF

(3.22)

which depends on parameters α and λ; we use notation _X_ ~ Gamma(α, λ). A notable feature of the gamma distribution is that it includes other distributions as special cases:

**1.** It is immediate to see that the distribution Gamma(1, λ) boils down to the exponential distribution.

**2.** A property of the gamma distribution is that if we consider _n_ independent gamma variables _X_ _i_ ~ Gamma(α, λ), _i_ = 1,..., _n_ , then

**3.** By putting the first two facts together, we see that Gamma( _n_ , λ) is just an Erlang distribution.

**4.** The chi-square distribution with _n_ degrees of freedom is obtained as Gamma .

The fundamental moments of a gamma variable are

This suggests that λ is a scale parameter, whereas α is a shape parameter. This is in fact the case, in the sense that if _X_ ~ Gamma(α, 1), then _X_ /λ ~ Gamma(α, λ). Actually, some authors prefer to use λ as the scale parameter, whereas others prefer using β = 1/λ. R offers both possibilities, as there are two alternatives in the gamma function family:

    > dgamma(2, shape=3, rate=2)
    [1] 0.2930502
    > dgamma(2, shape=3, scale=1/2)
    [1] 0.2930502

The following R snapshot produces the plot in Fig. 3.12, which illustrates the effect of the shape parameter for a normalized scale parameter:

**FIGURE 3.12** **PDFs of some gamma distributions.**

    > x <- seq(0,7,by=0.01)
    > plot(x,dgamma(x,shape=.5,rate=1),type=′1′,
    ylim=c(0,1.2),ylab=")
    > lines(x,dgamma(x,shape=1,rate=1))
    > lines(x,dgamma(x,shape=2,rate=1))
    > lines(x,dgamma(x,shape=3,rate=1))

## 3.2.6 EMPIRICAL DISTRIBUTIONS

Sometimes, no textbook distribution will fit the available data. In such a case, one could consider resorting to an empirical distribution. In the discrete case, we have just to specify a set of values and the associated probabilities. In the continuous case, we must come up with a sensible way to devise a PDF or a CDF based on a discrete set of observations.

Piecewise linear interpolation is a straightforward way of building a CDF. Given _n_ observations _X_ _i_ , _i_ = 1,..., _n_ , of the random variable _X_ , we first sort them and find the order statistics _X_ (1) ≤ _X_ (2) ≤ ··· ≤ _X_ ( _n_ ). Now we build a piecewise linear CDF   as follows:

(3.23)

In Fig. 3.13 we illustrate an example for _n_ = 6. After defining the empirical CDF, it is easy to sample from it using, e.g., the inverse transform method of Section 5.3.

**FIGURE 3.13** **Building an empirical CDF by piecewise linear interpolation.**

Kernel-based estimation is an alternative, possibly more sophisticated way to build an empirical distribution. Unlike the piecewise-linear interpolation approach, it aims at building a PDF, and it can be interpreted as a way to "smooth" a histogram. Roughly speaking, the idea is to center a suitable elementary density around each observation, to add all of the densities, and to scale the result according to the number of observations in such a way as to obtain a legitimate overall PDF.

R offers a density function to this aim, receiving a vector of observations and producing an object of class density. The function behavior can be controlled by setting several parameters, among which we emphasize:

  kernel, which is used to select the elementary PDF; natural options are gaussian (the default), rectangular, and triangular, but there are others.

  bw, which is used to select the bandwidth, i.e., the width of each elementary PDF.

To illustrate the role of bandwidth, let us generate a sample of observations by using a mixture of two normals:

    > set.seed(55555)
    > data <- c(rnorm(40,10,10), rnorm(30,75,20))
    > hist(data,nclass=15)
    > windows()
    > par(mfrow=c(1,2))
    > kd1 <- density(data, bw=2)
    > plot(kdl,main="")
    > kd2 <- density(data, bw=10)
    > plot(kd2, main="")
    > class(kd1)
    [1] "density"

The corresponding histogram is displayed in Fig. 3.14, and suggests a bimodal distribution. The kernel kd1 is indeed of class density, and it can easily be plotted, resulting in the two PDFs displayed in Fig. 3.15. We clearly see that a small bandwidth does not smooth data enough and produces a confusing PDF; a larger bandwidth yields a definitely more reasonable result.

**FIGURE 3.14** **Histogram of data generated by a mixture of normals.**

**FIGURE 3.15** **Kernel densities for different bandwidths.**

An extreme form of empirical distribution, in a sense, is the bootstrap approach sometimes employed in finance. The idea is to take a time series of prices or returns, and to generate future scenarios by sampling past outcomes. There are refinements of this idea, but it should be clear that if we take past observations and we just mix them randomly to come up with sample paths, we are implicitly assuming that longitudinal dependencies and correlations can be ruled out. Hence, we are relying on the efficient market hypothesis. On the other hand, a notable advantage of the bootstrap approach is that, when dealing with multidimensional time series, we automatically account for cross-sectional dependencies. In the next section, indeed, we move on to consider multivariate distributions, and we shall see that modeling such dependencies is far from trivial, and this explains why bootstrap is often advocated.

Finally, two general remarks are in order about empirical distributions:

**1.** Unless suitable countermeasures are adopted, they will never generate anything beyond the range of what was observed in the sample that we use to generate the distribution. This is quite clear from Fig. 3.13 and may rule out extreme scenarios, as the resulting distribution has finite support. One possibility is to append suitable tails in order to correct for this range effect. The PDF produced by kernel estimation approach, in fact, goes a bit beyond what is observed, depending on the selected bandwidth. More specific approaches to select tails have been proposed in the literature.

**2.** The idea of just fitting the data as they are in order to capture all of their nuances, without the oversimplification of textbook distributions, is quite appealing. However, we should always keep in mind the danger of overfitting. For instance, if we observe bimodal data, we should wonder if there is some fundamental reason behind that pattern, which should be represented in our uncertainty model, or if it is just the effect of sampling variability.

# 3.3 Multivariate distributions: Covariance and correlation

In the previous sections we have considered univariate distributions, which provide us with the nuts and bolts for modeling uncertainty. However, the need for modeling dependence is pervasive in finance and economics, and this leads us to consider multivariate distributions. We are interested in two main dimensions of dependence:

  Dependence between two variables observed at the same time instant, i.e., _cross-sectional dependence._ As an example, we are interested in the relationship between return of different assets over the same time horizon. This kind of dependence may be represented explicitly by a possibly complicated multivariate distribution or built implicitly, e.g., by factor models described in Section 3.8.2.

  Dependence between two observations of the same variable at different time instants, i.e., _longitudinal dependence._ For instance, we want to investigate the relationship between returns of the same asset in two consecutive days. Again, this kind of dependence may be represented explicitly by a possibly complicated multivariate distribution or built implicitly, e.g., by time series models described in Section 3.6.

Clearly, the decomposition of dependence in two orthogonal dimensions is an oversimplification. In some cases, we really need to consider both dimensions together, which leads to the analysis of panel data. In other cases, the decomposition may be a sensible modeling simplification. For instance, we might want to check the dependence of two bond defaults that do not really occur at the same time instant, but from a modeling point of view they could be considered as happening at the same time (unless we wish to model causal links between them).

In this section we start by considering the structure of multivariate distributions, emphasizing their complexity and the fact that a characterization of marginal distributions does not tell the whole story, unless we assume independence. Then, we deal with standard concepts like covariance and Pearson's correlation. Showing their potential limitations paves the way for Section 3.4, where we illustrate more advanced ideas revolving around the copula concept. Finally, we recall a couple of multivariate distributions, namely, the normal and Student's _t_ , which generalize their univariate counterparts.

## 3.3.1 MULTIVARIATE DISTRIBUTIONS

We know that univariate distributions may be described by the PMF in the discrete case or the PDF in the continuous case. Actually, the CDF is a more general concept, since it is valid in a general setting. So, the starting point to to get the picture about the dependence of two random variables is their joint CDF.

**DEFINITION 3.1 (Joint cumulative distribution function)** _The joint CDF of random variables X and Y is defined as_

_The streamlined notation_ P( _X_ < _x,Y_ ≤ _y_ ) _is typically used to denote a joint event._

The joint CDF is a function of two variables, _x_ and _y_ , and it fully characterizes the joint distribution of the random variables _X_ and _Y._ It is easy to see how the definition can be generalized to _n_ random variables.

The joint CDF is the most general tool to characterize a multivariate distribution. Other concepts may be defined, depending on the character of the involved random variables. In the case of two discrete variables, where _X_ may take values _x_ _i_ , _i_ = 1, 2, 3,..., and _Y_ may take values _y_ _j_ , _j_ = 1, 2, 3,..., we define the joint PMF as

When dealing with continuous random variables, we may define a joint PDF _f_ _X,Y_ ( _x, y_ ) such that

(3.24)

In general, we cannot be sure that such a function exists. To be precise, we should say that the two random variables are _jointly continuous_ when their joint PDF exists. Given the joint PDF, we may find the joint CDF by integration:

(3.25)

Also the joint PMF and PDF can be defined for _n_ random variables. Needless to say, specifying a function of _n_ variables requires a lot of effort unless one imposes some structure:

  One possibility for doing so is to define parametric families of distributions, depending on a limited number of parameters. For instance, multivariate generalizations of normal and _t_ distributions, to be discussed later, essentially depend on first- and second-order moments.

  Another possibility is to characterize each random variable individually, by resorting to the wide array of univariate distributions we are familiar with, and then put the pieces together in order to capture dependence. This kind of decomposition is the idea behind the copula approach.

We can do without the above machinery in one easy case, i.e., when the random variables are independent. In such a case, the events { _X_ ≤ _x_ } and { _Y_ ≤ _y_ } are independent, and the CDF factors into the product of two terms:

In general, whatever refers to a single variable, within the context of a multivariate distribution, is called _marginal._ In case of independence, the joint CDF is just the product of marginal CDFs. The same applies to joint PMFs and PDFs. A first question is: Given the joint CDF _F_ _X,Y_ ( _x, y_ ), how can we find the marginal CDFs _F_ _X_ ( _x_ ) and _F_ _Y_ ( _y_ ), pertaining to each individual variable, if we do not assume independence? In principle, the task for the CDF is fairly easy. We obtain the marginal CDFs for the two random variables as follows:

By the same token, _F_ _Y_ ( _y_ ) = _F_ _X,Y_ (+∞, _y_ ). In the discrete case, to obtain the marginal PMFs from the joint PMF, we just use the total probability theorem and the fact that events { _Y_ = _y_ _j_ } are disjoint:

If we want to obtain marginal PDFs from the joint PDF, we may work in much the same way as in the discrete case:

If we introduce the marginal density

we have

The marginal PDF for _Y_ is obtained in the same way, integrating with respect to _x_.

We see that, given a joint distribution, we may find the marginals. It is tempting to think that, given the two marginals, we may recover the joint distribution in some way. This is _not_ true, as the marginal distributions do not say anything about the link among random variables.

###   Example 3.3 Marginals are not enough: Part 1

Consider the discrete-time stochastic process

(3.26)

where   ~ N(0, 1). We immediately see that each _X_ _t_ is normal with expected value 0 and variance _t_ 2. This marginal distribution is the same for the process

(3.27)

where  _t_ is again standard normal, but we have one such variable for each time instant. Indeed, Eq. (3.26) describes a rather degenerate process, since uncertainty is linked to the realization of a _single_ random variable. The following R snapshot produces a set of sample paths for the two processes, illustrated in Figs. 3.16 and 3.17:

**FIGURE 3.16** **A few sample paths of the stochastic process ofEq. (3.26).**

**FIGURE 3.17** **A single sample path of the stochastic process ofEq. (3.27).**

    set.seed(55555)
    len <- 50
    times <- 0:len
    eps <- rnorm(10)
    lims <- c(len*min(eps),len*max(eps))
    plot(x=times,y=eps[1]*times,type=′1′, ylim=lims)
    points(x=times,y=eps[1]*times)
    for (k in 2:10)
        lines(x=times,y=eps[k]*times)
        points(x=times,y=eps[k]*times)
    windows()
    eps <- rnorm(300)
    path <- (0:300)*c(0,eps)
    plot (x=0:300,y=path,type=′1′)

Note the use of the variable lims, to set the limits of successive plots, and the function windows (), to open a new window for plotting without overwriting the previous one.

It is easy to see the two processes have little in common, despite the fact that their marginal PDFs are the same for every time instant.

The example above deals with dependence over time for a monodimensional stochastic process, i.e., in the longitudinal sense. Specifying marginals for each time instant falls short of capturing the structure of a stochastic process. We will make similar considerations later, when dealing with the Wiener process. The next example illustrates the case of cross-sectional dependence between two random variables: We may have quite different joint distributions, sharing the same marginals.

###   Example 3.4 Marginals are not enough: Part 2

Consider the following joint PDFs (in a moment we will check that they are legitimate densities):

They look quite different, but it is not too difficult to see that they yield the same marginals. The first case is easy:

By symmetry, we immediately see that _f_ _Y_ ( _y_ ) = 1 as well. Hence, the two marginals are two uniform distributions on the unit interval [0, 1]. Now let us tackle the second case. When integrating with respect to _y_ , we should just treat _x_ as a constant:

As before, it is easy to see by symmetry that _f_ _Y_ ( _y_ ) = 1 as well. Again, the two marginals are two uniform distributions, but the links between the two random variables are quite different.

Before closing the example, it is easy to see that _f_ _X,Y_ ( _x,y_ ) is a legitimate density as it is never negative and

Actually, this is just the area of a unit square. Checking the legitimacy of _g_ _X,Y_ ( _x, y_ ) is a bit more difficult. The easy part is checking that the integral over the unit square is 1. Given the marginals above, we may write

We should also check that the function is never negative; this is left as an exercise.

The two examples point out that there is a gap between two marginals and a joint distribution. The missing link is exactly what characterizes the dependence between the two random variables. This missing link is the subject of a whole branch of probability theory, which is called _copula theory_ ; a copula is a function capturing the essential nature of the dependence between random variables, separating it from the marginal distributions. Copulas are described in Section 3.4. In the next section we recall a more classical, simpler, but partial characterization of the links between random variables, based on covariance and correlation.

## 3.3.2 COVARIANCE AND PEARSON'S CORRELATION

If two random variables are not independent, we need a way to measure their degree of dependence. This is implicit in the joint distribution, which may be difficult if not impossible to manage and estimate, in general. The simplest measure in this vein is covariance.

**DEFINITION 3.2 (Covariance)** _The covariance between random variables X and Y is defined as_

_Usually, the covariance between X and Y is denoted by σ_ _XY_.

Covariance is the expected value of the product of two deviations from the mean, and its sign depends on the signs of the two factors. We have positive covariance when the events { _X_ > E[ _X_ ]} and { _Y_ > E[ _Y_ ]} tend to occur together, as well as the events { _X_ < E[ _X_ ]} and { _Y_ < E[ _Y_ ]}, because the signs of the two factors in the product tend to be the same. If the signs tend to be different, we have a negative covariance. What covariance tries to capture is whether the two variables tend to be on the same side of their respective means, both above or both below, in which case covariance is positive, or on opposite sides, in which case covariance is negative.

We recall a few properties of covariance, which are easy to check:

(3.28)

(3.29)

(3.30)

(3.31)

(3.32)

(3.33)

Equation (3.28) yields a convenient way of computing covariance. Equation (3.29) shows that covariance is, as the name suggests, a generalization of variance; this is also reflected in the notation σ _XX_ = σ _X_ 2. Equation (3.30) points out an important issue: Covariance is a measure of association, but it has nothing to do with cause–effect relationships. Equation (3.31) shows that covariance, just like variance, is not affected by a deterministic shift. Putting Eqs. (3.32) and (3.33) together, we may prove the following theorem.

**THEOREM 3.3 (Variance of a sum of random variables)** _Given a collection of random variables X_ _i_ , _i_ = 1,..., _n, we obtain_

Covariance is a generalization of variance. Hence, it is not surprising that it shares a relevant shortcoming: Its value depends on how we measure the underlying quantities. To obtain a measure that does not depend on units of measurement, we may consider the covariance between standardized versions of _X_ and _Y_ :

where μ _X_ , σ _Y_ , Σ _X_ , and σ _Y_ are expected values and standard deviations of _X_ and _Y_ , respectively, and we have used properties of covariance. This motivates the definition of the correlation coefficient

**DEFINITION 3.4 (Correlation coefficient)** _The correlation coefficient between random variables X and Y is defined as_

To be precise, the above definition corresponds to Pearson's correlation; later we will see that there are alternative definitions of similar measures. The coefficient of correlation is adimensional, and it can be easily interpreted, as it can take only values in the interval [−1, 1]. While we usually do not prove theorems in this book, we make an exception here as the proof is instructive.

**THEOREM 3.5** _The correlation coefficient ρ_ _XY_ _may only take values in the range_ [−1, 1]. _If_ ρ _XY_ = ±1. _then X and Y are related by Y_ = _a_ \+ _bX, where the sign of b is the sign of the correlation coefficient._

**PROOF** Consider the following linear combination of _X_ and _Y_ :

We know that variance cannot be negative; hence

This inequality immediately yields ρ _X,Y_ ≥ −1. By the same token, consider a slightly different linear combination:

We also know that if Var( _Z_ ) = 0, then _Z_ must be a constant. In the first case, variance will be zero if ρ = −1. Then, we may write

for some constant α. Rearranging the equality, we have

This can be rewritten as _Y_ = _a_ \+ _bX_ , and since standard deviations are non-negative, we see that the slope is negative. Considering the second linear combination yields a similar relationship for the case ρ = 1, where the slope _b_ is positive.

Given the theorem above, it is fairly easy to interpret a specific value of correlation:

  A value close to 1 shows a strong degree of positive correlation.

  A value close to −1 shows a strong degree of negative correlation.

  If correlation is zero, we speak of _uncorrelated variables._

A visual illustration of correlation is given in Fig. 3.18. Each scatterplot shows a sample of 100 joint observations of two random variables for different values of correlation. The effect of correlation is quite evident if we think to "draw a line" going through each cloud of point; the slope of the line corresponds to the sign of the correlation. In the limit case of ρ = ±1, the observations would exactly lie on a line.

**FIGURE 3.18** **Samples of jointly normal variables for different values of the correlation coefficient ρ.**

We easily see that if two variables are independent, then their covariance is zero. In fact, independence implies E[ _XY_ ] = _E_ [ _X_ ] · E[ _Y_ ]; this can be shown, e.g., by taking advantage of the factorization of the joint PDF into the product of the two marginals. Then, using the property (3.28), we see that independence implies that covariance is zero. However, the converse is _not_ true in general, as we may see from the following counterexample.

###   Example 3.5 Two dependent but uncorrected random variables

Let us consider a uniform random variable on the interval [−1, 1]; its expected value is zero, and on its support the density function is constant and given by _f_ _X_ ( _x_ ) =  . Now, define random variable _Y_ as

Clearly, there is a very strong interdependence between _X_ and _Y_ because, given the realization of _X, Y_ is perfectly predictable. However, their covariance is zero! We know from Eq. (3.28) that

but the second term is clearly zero, as E[ _X_ ] = 0. Furthermore,

because of the symmetry of the integrand function, which is an odd function, in the sense that _f_ (− _x_ ) = − _f_ ( _x_ ). Hence, the first term is zero as well, and the two variables have zero covariance and are uncorrelated. One intuitive way to explain the weird finding of this example is the following. First note that points with coordinates ( _X, Y_ ) lie on the upper half of the unit circumference _X_ 2 \+ _Y_ 2 = 1. But if _Y_ < E[ _Y_ ], we may have either _X_ > E[ _X_ ] or _X_ < E[ _X_ ]. In other words, if _Y_ is above its mean, there is no way to tell whether _X_ will be above or below its mean. This is illustrated in Fig. 3.19. A similar consideration applies when _Y_ > E[ _Y_ ].

**FIGURE 3.19** **A counterexample about covariance.**

The example shows that covariance is not really a perfect measure of dependence, as it may be zero in cases in which there is a very strong dependence. In fact, covariance is rather a measure of _concordance_ between a pair of random variables, which is related to the signs of the deviations with respect to the mean. Furthermore, covariance measures a _linear_ association between random variables. A strong nonlinear link, as the one in Fig. 3.19, may not be detected at all, or only partially.

## 3.3.3 R FUNCTIONS FOR COVARIANCE AND CORRELATION

In real life, covariances and correlations must be estimated on the basis of empirical data. We will consider estimation issues in the next chapter, but R offers two basic and easy-to-use functions to estimate covariance and correlation. Not quite surprisingly, they are called cov and cor. To illustrate them, let us generate two independent samples from the standard normal distribution, and combine the observations in order to induce correlation. It is easy to show, and is left as an exercise, that if variables _Z_ 1, _Z_ 2 ~ N(0, 1) are independent, then the variables _X_ 1 and _X_ 2 defined as

(3.34)

are still standard normal but have correlation ρ:

    > set.seed(55555)
    > rho <- 0.8
    > Z1 <- rnorm(100000)
    > Z2 <- rnorm(100000)
    > X1 <- Z1
    > X2 <- rho*Z1 + sqrt (1-rho^2) *Z2
    > cor(Z1,Z2)
    [1] −0.00249354
    > cor(X1,X2)
    [1] 0.8002211
    > cov(X1,X2)
    [1] 0.7978528

Note that covariance and correlation, since the two variables are standard, should be the same, but they are not because of sampling errors. In the above snapshot, we have invoked cov and cor by passing two vectors as input arguments. By doing so, we obtain covariance and correlation between the corresponding variables. If we are interested in covariance and correlation matrices, we should collect the input data in a matrix:

    > X <- as.matrix(cbind(2*X1,-3*X2))
    > cov(X)
    [,1] [,2]
    [1,] 3.996719 −4.787117
    [2,] −4.787117 8.954152
    > cor(X)
    [,1] [,2]
    [1,] 1.0000000 −0.8002211
    [2,] −0.8002211 1.0000000
    > cov2cor(cov(X))
    [,1] [,2]
    [1,] 1.0000000 −0.8002211
    [2,] −0.8002211 1.0000000

Note the use of the function cbind to collect two vectors side by side as columns, and as.matrix to transform the result to a matrix. The function cov2cor can be used to transform the former into the latter. Once again, we suggest to compare the sample estimates with the theoretical values.

If you try getting online help on cor, you will notice that there is an additional parameter, method = c ("pearson", "kendall", "spearman"). By default, we obtain Pearson's correlations, but there are alternative definitions of correlation, which we discuss later in Section 3.4. To understand the rationale between them, let us consider the following snapshot:

    > set.seed(55555)
    > X=runif (100000)
    > cor(X,X)
    [1] 1
    > cor(X,X^2)
    [1] 0.9683232
    > cor(X,X^4)
    [1] 0.8660849

As expected, the correlation between _X_ and itself is 1, but if we raise _X_ to increasing powers, we decrease correlation. This is due to the nonlinearity in _X_ 2 and _X_ 4, which is lost in a linear measure of correlation. The larger the power, the less linear the resulting relationship, which is reflected in a diminishing value of Pearson's correlation. However, we may well argue that, at a deeper level, there is no substantial difference in the dependence between _X_ and itself and between _X_ and _X_ 4. We will see that Kendall's and Spearman's correlations avoid this issue.

## 3.3.4 SOME TYPICAL MULTIVARIATE DISTRIBUTIONS

In this section we illustrate two commonly used multivariate distributions, namely, the multivariate counterparts of normal and _t_ distributions. They are similar in the sense that:

**1.** They are both characterized by the vector of expected values μ and the covariance matrix Σ.

**2.** They belong to the family of elliptical distributions.

The essential difference is that, just like the univariate case, the _t_ distribution has fatter tails, which are related to the number of degrees of freedom.

### 3.3.4.1 Multivariate normal

An _n_ -dimensional multivariate normal variable  , has PDF

(3.35)

Strictly speaking, this applies to a nonsingular distribution, i.e., when Σ is positive definite. If the covariance matrix is only positive semidefinite, then there is a linear combination of variables, with coefficients **w** , such that

From a statistical point of view this means that there is some redundancy in the variables, since we may find a linear combination that is constant; from a mathematical point of view this implies that the covariance matrix cannot be inverted and we cannot write the PDF as in Eq. (3.35).

We immediately see that, when μ, = 0 and Σ = **I** , the joint density boils down to

i.e., the product of _n_ independent standard densities. We may obtain a generic multivariate normal from the standard **Z** ~ **N** ( **0, I** ), and the other way around, by the transformations

(3.36)

where **A** is a matrix such that Σ = **AA** . These formulas are straightforward generalizations of typical standardization (also called _whitening_ in this context) and destandardization. Possible choices of **A** are the lower triangular Cholesky factor **L** of Σ, or its square root Σ−1/2; unlike the Cholesky factor, the square root is symmetric and is defined by Σ = Σ−1/2Σ−1/2.

###   Example 3.6 A Cholesky factor

Let us consider two standard normal variables with correlation coefficient ρ. The covariance matrix and the corresponding lower triangular Cholesky factor are

The reader is urged to check that indeed Σ = **LL**  , and that using this matrix in Eq. (3.36) generates correlated normals as in Eq. (3.34).

The following snapshot shows two R functions, chol and sqrtm from package expm to carry out the above factorizations:

    > Sigma <- matrix(c(10,2, 2, 8), nrow=2, ncol=2)
    > A <- chol (Sigma)
    > t(A) %*% A
    [,1] [,2]
    [1,] 10 2
    [2,] 2 8
    > library(expm)
    > sqrtSigma <- sqrtm(Sigma)
    > sqrtSigma
    [,1] [,2]
    [1,] 3.1443790 0.3359774
    [2,] 0.3359774 2.8084015
    > sqrtSigma %*% sqrtSigma
    [,1] [,2]
    [1,] 10 2
    [2,] 2 8

Note that chol actually returns the upper triangular factor **L** . As we shall see in Chapter 5, these transformations are also used to generate samples of correlated multivariate normals from a set of independent standard variates.

We may generalize the well-known property that a linear combination of (jointly) normals is normal as follows.

**PROPERTY 3.6** _Let_ **X** 1, **X** 2,..., **X** _r_ _be m_ _i_ - _dimensional, independent normal variables_ , **X** _i_ ~ **N** (μ _i_ , Σ _i_ ), _i_ = 1,..., _r. Consider a vector_   _and matrices_  . _Then_

Later, we will appreciate the usefulness of the following theorem.

**THEOREM 3.7 (Conditional distribution for multivariate normals)** _Let us consider an n-dimensional multivariate normal variable X_ ~ **N** (μ, Σ) _and the partition_ **X** = [ **X** 1, **X** 2] , _where_ **X** 1 _and_ **X** 2, _have dimensions n_ 1 _and n_ 2, _respectively, with n_ 1 \+ _n_ 2 = _n. Let us define the corresponding partitions_

_where vectors and matrices have the appropriate dimensions. Then, the conditional distribution of X_ 1, _given that X_ 2 = _x, is_  , _where_

###   Example 3.7 Conditioning in a bivariate normal

Consider a bivariate normal distribution consisting of two standard normals _Z_ 1 and _Z_ 2 with correlation ρ. Then

and by Theorem 3.7 the distribution of _Z_ 1 conditional on _Z_ 2 = _z_ is normal with

The last theorem is useful for path generation by the Brownian bridge (Section 6.2.3) and variance reduction by stratification (Section 8.5).

### 3.3.4.2 Multivariate Student's _t_ distribution

The multivariate _t_ distribution is characterized by the PDF

(3.37)

With respect to the multivariate normal, there is an additional parameter, ν, corresponding to the degrees of freedom, and the notation tν(μ, Σ) is used. The essential moments are

We notice that when ν → ∞, the covariance matrix tends to Σ, just like in the univariate case. More generally, the PDF of the multidimensional _t_ tends to the PDF of a multivariate normal when ν → ∞. The matrix Σ, in this case, is referred to as a scale parameter, whereas ν is a shape parameter. The most relevant difference between the multivariate normal and _t_ distributions is seen in the tails, which are influenced by ν. If we set aside the complicated normalization factors involving the gamma function and look inside the big square parentheses in Eq. (3.37), we notice that the level curves of the PDF are essentially determined by the same term as in the normal case:

If we fix the value of this quantity and draw the corresponding level curve, we obtain an ellipse, which explains why we speak of elliptical distributions. Both distributions are symmetrical with respect to the expected value.

### 3.3.4.3 The mvtnorm package

The mvtnorm package offers functions to calculate the PDF and to generate random samples from multivariate normal and _t_ distributions:

  dmvnorm (x, mean, sigma, log=FALSE) gives the PDF for a multivariate normal; the parameter log can be used to obtain the log-density, i.e, the logarithm of the PDF, which may be useful to "compress" the range of values.

  The function for the _t_ distribution, dmvt (x, delta, sigma, df, log = TRUE, type = "shifted"), has a different default for log; the parameter delta is interpreted as a shift, i.e., a noncentrality parameter.

The corresponding r functions for random variate generation are rmvnorm and rmvt; there are also q and p functions, but they are of course more problematic than the corresponding univariate functions. The script in Fig. 3.20 shows how to use R functions to plot the _logarithm_ of the density of these two distributions. The resulting contour plots are displayed in Fig. 3.21. We should not be surprised by the negative values, since we are displaying log-densities to make the plots more readable. We immediately observe the elliptic character of both densities and the effect of positive correlation. The most striking difference, as we have remarked, is in the fatter tails of the _t_ distribution.

**FIGURE 3.20** **Script to illustrate the** mvtnorm **package.**

**FIGURE 3.21** **Contour plots of the log-density of multivariate normal and _t_ distributions.**

# 3.4 Modeling dependence with copulas

The idea behind copulas is to factor an _n_ -dimensional multivariate distribution into two components:

**1.** A set of _n_ univariate marginal CDFs, _F_ 1( _x_ 1),..., _F_ _n_ ( _x_ _n_ ). Note that CDFs are used, rather than PDFs; incidentally, this allows for more generality, since CDFs have wider applicability than PDFs and PMFs, even though in the following treatment we will only consider continuous distributions.

**2.** A function _C_ ( _u_ 1,..., _u_ _n_ ), called copula, which captures the dependence among the random variables in a "standard" way. The copula maps the _n_ -dimensional hypercube [0, 1] _n_ into the unit interval [0, 1], and it may be interpreted as the joint CDF of a multivariate distribution with uniform marginals.

The idea is to express the _n_ -dimensional CDF as

Indeed, since _C_ (···) must map a set of marginal probabilities into another probability, it must be a function mapping [0, 1] _n_ into [0, 1]. However, there are additional requirements that we detail below, referring to a two-dimensional copula for the sake of simplicity.

Consider a two-dimensional copula _C_ ( _u_ 1, _u_ 2). A reasonable requirement is that, for any _u_ 1, _u_ 2   [0, 1],

This amounts to saying that if _u_ 1 = P{ _X_ 1 ≤ _x_ 1} = 0, then _F_ X( _x_ 1, _x_ 2) = 0 for whatever value of _x_ 2. In fact, below the support of the distribution of _X_ 1, the CDF must be zero; or, if the support is unbounded below, the limit for _x_ 1 → −∞ must be 0. A similar consideration applies to the second variable _x_ 2, as well as to any variable in a multidimensional case. This leads to the following general definition.

**DEFINITION 3.8 (Grounded function)** _Consider a function H_ (···) _of n variables, with domain S_ 1 × ··· × _S_ _n_ , _where each set S_ _k_ _has a smallest element a_ _k_. _We say that H is grounded if H_ ( _y_ 1,..., _y_ _n_ ) = 0, _whenever at least one y_ _k_ = _a_ _k_.

Thus, we have to require that a copula is a grounded function. In the case of a copula, _S_ _k_ = [0, 1] and _a_ _k_ = 0. After looking below the support, let us move to the opposite side and have a look above it. There, a CDF must take the value 1; or, if the support is unbounded above, the limit for _x_ _k_ → +∞ must be 1. More generally, the copula should have sensible margins.

**DEFINITION 3.9 (Margins)** _Consider a function H_ (···) _of n variables with domain S_ 1 × ··· × _S_ _n_ , _where each set S_ _k_ _has a largest element b_ _k_. _The one-dimensional margin of H corresponding to the kth variable is the function_

_It is possible to define higher-dimensional margins, but we will simply refer to one-dimensional margins as margins._

On the basis of this definition, we see that a copula _C_ (···) should have margins _C_ _k_ (·) such that

As a last requirement, consider a two-dimensional CDF and the region illustrated in Fig. 3.22. The darkest rectangle in the figure is the set [ _a_ 1, _b_ 1] × [ _a_ 2, _b_ 2]. Let us denote this set by _S_ ; given the CDF _F_ X( _x_ 1, _x_ 2), it is easy to see that

**FIGURE 3.22** **Illustrating a property of multidimensional CDFs.**

Since this is a probability, the expression on the right must be positive. This leads to the following definition.

**DEFINITION 3.10 (2-increasing function)** _Consider a function H_ (·, ·) _of two variables, and let S_ = [ _a_ 1, _b_ 1] × [ _a_ 2, _b_ 2] _be a rectangle with all vertices in the domain of H_ (·, ·). _We define the H-volume of S as_

_We say that the function is 2-increasing if V_ _H_ ( _S_ ) ≥ 0 _for any such rectangle._

The definition may be extended to _n_ -dimensional functions, but we omit the technicalities involved. It should be noted that the 2-increasing property does not imply and is not implied by the fact that the function is nondecreasing in each of its arguments. This is well illustrated by the following counterexamples.

###   Example 3.8 Nondecreasing vs. 2-increasing functions

Consider the function

defined on _I_ 2 = [0, 1]2. It is immediate to check that the function is nondecreasing in _x_ 1 and _x_ 2. However, it is not 2-increasing, as the _H_ -volume of _I_ 2 is

On the contrary, let us consider

defined on _I_ 2 = [0, 1]2. Clearly, the function is decreasing in _x_ 1 for _x_ 2 < 0.5, and it is decreasing in _x_ 2 for _x_ 1 < 0.5. Nevertheless, it is 2-increasing. To see this, consider the rectangle _S_ = [ _a_ 1, _b_ 1] × [ _a_ 2, _b_ 2], where _b_ 1 ≥ _a_ 1 and _b_ 2 ≥ _a_ 2, and compute its _G_ -volume:

All of the above considerations are summarized in the following definition.

**DEFINITION 3.11 (Copula)** _An n-dimensional copula is a function C_ (···), _defined on the domain I_ _n_ = [0, 1] _n_ , _such that:_

**1.** _C_ (···) _is grounded and n-increasing._

**2.** _C_ (···) _has margins C_ _k_ , _k_ = 1,..., _n_ _satisfying C_ _k_ ( _u_ ) = _u, for u_   [0, 1].

We have stated requirements such that, given an _n_ -dimensional distribution with marginal CDFs _F_ _k_ , the function

(3.38)

is a sensible CDF. One might wonder whether we can also go the other way around: Given a joint CDF, is it possible to decompose it in the form of Eq. (3.38)? The answer is positive, thanks to the following fundamental theorem.

**THEOREM 3.12 (Sklar's theorem)** _Let F_ X _be an n-dimensional CDF with continuous marginal CDFs F_ 1(·),..., _F_ _n_ (·). _Then, there exists a unique copula C_ (···) _such that_

Note that the above theorem statement asserts uniqueness for _continuous_ distributions. In fact, Sklar's theorem also states the existence of a copula for discrete distributions, but there are technical issues concerning uniqueness that we want to avoid here. Since we are just dealing with continuous distributions, we assume invertibility of marginal CDFs and rewrite Eq. (3.38) as

(3.39)

When looking at this equation, it is important to realize that if we evaluate a marginal CDF _F_ _k_ ( _x_ ) for _x_ = _X_ _k_ , where _X_ _k_ is a random variable characterized by that CDF _F_ _k_ , then _F_ _k_ ( _X_ _k_ ) is a uniform random variable. To see this, note that, assuming invertibility of the margin and taking _u_   [0, 1],

(3.40)

However, this is just the CDF of a uniform distribution. This fact plays a fundamental role in random variate generation. Indeed, having defined the general concept of a copula, the following tasks must be typically carried out:

**1.** Choose a copula function, along with marginals, to characterize a multivariate distribution of interest.

**2.** Estimate the parameters characterizing the copula.

**3.** Generate random variates based on the selected copula.

We leave the second and the third task to Chapters 4 and 5, respectively. Here we describe a few common copula functions as examples.

### The product copula

The product copula is defined as

(3.41)

It is no surprise that, as it turns out, the continuous random variables in the vector ( _X_ 1,..., _X_ _n_ )  are independent if and only if their copula is the product copula.

### The normal copula

The normal (or Gaussian) copula is obtained by using the CDF of standard normals in Eq. (3.40). Let Φ be the CDF of a standard normal and Φ **R** _n_ be the joint CDF for an _n_ -dimensional standard normal distribution with correlation matrix **R**. Then, we may define the normal copula function

(3.42)

Note that this copula cannot be expressed analytically; nevertheless, it is possible to sample from the normal copula.

### The _t_ copula

The _t_ copula is built much like the normal copula, on the basis of the CDF of a _t_ distribution with ν degrees of freedom:

(3.43)

where _t_ ν is the CDF of a _t_ distribution with ν degrees of freedom and   is the CDF of a multivariate _t_ distribution with ν degrees of freedom and correlation matrix **R**.

## 3.4.1 KENDALL'S TAU AND SPEARMAN'S RHO

We have already illustrated a few pitfalls of Pearson's correlation:

  It is only a measure of concordance, rather than dependence; in fact, it essentially captures linear associations.

  It is affected by transformations in the data, even when such transformations do not really affect the underlying association and true dependence.

This has lead to the proposal of different correlation coefficients.

**DEFINITION 3.13 (Kendall's tau)** _Consider independent and identically distributed random vectors_ **X** = [ _X_ 1, _X_ 2]  _and_ **Y** = [ _Y_ 1, _Y_ 2] . _Kendall's tau is defined as_

In plain words, Kendall's tau measures the probability of concordance minus the probability of discordance. Note that in covariance, and as a consequence in defining Pearson's correlation, we take an expected value, which is influenced by the numerical values attained by the random variables. Here we are using a probability in a way that captures some more essential features. This is essentially why tau is not sensitive to monotonic transformations.

**DEFINITION 3.14 (Spearman's rho)** _Consider a joint CDF F_ x( _x_ 1, _x_ 2), _with marginals F_ 1(·) _and F_ 2 (·). _Spearman's rho is defined as the Pearson's correlation between the transformed random variables F_ 1( _X_ 1) _and F_ 2( _X_ 2):

(3.44)

To get an intuitive grasp of the definition, let us recall that _F_ 1 ( _X_ 1) and _F_ 2( _X_ 2) are uniformly distributed. Hence, in the definition of ρ _S_ we pick the two random variables and map them into uniform variables through their CDF, which is a monotonic transformation into the unit interval [0, 1]. Hence, we are actually correlating the _ranks_ of the realizations of the two variables. Indeed, Spearman's rho is also known as rank correlation. We should mention that sample counterparts have been defined for both Kendall's tau and Spearman's rho, and that there is a deep connection between them and copulas, as well as alternative definitions.

###   Example 3.9 Kendall's tau and Spearman's rho

The script in Fig. 3.23 illustrates the fact that Spearman's rho and Kendall's tau are not influenced by certain transformations of the underlying random variables. For our illustration purposes, we are using their sample counterparts here. We generate a sample of standard normal variables _X_ 1 and _X_ 2, with correlation 0.8. Then we take the exponential of the first sample, _F_ 1 = _e_ _X_ 1, and we apply the cor function to estimate different correlations. By the way, in R there is no need to specify the pearson method, as it is the default; hence cor (X1,x2,method='pearson') and cor (X1, X2) are equivalent. Running the script, we obtain

**FIGURE 3.23** **Illustrating invariance properties of Spearman's rho and Kendall's kappa.**

    > set.seed(55555)
    > tryCorrelation()
    Correlations
        Pearson: (X1,X2)= 0.8091203 (F1,X2)= 0.6192021
        Spearman: (X1,X2)= 0.7952385 (F1,X2)= 0.7952385
        Kendall: (X1,X2)= 0.6000901 (F1,X2)= 0.6000901

We observe that Pearson's correlation for the transformed variable is smaller: This is due to the nonlinear nature of the exponential transformation, which in this case weakens the linear relationship between the two random variables. The other two correlation measures are not affected.

## 3.4.2 TAIL DEPENDENCE

A relevant feature of multivariate distributions, especially for risk management, is tail dependence. The idea is focusing on the joint behavior of random variables when extreme events occur. For the sake of simplicity, we deal here with the upper tail dependence for two random variables _X_ 1 and _X_ 2; the case of lower tail dependence is similar.

**DEFINITION 3.15 (Upper tail dependence)** _Let us consider two jointly continuous random variables X_ 1 _and X_ 2 _with marginal CDFs F_ 1(·) _and F_ 2(·). _The coefficient of upper tail dependence is defined as_

(3.45)

_provided that the limit exists. If_ λ _U_ = 0, _then the two random variables are said to be asymptotically independent in the upper tail; otherwise, they are said to be asymptotically dependent in the upper tail._

Since _u_ is reaching its upper limit in the definition, the coefficient λ _U_ captures the joint behavior of _X_ 1 and _X_ 2 in the limit of the upper-right (North–East) quadrant. It turns out that this property is related to the copula, which means that it is invariant for strictly increasing transformations of the random variables.

Let us recall the following formula from elementary probability:

(3.46)

where   is the complement of event _E._ It is easy to see that the probability in Eq. (3.45) can be rewritten as

(3.47)

Therefore, using the copula _C_ (·, ·) between _X_ 1 and _X_ 2 and assuming that the limit exists, we may rewrite the above probability as

(3.48)

An interesting feature of the normal copula is that, when the correlation coefficient ρ is strictly less than 1, it can be shown that λ _U_ = 0, hence, we have tail independence. It is often argued that this fact points out another limitation of the multivariate normal distribution for risk management.

# 3.5 Linear regression models: A probabilistic view

Linear regression models are traditionally introduced within an inferential statistics framework. We will see how R can be used to accomplish classical estimation by ordinary least squares in Section 4.4. However, a probabilistic view is also possible, where we consider the problem of approximating a random variable _Y_ by an affine transformation _a_ \+ _bX_ of another random variable _X._ This is quite useful in order to better appreciate:

  Certain data reduction methods, which are dealt with later in Section 3.8

  Variance reduction based on control variates, which is discussed in Section 8.3

The key point is how to define a measure of distance between _Y_ and _a_ \+ _bX_. One possibility is to consider the variance of their difference. If this variance is close to zero, we may argue that the approximation tracks its target pretty well. If we do so and minimize tracking error variance, we formulate the following distance:

which is minimized with respect to _b_ by setting

(3.49)

We immediately see one issue with the above choice: The coefficient _a_ plays no role. A reasonable choice is to select _a_ such that the approximation is unbiased:

(3.50)

The reader familiar with linear regression in inferential statistics will find the above formulas quite familiar; in fact, least-squares estimators of the coefficients are just sample counterparts of the expressions in Eqs. (3.49) and (3.50). We may also set the two coefficients together, in a more convincing way, by minimizing the mean square deviation, rather than variance:

Using the above calculation and a well-known property of variance, the objective function may be rewritten as follows:

Now we enforce the optimality conditions, taking partial derivatives with respect to _a_ and _b_ , respectively. The first condition,

yields the result in Eq. (3.50), whereas the second one is

which yields the coefficient in Eq. (3.49).

Within this framework, it is also easy to see that we obtain a kind of "orthogonal projection," in the sense that the approximation _a_ + _bX_ and the residual _Y_ − ( _a_ \+ _bX_ ) are uncorrected:

Thus, the optimal approximation uses the maximum information about _Y_ that can be included in _bX_ ; the residual variability is orthogonal.

# 3.6 Time series models

The previous section was mainly aimed at cross-sectional dependence, i.e., the relationship between different random variables observed at the same time. In this section we move on to longitudinal dependence and consider discrete-time models based on time series. Time series models are an important topic within econometrics, and here we can only afford to scratch the surface. In doing so, we have the following objectives:

  To offer a quick outline of basic models for the sake of unfamiliar readers

  To show some R functions for the analysis and the analysis of time series data

  To illustrate how R can be used to simulate sample paths

In the next chapter we will see some more R functions to estimate time series models. Nevertheless, the tone in this section is a bit different from the rest of the chapter. The theory behind time series is far from trivial, and we will rely on simple simulations to get an intuitive feeling for them. Furthermore, some concepts we use here, such as the analysis of autocorrelation, are a bit halfway between probability and statistics. We include them here as they are part of the toolbox for model building. In fact, applying time series requires the following steps:

**1.** _Model identification_ : We should first select a model structure, i.e., its type and its order.

**2.** _Parameter estimation_ : Given the qualitative model structure, we must fit numerical values for its parameters.

We cover here the first task, and estimation methods are deferred to the next chapter. After a few preliminary considerations, we will move on to consider two fundamental classes of time series models:

  Moving-average models, in Section 3.6.1

  Autoregressive models, in Section 3.6.2

Then, in Section 3.6.3 we will see how these modeling approaches can be integrated in the more general class of ARIMA (autoregressive integrated moving average) processes, also known as Box–Jenkins models. The next step, in Section 3.6.4, is to generalize a bit, moving from scalar time series to multidimensional vector autoregressive models. Finally, in Section 3.6.5 we will present some more specific model classes, like ARCH and GARCH, to represent stochastic volatility. These models are quite relevant for financial applications. As we shall see, time series modeling offers many degrees of freedom, maybe too many; when observing a time series, it may be difficult to figure out which type of model is best-suited to capture the essence of the underlying process. This is why model identification plays a pivotal role.

In its basic form, time series theory deals with _weakly stationary processes_ , i.e., time series _Y_ _t_ with the following properties, related to first- and second-order moments:

**1.** The expected value of _Y_ _t_ does not change in time: E[ _Y_ _t_ ] = μ.

**2.** The covariance between _Y_ _t_ and _Y_ _t_ + _k_ depends only on time lag _k_ , and not on _t_.

The second condition deserves some elaboration.

**DEFINITION 3.16 (Autocovariance and autocorrelation)** _Given a weakly stationary stochastic process Y_ _t_ , _the function_

_is called_ **autocovariance of the process with time lag** _k. The function_

_is called the_ **autocorrelation function** _(ACF)._

The definition of autocorrelation relies on the fact that variance is constant as well:

In practice, autocorrelation may be estimated by the sample autocorrelation function (SACF), given a sample path _Y_ _t_ , _t_ = 1,..., _T_ :

(3.51)

where   is the sample mean of _Y_ _t_. The expression in Eq. (3.51) may not look quite convincing, since the numerator and the denominator are sums involving a different number of terms. In particular, the number of terms in the numerator is decreasing in the time lag _k_. Thus, the estimator looks biased and, for a large value of _k, R_ _k_ will vanish. However, this is what one expects in real life. Furthermore, although we could account for the true number of terms involved in the numerator, for large _k_ the sum involves very few terms and is not reliable. Indeed, the form of sample autocorrelation in Eq. (3.51) is what is commonly used in statistical software packages, even though alternatives have been proposed. If _T_ is large enough, under the null hypothesis that the true autocorrelations ρ _k_ are zero, for _k_ ≥ 1, the statistic   is approximately normal standard. Since _z_ 0.99 = 1.96 ≈ 2, a commonly used approximate rule states that if

(3.52)

then the sample autocorrelation at lag _k_ is statistically significant. For instance, if _T_ = 100, autocorrelations outside the interval [−0.2, 0.2] are significant. We should keep in mind that this is an approximate result, holding for a large number _T_ of observations. We may plot the sample autocorrelation function at different lags, obtaining an _autocorrelogram_ that can be most useful in pointing out hidden patterns in data.

###   Example 3.10 Detecting seasonality with autocorrelograms

Many time series in economics display seasonality. Consider the time series displayed on the left in Fig. 3.24. A cursory look at the plot does not suggest much structure in the data, but we may try to get a clue by plotting the ACF on the right. However, the ACF does not look quite helpful either. We notice decreasing autocorrelations, many of which are significant; the plot displays two horizontal bands outside which autocorrelation is statistically significant. Notice that

**FIGURE 3.24** **A seasonal time series and its autocorrelogram.**

and the positioning of the two horizontal bands is consistent with Eq. (3.52). In fact, the time series is seasonal and has been generated by simulating the model

which features an additive seasonal component _S_ _t_ superimposed on a linear trend _a_ \+ _bt_ , plus a noise term  _t_. The following **R** code shows how such a model can be simulated, and how the acf function can be used to plot the SACF:

    set.seed(55555)
    t <- 1:40
    factors <- c(15, 30, −10, −35)
    Y <- 100+10*t+factorS[t%%4+1]+rnorm(length(t),0,20)
    par(mfrow=c(1,2))
    plot(t,Y,type=′1′)
    points(t,Y,pch=15)
    acf(Y)

The seasonal factors are collected in the vector (15, 30, −10, −35); note that the average factor, for additive models, is zero, and that the indexing t%%4+1 repeats the four factors in a cycle over the simulation horizon; we recall that the operator %% corresponds to the mathematical operator mod and yields the remainder of the integer division. The noise terms  _t_ are normally distributed, which need not be the case in general. The problem actually is in the trend. We should detrend the series in order to see more structure, which can be accomplished by linear regression:

    mod <- lm(Y~t)
    dY <- Y-mod$fitted
    plot(t,dY,type=′1′)
    points(t,dY,pch=15)
    acf(dY)
    par(mfrow=c(1,1))

The first statement above uses lm to estimate a linear regression model, relating variable _Y_ with time _t_ ; the model is assigned to the variable mod, which is a rich data structure including fitted values, among other things. By subtracting the fitted trendline, we detrend the time series. The new ACF, displayed in Fig. 3.25 shows a quite different pattern: The autocorrelogram is indeed a useful tool to spot seasonality and other hidden patterns in data.

**FIGURE 3.25** **Detrended seasonal time series and its autocorrelogram.**

In the above example, we have used an important building block in time series modeling, called _white noise_ , which is a sequence of i.i.d. (independent and identically distributed) random variables. If they are normal, we have a Gaussian white noise. In other models, noise may be "colored," i.e., there may be some autocorrelation between the driving random terms  _t_. As we have mentioned, the first step in modeling a time series is the identification of the model structure. In the following we outline a few basic ideas that are used to this aim, pointing out the role of autocorrelation in the analysis.

## 3.6.1 MOVING-AVERAGE PROCESSES

A _finite-order moving-average process of order q_ , denoted by MA( _q_ ), can be expressed as

where random variables  _t_ are white noise, with E[  _t_ ] = 0 and Var(  _t_ ) = σ2. These variables play the role of _random shocks_ and drive the process. It is fairly easy to see that the process is weakly stationary. In fact, a first observation is that expected value and variance are constant:

The calculation of autocovariance is a bit more involved, but we may take advantage of the uncorrelation of white noise:

As a consequence, the autocorrelation function is

(3.53)

Thus, the autocorrelation function depends only on the lag _k_. We also notice that the autocorrelation function cuts off for lags larger than the order of the process. This makes sense, since the process _Y_ _t_ is a moving average of the driving white noise  _t_. Hence, by checking whether the sample autocorrelation function cuts off after a time lag, we may figure out whether a time series can be modeled as a moving average, as well as its order _q_. Of course, the sample autocorrelation will not be exactly zero for _k_ > _q_ ; nevertheless, by using the autocorrelogram and its significance bands, we may get some clue.

###   Example 3.11 Moving-average processes

Let us consider a simple MA(1) process

where  _t_ is a sequence of uncorrected standard normal variables (Gaussian white noise). In Fig. 3.26 we show a sample path and the corresponding sample autocorrelogram obtained by the following R code (note how vectorization avoids a for loop):

**FIGURE 3.26** **Sample path and corresponding sample ACF for the moving-average process _Y_ _t_ = 40 +  _t_ \+ 0.8  _t_ −1.**

    set.seed(55555)
    t <- 1:100
    epsVet <- rnorm(101)
    Y <- 40+epsVet[2:101]+0.8*epsVet[1:100]
    par(mfrow=c(1,2))
    plot(t,Y,type='l')
    points (t,Y,pch=15)
    acf(Y)

Incidentally, in this section we are writing our own code to simulate ARMA models for illustrative purposes, but R offers a ready-to-use function arima.sim, which will be used later in Section 4.5. The sample autocorrelation looks significant at time lag 1, which is expected, given the nature of the process. Note that, by applying Eq. (3.53), we find that the autocorrelation function, for a MA(1) process _Y_ _t_ = μ +  _t_ − θ1  _t_ −1, is

(3.54)

(3.55)

Figure 3.27 shows the sample path and autocorrelogram of a slightly different MA(1) process (the R code is essentially the same as above):

**FIGURE 3.27** **Sample path and corresponding ACF for the moving-average process _Y_ _t_ = 40 +  _t_ − 0.8  _t_ −1.**

The change in sign in θ1 has a noticeable effect on the sample path; an upswing tends to be followed by a downswing, and vice versa. The autocorrelogram shows a cutoff after time lag 1, and a negative autocorrelation.

If we increase the order of the process, we should expect more significant autocorrelations. In Fig. 3.28, we repeat the exercise for the MA(2) process

**FIGURE 3.28** **Sample path and corresponding ACF for the moving-average process _Y_ _t_ = 40 +  _t_ \+ 0.9  _t_ −1 \+ 0.5  _t_ −2.**

using the following R code:

    set.seed(55555)
    t <- 1:100
    epsVet <- rnorm(102)
    Y<-40+epsVet [3:102]+0.8*epsVet [2:101]+0.6*epsVet [1:100]
    par(mfrow=c(1,2))
    plot (t,Y,type='l')
    points(t,Y,pch=15)
    acf(Y)

We observe that, in this case, the autocorrelation function cuts off after time lag _k_ = 2.

We should mention that sample autocorrelograms are a statistical tool. It may well be the case that, for the moving-average processes in the examples, we get different pictures when different sample paths are randomly generated. This is a useful experiment to carry out with the help of statistical software.

## 3.6.2 AUTOREGRESSIVE PROCESSES

In finite-order moving-average processes, only a finite number of past realizations of white noise influence the value of _Y_ _t_. This may be a limitation for those processes in which all of the previous realizations have an effect, even though this possibly fades in time. In principle, we could consider an infinite-order moving-average process, but having to do with an infinite sequence of θ _q_ coefficients does not sound quite practical. Luckily, under some technical conditions, such a process may be rewritten in a compact form involving time-lagged realizations of the _Y_ _t_ itself. This leads us to the definition of an _autoregressive process_ of a given order. The simplest such process is the autoregressive process of order 1, AR(1):

(3.56)

One could wonder under which conditions this process is stationary, since here we cannot use the same arguments as in the moving-average case. A _heuristic_ argument to find the expected value μ = E[ _Y_ _t_ ] is based on taking expectations and dropping the time subscript in Eq. (3.56):

The argument is not quite correct, as it leads to a sensible result if the process is indeed stationary, which is the case if |ϕ| < 1. Otherwise, intuition suggests that the process will grow without bounds. The reasoning can be made precise by using the infinite-term representation of _Y_ _t_ , which is beyond the scope of this book. Using the correct line of reasoning, we may also prove that

In particular, we have

and we may also observe that, for a stationary AR(1) process,

(3.57)

We notice that autocorrelation is decreasing, but it fades away with no sharp cutoff.

###   Example 3.12 Autoregressive processes

In Figs. 3.29 and 3.30, we show a sample path and the corresponding sample autocorrelogram for the two AR(1) processes

**FIGURE 3.29** **Sample path and corresponding ACF for the autoregressive process _Y_ _t_ = 8 + 0.8 _Y_ _t_ −1 \+  _t_.**

**FIGURE 3.30** **Sample path and corresponding ACF for the autoregressive process _Y_ _t_ = 8 − 0.8 _Y_ _t_ −1 \+  _t_.**

respectively. The R code cannot avoid a for loop in this case:

    set.seed(55555)
    t <- 1:100
    epsVet <- rnorm(101)
    Y <- 40+epsVet[2:101]+0.8*epsVet[1:100]
    par(mfrow=c(1,2))
    plot (t,Y,type='l')
    points(t,Y,pch=15)
    acf(Y)

Notice that the change in sign in the ϕ coefficient has a significant effect on the sample path, as well as on autocorrelations. In the first case, autocorrelation goes to zero along a relatively smooth path. This need not be the case, as we are working with _sample_ autocorrelations. Nevertheless, at least for significant values, we observe a monotonic behavior consistent with Eq. (3.57). The sample path of the second process features evident up- and downswings; we also notice an oscillatory pattern in the autocorrelation.

The autocorrelation behavior of AR processes does not feature the cutoff properties that help us determine the order of an MA process. A tool that has been developed for the identification of AR processes is the _partial autocorrelation function_ (PACF). The rationale behind PACF is to measure the degree of association between _Y_ _t_ and _Y_ _t−k_ , removing the effects of intermediate lags, i.e., _Y_ _t_ −1,..., _Y_ _t−k_ +1. We cannot dwell too much on PACF, but we may at least get a better intuitive feeling as follows. Consider three random variables _X, Y_ , and _Z_ , and imagine regressing _X_ and _Y_ on _Z_ :

Note that we are considering a probabilistic regression, not a sample-based regression. From Section 3.5 we know that

Furthermore, we have regression errors

which may be regarded as the random variables _X_ and _Y_ , after the effect of _Z_ is removed. The correlation ρ( _X, Y_ ) may be large because of the common factor _Z_ (which plays the role of a "lurking" variable). If we want to get rid of it, we may consider the partial correlation ρ( _X_ *, _Y_ *). On the basis of this intuition, we might consider estimating the partial autocorrelation between _Y_ _t_ and _Y_ _t−k_ by the following linear regression:

The inclusion of intermediate lagged variables _Y_ _t_ −1,..., _Y_ _t−k_ +1 allows to capture their effect explicitly by the regression coefficients _b_ 1,..., _b_ _k_ −1. Then, we could use _b_ _k_ as an estimate of partial autocorrelation. Actually, this need not be the sounder approach, but software packages provide us with ready-to-use functions to estimate the PACF by its sample counterpart. In Fig. 3.31 we show the SPACF for the two AR(1) processes of Example 3.12. The plot was obtained by the pacf function. The example also shows how to use expression to build a mathematical annotation for the plot, including subscripts, etc.:

**FIGURE 3.31** **Sample partial autocorrelation function for the autoregressive processes of Example 3.12.**

    expr1 = expression(Y[t] == 8 + 0.8 %*% Y[t-1] + epsilon[t]) pacf(Y, main = expr1)

We see that the SPACF cuts off after lag 1, even though, due to statistical sampling errors, it seems that there is a significant value at larger lags in the first case. This suggests how to use this function to identify the order of an AR model.

## 3.6.3 ARMA AND ARIMA PROCESSES

Autoregressive and moving-average processes may be merged into ARMA (autoregressive moving-average) processes like

(3.58)

The model above is referred to as ARMA( _p, q_ ) process, for self-explanatory reasons. Conditions ensuring stationarity have been developed for ARMA processes, as well as identification and estimation procedures. Clearly, the ARMA modeling framework provides us with plenty of opportunities to fit historical data with a model that can be used to create scenarios by Monte Carlo sampling. However, it applies only to stationary data. It is not too difficult to find real-life examples of data processes that are nonstationary. Just think of stock market indices; most investors really wish that the process is not stationary.

###   Example 3.13 A nonstationary random walk

A quite common building block in many financial models is the _random walk_. An example of random walk is

(3.59)

where η _t_ is a sequence of independent and standard normal random variables. This is actually an AR process, but from Section 3.6.2 we know that it is nonstationary, as ϕ = 1. Three sample paths of this process are shown in Fig. 3.32. They look quite different, and a subjective comparison of the sample paths would not suggest that they are realizations of the same stochastic process. However, the ACFs show a common pattern: Autocorrelation fades out very slowly. Indeed, this is a common feature of nonstationary processes. The PACFs display a very strong partial autocorrelation at lag 1, which cuts off immediately in all of the three cases.

**FIGURE 3.32** **Three sample paths and the corresponding ACF and PACF for the random walk _Y_ _t_ = _Y_ _t_ −1 \+ 0.05η _t_.**

Since the theory of stationary MA and AR processes is well developed, it would be nice to find a way to apply it to nonstationary processes as well. A commonly used trick to remove nonstationarity in a time series is _differencing_ , by which we consider the time series

(3.60)

Applying differencing in R is obtained by the diff function, which is applied in the following R snapshot to the random walk of Eq. (3.59):

    set.seed(5)
    epsVet <- c(0, rnorm(100)) # first value is dummy
    Y <- c (4,numeric(100))
    for (t in 2:101)
    Y[t] <- Y[t-1] + 0.05*epsVet[t]
    dY <- diff(Y)
    par(mfrow=c(2,2))
    plot(1:100, Y[−1], type='l')
    points (1:100,Y[−1],pch=15)
    acf(Y)
    plot(1:100,dY,type='l')
    points(1:100,dY,pch=15)
    acf(dY)

In Fig. 3.33 we show the sample path and the differenced series, along with their ACFs. The shape of the ACF is not surprising, since the differenced process is just white noise.

**FIGURE 3.33** **The effect of differencing on the simple random walk ofEq. (3.59).**

###   Example 3.14 What is nonstationarity, anyway?

A time series with trend

(3.61)

where  _t_ is white noise, is clearly nonstationary, as it features a deterministic trend. However, nonstationarity is a subtler concept than that. A little digression is in order to clarify the nature of nonstationarity in a random walk like

(3.62)

The sample paths that we have generated in Example 3.13 show that this random walk does not feature a deterministic trend. The recursive unfolding of Eq. (3.62) yields

Therefore, we find that

Hence, we must have a different kind of nonstationarity in the random walk of Eq. (3.62) than in the process described by Eq. (3.61). To investigate the matter, let us consider the expected value of the increment _Y_ ' _t_ = _Y_ _t_ − _Y_ _t_ −1, conditional on _Y_ _t_ −1:

Therefore, given the last observation _Y_ _t_ −1, we cannot predict whether the time series will move up or down. This shows a connection between a stationary random walk and the efficient market hypothesis. Now, let us consider a stationary AR(1) process

where ϕ   (−1, 1). The increment in this case is

Since (ϕ − 1) < 0, we have

This suggests that a stationary AR(1) process is _mean reverting_ , in the sense that the process tends to return to its expected value; the nonstationary random walk does not enjoy this property.

If we introduce the _backshift operator B_ , defined by

we may express the first difference in Eq. (3.60) as

Sometimes, differencing must be repeated in order to obtain a stationary time series. We obtain _second-order differencing_ by repeated application of (first-order) differencing:

This suggests that we may formally apply the algebra of polynomials to the backshift operator, in order to find differences of arbitrary order. By introducing the operator polynomials

we can rewrite the ARMA model of Eq. (3.58) in the compact form:

(3.63)

We may also extend the class of stationary ARMA models, in order to allow for nonstationarity. Doing so, we find the more general class of ARIMA (autoregressive integrated moving average) processes, also known as _Box–Jenkins models_. An ARIMA( _p, d, q_ ) process can be represented as follows:

(3.64)

where Φ( _B_ ) and Θ( _B_ ) are polynomials of order _p_ and _q_ , respectively, and _d_ is a differencing order such that the process _Y_ _t_ is stationary, whereas, if we take differences of order ( _d_ − 1), the process is still nonstationary. The name "integrated" stems from the fact that we obtain the nonstationary process by integrating a stationary one, i.e., by undoing differencing. In most practical applications the order _d_ is 0 or 1. Box–Jenkins models can be extended to cope with seasonality, but a full account of this class of models is beyond the scope of this book; we refer the reader to the references provided at the end of this chapter.

## 3.6.4 VECTOR AUTOREGRESSIVE MODELS

If you want to generalize the scalar time series models that we have seen so far, in principle, there is no limit to what one can conceive. Simultaneous equations can be built, resulting in quite rich structures. In practice, an overly complicated model need not be the smartest option. On the one hand, there may be issues with estimation and the assessment of properties such as stationarity. On the other hand, the predictive power of a seemingly rich model may be disappointing due to overfitting issues. This is why vector autoregressive models, VAR for short, have been proposed. Let **Y** _t_    _k_ be a vector of random variables; a VAR model can be represented in reduced form as

(3.65)

where ϕ( _B_ ) is a polynomial in the backshift operator _B_ and **c** ,  _t_    _k_. More explicitely, a VAR( _p_ ) model is

(3.66)

where  . Note that the error term  _t_ need not be a vector or independent variables.

###   Example 3.15 Simulation of a VAR(1) model

Let us consider the bidimensional model

where the error terms are standard normals with correlation ρ = 0.5. In matrix form, we may write the above equations as

where

and

The following R script produces the time series displayed in Fig. 3.34.

**FIGURE 3.34** **Bidimensional time series corresponding to a VAR(1) model.**

    require(MASS)
    tPhil <- t(matrix(c(0.7, 0.2, 0.2, 0.7), 2, 2))
    c.vec <- c(−0.7, 1.3)
    mu.vec <- numeric(2)
    cov.mat <- matrix(c(1, 0.5, 0.5, 1), 2, 2)
    set.seed(55555)
    T <- 100
    Ypath <- matrix(0, T, 2)
    Ypath[1,] <- c(1,1) # initial condition
    epsPath <- mvrnorm(n=T, mu.vec, cov.mat)
    for (t in 2:T)
        Ypath[t,] <- c.vec+Ypath[t-1,]%*%tPhil+epsPath[t,]
    plot.ts(Ypath)

Note that, in order to collect the time series in a matrix with rows corresponding to observations, we have to transpose Φ. We also need the MASS package to sample from a multivariate normal.

## 3.6.5 MODELING STOCHASTIC VOLATILITY

When observing financial time series of returns, the following stylized facts may be noticed:

  Volatility is not constant and there are volatility clusters, i.e., blocks of consecutive periods featuring high or low volatility.

  Returns are not quite symmetric, as there is a negative skew.

The first feature is referred to as heteroskedasticity, in contrast to the homoskedasticity assumption, whereby volatility is constant. To model volatility, it is convenient to build a time series of returns. However, returns over consecutive periods cannot be added, as they should rather be multiplied. This is why a series of log-returns _u_ _t_ is usually represented:

where _S_ _t_ is the price of the relevant financial asset. We want to capture the conditional variance of the time series, which can be expressed as:

It is often the case that E[ _u_ _t_ ] ≈ 0, if the time bucket is small enough. Hence, the expected return term is typically dropped. To build a model, we might relate volatility to the last shock:

(3.67)

A model like this is called ARCH(1), where the acronym stands for _autoregressive conditionally heteroskedastic_ , and 1 means that only _u_ _t_ with lag 1 plays a role in Eq. (3.67). This is just a part of an overall model describing _u_ _t_. We may build an ARCH( _q_ ) model as

(3.68)

An ARCH model is, in a sense, an MA-style model for volatility. We may also consider an AR-style model like

(3.69)

which is known as GARCH(1, 1), i.e., a generalized ARCH model involving two variables with lag 1.

A GARCH model should be coupled with another equation describing the evolution of a time series of interest. Let us consider a time series of return generated as an expected return plus a random shock _u_ _t_ :

where

Note that σ _t_ is the standard deviation, i.e., the volatility of the return time series, and it follows Eq. (3.69). One may use a fatter-tailed distribution to generate the shocks. The code in Fig. 3.35 generates the two plots displayed in Fig. 3.36.

**FIGURE 3.35** **Code to generate a sample path of a GARCH(1, 1) model.**

**FIGURE 3.36** **Sample path of a GARCH(1, 1) model.**

# 3.7 Stochastic differential equations

Many financial engineering models rely on continuous-time stochastic processes. One reason is that, in fact, time is continuous in the real world, but another one is that this modeling framework allows for the development of a rich theory. The theory is translated into practical models that allow for an analytical solution in some lucky cases, and even if we are not that lucky, we may still resort to numerical methods, including Monte Carlo. In this section we first illustrate a gentle transition from discrete- to continuous-time models, which leads to the introduction of a fundamental building block in continuous-time stochastic modeling, the Wiener process. Continuous-time models are typically expressed as stochastic differential equations, which not only generalize deterministic differential equations but also feature thorny technical issues that require the introduction of a new concept: The stochastic integral in the sense of Itô. This, in turn, leads to an essential result, the celebrated Ithon's lemma, which is in some sense an extension of the chain rule for the derivative of composite functions in familiar calculus. In fact, it turns out that the familiar calculus rules do not work in this context and must be suitably modified. This is essential in tackling stochastic differential equations analytically, when possible, and numerically via sample path generation. Then, we introduce the family of Itô stochastic differential equations, which includes geometric Brownian motion, and some generalizations. In the following, we will mostly consider the price of a stock share as the relevant variable, but we can also model interest rates, exchange rates, macroeconomic variables, etc.

## 3.7.1 FROM DISCRETE TO CONTINUOUS TIME

It is a good idea to start with a discrete-time model and then derive a continuous-time model heuristically. Consider a time interval [0, _T_ ], and imagine that we discretize it by a sequence of time periods of length δ _t_ , such that _T_ = _n_ · δ _t_ ; discrete-time instants are indexed by _t_ = 0, 1, 2,..., _n_. Let _S_ _t_ be, for instance, the price of a stock share at time _t_. One possible and reasonable model for the evolution of this price over time is the multiplicative form:

(3.70)

where _u_ _t_ is a non-negative random variable and the initial price _S_ 0 is known. If we consider continuous random variables _u_ _t_ , the model is continuous-state. In simplest model the variables _u_ _t_ are i.i.d., which may be justified by the efficient market hypothesis, if we believe in it (or by the lack of an alternative model we trust enough). The multiplicative model ensures that prices will stay non-negative, which is an obvious requirement for stock prices. If we used an additive model, such as _S_ _t_ +1 = _u_ _t_ \+ _S_ _t_ , we should admit negative values for the random variables _u_ _t_ , in order to represent price drops; therefore, we would not have the guarantee that _S_ _t_ ≥ 0. With the multiplicative form, a price drops when _u_ _t_ < 1, but it remains positive. Furthermore, the actual impact of a price change depends on the present stock price; a $1 increase is a different matter if the current price is $100 rather than $5. This is easily accounted for by the multiplicative form, which essentially deals with percent changes in price.

In order to select a sensible probability distribution for the random variables _u_ _t_ , let us use a typical trick of the trade and consider the natural logarithm of the stock price:

The random variable _z_ _t_ is the increment in the logarithm of price, and a common assumption is that it is normally distributed, which implies that _u_ _t_ is lognormal. Starting from the initial price _S_ 0 and unfolding (3.70) recursively, we find

which implies that

Since the sum of normal random variables is still a normal variable, log _S_ _t_ is normally distributed, which in turn implies that stock prices are lognormally distributed, according to this model. To put it in another way, the product of lognormals is still lognormal. Using the notation

we see that

(3.71)

(3.72)

where we take advantage of the intertemporal independence of the variables _z_ _t_ when calculating variance. The important point to see here is that the expected value and the variance of the increment in the logarithm of the stock price scale linearly with time; this implies that standard deviation scales with the square root of time, whereas expected value scales linearly. This property is known as the square-root rule and implies that on a very short time period volatility, related to a standard deviation, dominates drift, related to an expected value.

Our aim is to build a continuous-time model, so let us consider first what we typically do in a deterministic case to accomplish the task. The familiar approach is to take the limit of a difference equation, for δ _t_ → 0, and build a differential equation. Informally, if there were no uncertainty, we would recast what we have seen in discrete time as

Taking the limit as δ _t_ → 0, we find a relationship between differentials:

Integrating both differentials over the interval [0, _t_ ] yields

(3.73)

In the deterministic case, as we remarked earlier in this book, it is customary to write the differential equation as

or, equivalently, as

where we have used standard calculus, in particular the chain rule of derivatives, to rewrite the differential

(3.74)

To make the model stochastic (and more realistic), we have to include a noise term, which entails a few important changes. First, we should write the equation in the form

(3.75)

where _dW_ ( _t_ ) can be considered as the increment of a stochastic process over the interval [ _t, t_ \+ _dt_ ]. Equation (3.75) is an example of a rather tricky object, called a _stochastic differential equation_. It is reasonable to guess that the solution of a stochastic differential equation is a stochastic process, rather than a deterministic function of time. This topic is quite difficult to deal with rigorously, as it requires some background in measure theory and stochastic calculus, but we may grasp the essentials by relying on intuition and heuristic arguments.

The first thing we need is to investigate which type of continuous-time stochastic process _W_ ( _t_ ) we can use as a building block. In the next section we introduce such a process, called the Wiener process, which plays more or less the same role as the process _z_ _t_ above. It turns out that this process is not differentiable, whatever this may mean for a stochastic process; informally, we will see that its sample paths are quite jagged indeed. Hence, we **cannot** write the stochastic differential equation as

Actually, a stochastic differential equation must be interpreted as a shorthand for an _integral_ equation much like (3.73), involving the increments of a stochastic process. This calls for the definition of a _stochastic integral_ and the related stochastic calculus. A consequence of the definition of the stochastic integral is that working with differentials as in Eq. (3.74) is not possible. We need a way to generalize the chain rule for differentials from the deterministic to the stochastic case. This leads to a fundamental tool of stochastic calculus called _Itô's lemma._

## 3.7.2 STANDARD WIENER PROCESS

In the discrete-time model, we have assumed normally distributed increments in logarithmic prices, and we have also seen that the expected value of the increment of the logarithm of price scales linearly with time, whereas standard deviation scales with the square root of time.

In discrete time, we could consider the following process as a building block:

where  _t_ is a sequence of independent standard normal variables. We see that, for _k_ > _j_ ,

(3.76)

which implies that

In order to move to continuous time, we may define the standard Wiener process as a continuous-time stochastic process characterized by the following properties:

**1.** _W_ (0) = 0, which is actually a convention.

**2.** Given any time interval [ _s,t_ ], the increment _W_ ( _t_ ) − _W_ ( _s_ ) is distributed as N(0, _t_ − _s_ ), a normal random variable with expected value zero and standard deviation  . These increments are stationary, in the sense that they do not depend on where the time interval is located, but only on its width.

**3.** Increments are independent: If we take time instants _t_ 1 < _t_ 2 ≤ _t_ 3 < _t_ 4, defining two nonoverlapping time intervals, then _W_ ( _t_ 2) − _W_ ( _t_ 1) and _W_ ( _t_ 4) − _W_ ( _t_ 3) are independent random variables.

Note how the second condition makes sure that standard deviation scales with the square root of time, like in the discrete-time model. To see the importance of the independent increments assumption, let us consider the sample paths of a process defined as  , with   ~ N(0, 1), which are shown in Fig. 3.37. This is a "degenerate" stochastic process, since knowledge of just one point on a sample path yields the complete knowledge of the whole sample path, which makes the process quite predictable. However, if we only look at the marginal distributions of _Q_ ( _t_ ), this process just looks like the Wiener process, since

**FIGURE 3.37** **Sample paths of a "degenerate" stochastic process.**

It is lack of independence that makes the difference, as we may also check empirically. We may recast Eq. (3.76) as

(3.77)

where the notation  _t_ +δ _t_ , rather than  _t_ , may be used to emphasize the fact that this standard normal shock conveys new information. This discretization immediately allows us to write a piece of R code to generate sample paths of the standard Wiener process, which is illustrated in Fig. 3.38 and produces the plot in Fig. 3.39, which is quite different from Fig. 3.37, even though the marginals are the same.

**FIGURE 3.38** **Simple R function and script to sample the standard Wiener process.**

From Fig. 3.39, we observe that sample paths of the Wiener process look continuous, but not differentiable. This may be stated precisely, but not very easily. Introducing continuity and differentiability rigorously calls for specifying a precise concept of stochastic convergence, as we should say that the Wiener process is nowhere differentiable with probability 1. To get an intuitive feeling for this fact, let us consider the increment ratio

Given the defining properties of the Wiener process, it is easy to see that

If we take the limit for δ _t_ → 0, this variance goes to infinity. Strictly speaking, this is no proof of nondifferentiability of _W_ ( _t_ ), but it does suggest that there is some trouble in using an object like _dW_ ( _t_ )/ _dt_ ; indeed, you will never see a notation like this. We only use the _differential dW_ ( _t_ ) of the Wiener process. Informally, we may think of _dW_ ( _t_ ) as a random variable with distribution N(0, _dt_ ). Actually, we should think of this differential as an increment, which may be integrated as follows:

This looks reasonable, doesn't it? We may even go further and use _W_ ( _t_ ) as the building block of stochastic differential equations. For instance, given real numbers _a_ and _b_ , we may imagine a stochastic process _X_ ( _t_ ) satisfying the equation

(3.78)

This equation defines a _generalized Wiener process_ and may be solved by straightforward integration over the time interval [0, _t_ ]:

which may be rewritten as

Hence, for the generalized Wiener process we find

However, if we consider something more complicated, like

(3.79)

things are not that intuitive. A process satisfying an equation like (3.79) is called an Itô process. We could argue that the solution should be something like

(3.80)

Here the first integral looks like a standard Riemann integral of a function over time, but what about the second one? We need to assign a precise meaning to it, and this leads to the definition of a stochastic integral.

## 3.7.3 STOCHASTIC INTEGRATION AND ITÔ'S LEMMA

In a stochastic differential equation defining a process _X_ ( _t_ ), where a Wiener process _W_ ( _t_ ) is the driving factor, we may assume that the value _X_ ( _t_ ) depends only on the history of _W_ ( _t_ ) over the time interval from 0 to _t_. Technically speaking, we say that process _X_ ( _t_ ) is _adapted_ to process _W_ ( _t_ ). Now let us consider a stochastic integral like

(3.81)

How can we assign a meaning to this expression? To begin with, it is reasonable to guess that a stochastic integral is a random variable. If we integrate a deterministic function of time we get a number; so, it is natural to guess that, by integrating a stochastic process over time, we should get a random variable. Furthermore, the stochastic integral above looks related to the sample paths of process _W_ ( _t_ ), and an approximation could be obtained by partitioning the integration interval into small subintervals defined by points 0 = _t_ 0, _t_ 1, _t_ 2,..., _t_ _n_ = _T_ and considering the sum

(3.82)

If you recall how the Riemann integral was introduced in your first calculus course, as the limit of sums over partitions, the above definition does not look too different. Yet, there is a subtle and crucial point. It is very important to notice how we have chosen the time instants in the expression above: _X_ ( _t_ _k_ ) is taken at the left extreme of the interval ( _t_ _k_ , _t_ _k_ +1). This is actually _one_ possible choice: Why not take the value of _X_ in the midpoint? This choice has an important effect: _X_ ( _t_ _k_ ) is a random variable which is independent from the increment _W_ ( _t_ _k_ +1) − _W_ ( _t_ _k_ ) by which it is multiplied. This choice is what makes the stochastic integral in the sense of Itô a bit peculiar, and it may be motivated as follows.

###   Example 3.16 Financial motivation of Itô integrals

Consider a set of _n_ assets, whose prices are modeled by stochastic processes _S_ _i_ ( _t_ ), _i_ = 1,..., _n_ , which are described by stochastic differential equations like (3.79), and assume that we have a portfolio strategy represented by functions _h_ _i_ ( _t_ ). These functions represent the number of stock shares of each asset _i_ that we hold at time _t._ But which functions make sense? An obvious requirement is that functions _h_ _i_ (·) should not be anticipative: _h_ _i_ ( _t_ ) may depend on all the history so far, over the interval [0, _t_ ], but clairvoyance should be ruled out. Furthermore, we should think of _h_ _i_ ( _t_ ) as the number of shares we hold over a time interval of the form [ _t_ , _t_ \+ _dt_ ).

Now, assume that we have an initial wealth that we invest in the portfolio, whose initial value, depending on the portfolio strategy represented by the functions _h_ _i_ (·), is

where we have grouped _h_ _i_ ( _t_ ) and _S_ _i_ ( _t_ ) in column vectors. What about the dynamics of the portfolio value? If the portfolio is self-financing, i.e., we can trade assets but we do not invest (nor withdraw) any more cash after _t_ = 0, it can be shown that the portfolio value will satisfy the equation

This looks fairly intuitive and convincing, but some careful analysis is needed to prove it (see, e.g., [1, Chapter 6]). Then, we may reasonably guess that the wealth at time _t_ = _T_ will be

However, it is fundamental to interpret the stochastic integral as the limit of an approximation like (3.82), i.e.,

The number of stock shares we hold at time _t_ _k_ does _not_ depend on future prices **S** ( _t_ _k_ +1). First we allocate wealth at time _t_ _k_ , and _then_ we observe return over the time interval ( _t_ _k_ , _t_ _k_ +1). This makes financial sense and is why Itô stochastic integrals are defined the way they are.

Now, if we take approximation (3.82) and consider finer and finer partitions of the interval [0, _t_ ], letting _n_ → ∞, what do we obtain? The answer is technically involved. We must select a precise concept of stochastic convergence and check that everything makes sense. Using mean square convergence, it can be shown that the definition makes indeed sense and provides a rigorous definition of the stochastic integral in the sense of Itô.

The definition of stochastic integral has some important consequences. To begin with, what is the expected value of the integral in Eq. (3.81)? This is a good starting question, because the stochastic integral is a random variable, and the first and foremost feature of a random variable is its expected value. We may get a clue by considering the approximation (3.82):

where we have used the independence between _X_ ( _t_ _k_ ) and the increments of the Wiener process over the time interval ( _t_ _k_ , _t_ _k_ +1), along with the fact that the expected value of these increments is zero. This shows that the integral of Eq. (3.81) is a random variable with expected value zero, but can we say something more? The definition of stochastic integral does not yield a precise way to compute it practically. We may try, however, to consider a specific case to get some intuition. The following example illustrates one nasty consequence of the way in which we have defined the stochastic integral.

###   Example 3.17 Chain rule and stochastic differentials

Say that we want to "compute" the stochastic integral

Analogy with ordinary calculus would suggest using the chain rule of differentiation of composite functions to obtain a differential which can be integrated directly. Specifically, we _might guess_ that

This in turn _would_ suggest that

Unfortunately, this _cannot_ be the correct answer, as it contradicts our previous findings. We have just seen that the expected value of an integral of this kind is zero, but:

We see that the two expected values do not match at all, and there must be something wrong somewhere.

Example 3.17 shows that the chain rule does not work in Itô stochastic calculus. To proceed further, we need to find the right rule, and the answer is Itô's lemma. As usual in this section, we cannot follow rigorous arguments, but we may give an informal, yet quite instructive argument (following [12, Chapter 13]). The argument is instructive as it provides us with valuable intuition explaining what went wrong in Example 3.17. Let us recall that an Itô process _X_ ( _t_ ) satisfies a stochastic differential equation such as

(3.83)

which is in some sense the continuous limit of

(3.84)

where  ( _t_ ) ~ N(0, 1). What we actually need is a way to derive a stochastic differential equation for a function _F_ ( _X, t_ ) of _X_ ( _t_ ), as this plays the role of the derivative of a composite function, which is what the chain rule is used for.

Let us take a little step back in the realm of deterministic calculus, and consider what is the rationale behind Taylor's expansion of a function _G_ ( _x, y_ ) of two variables. The key ingredient we need for our reasoning is the formula for the differential of such a function:

which indeed may be obtained from Taylor's expansion:

when δ _x_ , δ _y_ → 0. Now we may apply this Taylor expansion to _F_ ( _X, t_ ), limiting it to the leading terms. In doing so it is important to notice that the term √δ _t_ in Eq. (3.84) needs careful treatment when squared. In fact, we have something like

which implies that the term in (δ _X_ )2 cannot be neglected in the approximation. Since   is a standard normal variable, we have E[ 2] = 1 and _E_ [ 2 δ _t_ ] = δ _t._ A delicate point is the following: It can be shown that, as δ _t_ tends to zero, the term  2 δ _t_ can be treated as nonstochastic, and it is equal to its expected value. An informal (far from rigorous) justification relies on the variance of this term:

which, for δ _t_ → 0, can be neglected with respect to first-order terms. Here we have used the fact E[ 4] = 3, which can be checked by using moment generating functions or, cheating a bit, by recalling that the kurtosis of any normal, including a standard one, is 3. A useful way to remember this point is the _formal_ rule

(3.85)

Hence, when δ _t_ tends to zero, in the Taylor expansion we have

Neglecting higher-order terms and taking the limit as both δ _X_ and δ _t_ tend to zero, we end up with

which, substituting for _dX_ , becomes the celebrated **Itô's lemma:**

(3.86)

Although this proof is far from rigorous, we see that all the trouble is due to the term of order  , which is introduced by the increment of the Wiener process.

It is instructive to see that if we set _b_ ( _X, t_ ) = 0 in the Itô differential equation, i.e., if we eliminate randomness, then we step back to familiar ground. In such a case, the stochastic differential equation is actually deterministic and can be rewritten as

where we use the notation _x_ rather than _X_ to point out the deterministic nature of function _x_ ( _t_ ). Then, Itô's lemma boils down to

(3.87)

which can be rearranged to yield the following chain rule for a function _F_ ( _x, t_ ):

(3.88)

In order to grasp Itô's lemma, let us consider Example 3.17 once more.

###   Example 3.18 Itô's lemma

In order to compute the stochastic integral of _W_ 2( _t_ ), we may apply Itô's lemma to the case _X_ ( _t_ ) = _W_ ( _t_ ), by setting _a_ ( _X,t_ ) ≡ 0, _b_ ( _X, t_ ) ≡ 1, and _F_ ( _X, t_ ) = _X_ 2( _t_ ). Hence, we have

(3.89)

(3.90)

(3.91)

It is important to point out that in Eq. (3.89) the partial derivative with respect to time is zero; it is true that _F_ ( _X_ ( _t_ ), _t_ ) depends on time through _X_ ( _t_ ), but here we have no direct dependence on _t_ , thus the _partial_ derivative with respect to time vanishes. To put it another way, the dependence of _X_ ( _t_ ) with respect to time does not play any role when taking the partial derivative with respect to _t_ , because _X_ ( _t_ ) is held fixed. You may also have a look back at Eq. (3.88) to see the relationship between the partial and a total derivative of _F_ ( _X_ ( _t_ ), _t_ ) with respect to _t._

The application of Itô's lemma yields

(3.92)

It is essential to notice that _dt_ is exactly the term which we would _not_ expect by applying the usual chain rule. But this is the term that allows us to get the correct expected value of _W_ 2( _T_ ). Indeed, by straightforward integration of Eq. (3.92) now we find

Taking the expected values yields

which is coherent with our findings in Example 3.17.

Let us summarize our findings: With respect to Eq. (3.87), in Itô's lemma we have an extra term in _dW_ , which is expected given the form of the differential equation defining the stochastic process _X_ ( _t_ ), and an unexpected term

In deterministic calculus, second-order derivatives occur in second-order terms linked to (δ _t_ )2, which can be neglected; but here we have a term of order   which must be taken into account even when it is squared.

## 3.7.4 GEOMETRIC BROWNIAN MOTION

Itô's lemma may be used to find the solution of stochastic differential equations, at least in relatively simple cases. A very important example is geometric Brownian motion. Geometric Brownian motion is defined by a specific choice of the functions in Itô stochastic differential equation:

where μ and σ are constant parameters referred to as drift and volatility, respectively. Intuition would suggest to rewrite the equation as

and then to consider the differential of _d_ log _S_ , which would be _dS/S_ in deterministic calculus, to find the corresponding integral. However, we have seen that some extra care is needed when dealing with stochastic differential equations. Nevertheless, the above intuition deserves further investigation, so let us apply Itô's lemma and find the stochastic differential equation for _F_ ( _S, t_ ) − log _S_ ( _t_ ). As a first step we compute the partial derivatives:

Then, putting all of them together we find

Now we see that our guess above was not that bad, as this equation may be easily integrated and yields

Recalling that _W_ ( _t_ ) has a normal distribution and can be written as _W_ ( _t_ ) =  , where   ~ N(0, 1), we conclude that the logarithm of price is normally distributed:

We can rewrite the solution in terms of _S_ ( _t_ ):

or

(3.93)

On the one hand, this shows that prices, according to the geometric Brownian motion model, are lognormally distributed. On the other hand, now we have a way to generate sample paths, based on a discretization of a prescribed time horizon in time periods of length δ _t_ , like we did with the standard Wiener process. A simple R function to do so is given in Fig. 3.40. This function, when given the drift and volatility selected in the script, yields the plot in Fig. 3.41.

**FIGURE 3.40** **Simple R function to sample geometric Brownian motion.**

**FIGURE 3.41** **A sample path of geometric Brownian motion.**

By using properties of the lognormal distribution and letting ν = μ −σ2/2, we also find

(3.94)

(3.95)

from which we see that the drift parameter μ is linked to the continuously compounded return. The volatility parameter σ is related to standard deviation of the increment of logarithm of price. The roles of drift and volatility can also be grasped intuitively by considering the following approximation of the differential equation:

where δ _S/S_ is the return of the asset over small time interval δ _t_. According to this approximation, we see that return can be approximated by a normal variable with expected value μδ _t_ and standard deviation σ√δ _t._ Actually, this normal distribution is only a local approximation of the "true" (according to the model) lognormal distribution.

## 3.7.5 GENERALIZATIONS

Geometric Brownian motion is not the only type of stochastic process relevant in finance, and the Wiener process is not the only relevant building block. One of the main features of these processes is the continuity of sample paths. However, discontinuities do occur sometimes, such as jumps in prices. In this case, different building blocks are used, such as the Poisson process, which counts the events occurring at a certain rate. We should also note that continuous sample paths do not make sense for certain state variables such as credit ratings. Another point is that the lognormal distribution, that we get from geometric Brownian motion, is a consequence of the normality associated with the Wiener process. Distributions with fatter tails are typically observed, questioning the validity of the simple models based on Gaussian diffusions.

Indeed, several alternative processes have been proposed and are commonly used in financial engineering. Here we just mention some possible generalizations, leaving further details to Chapter 6 on path generation and Chapter 11 on pricing derivatives.

**Correlated Wiener processes.** The need to model the dynamics of a set of asset prices arises when we deal with a portfolio, or when we want to price a rainbow option depending on multiple underlying assets. The simplest model that we may adopt is a direct generalization of geometric Brownian motion. According to this approach, the prices _S_ _i_ ( _t_ ) of assets _i_ = 1,..., _n_ satisfy the equation

where the Wiener processes _W_ _i_ ( _t_ ) are not necessarily independent. They are characterized by a set of instantaneous correlation coefficients ρ _ij_ , whose meaning can be grasped by an extension of the formal rule of Eq. (3.85):

From a Monte Carlo point of view, sample path generation of these processes require the generation of increments

where   and  _j_ are standard normals with correlation ρ _ij_.

**Ornstein–Uhlenbeck processes.** In geometric Brownian motion, we have an exponential trend on which a diffusion process is superimposed. This exponential growth makes no sense for several financial variables, which rather feature mean reversion. Then, we may resort to Ornstein–Uhlenbeck processes like

They still are Gaussian diffusions, but we observe that the drift can change sign, pulling the process back to a long-run average  , with speed γ. The application of this model to a short-term interest rate _r_ ( _t_ ) yields the Vasicek model:

**Square-root diffusions.** One drawback of Ornstein–Uhlenbeck processes is that they do not prevent negative values, which make no sense for stock prices and interest rates. A possible adjustment is the following:

This is an example of a square-root diffusion, which in the context of short-term interest rates is known as the Cox–Ingersoll–Ross model. For a suitable choice of parameters, it can be shown that a square-root diffusion stays non-negative. This changes the nature of the process, which is not Gaussian anymore, as we will see in Chapter 6. Similar considerations hold when modeling a stochastic and time-varying volatility σ( _t_ ). Geometric Brownian motion assumes constant volatility, whereas in practice we may observe time periods in which volatility is higher than usual. In a discrete-time setting, ARCH and GARCH models may be used. In continuous-time, one possible model for stochastic volatility is the Heston model, which consists of a pair of stochastic differential equations:

where _V_ ( _t_ ) = σ _2_ ( _t_ ),   is a long-term value, and different assumptions can be made on the correlation of the two driving Wiener processes.

**Jump–diffusions.** In order to account for jumps, we may devise processes with both a diffusion and a jump component, such as

where _Y_ ( _t_ ) is a compound Poisson process. From a formal point of view, we require the definition of a stochastic integral of a stochastic process _Y_ ( _t_ ) with respect to a Poisson process _N_ ( _t_ ):

where _N_ ( _t_ ) is the number of jumps occurred up to time _t_ in the Poisson process, and _t_ _i_ , _i_ = 1,..., _N_ ( _t_ ), are the time instants at which jumps occur. We note that the Wiener and the Poisson process, which are the basic building blocks of jump–diffusions, are quite different, but they do have an important common feature: stationary and independent increments. In fact, the class of Lévy processes generalizes both of them by emphasizing this feature.

# 3.8 Dimensionality reduction

Monte Carlo methods are often the only feasible approach to cope with high-dimensional problems. Nevertheless, they may require an excessive computational effort to produce a reliable answer. Hence, strategies to reduce the dimensionality of a problem are welcome anyway. Moreover, they can also be put to good use when applying alternative approaches, such as low-discrepancy sequences. Apart from computational convenience, there is another driving force behind dimensionality reduction. While in this chapter we are mainly concerned with model building, in the next one we will deal with model estimation. A rich model may be quite appealing in principle, but it turns into a nightmare if it calls for a huge amount of data for its estimation, and a badly estimated data will make the output of the most sophisticated Monte Carlo simulation utterly useless, if not worse.

Data and dimensionality reduction methods are an important and wide topic in multivariate statistics, and we cannot provide anything but some hints on the most relevant tools for applications in finance and economics, to provide the reader with a feeling for them. In this section we outline two methods:

  Principal component analysis, PCA for short

  Factor models

The second approach should not be confused with factor analysis, which is another multivariate analysis technique for dimensionality reduction. The main difference is that factor analysis techniques deal with latent and nonobservable factors, arguably due to their origin in psychology. The factor models we consider here are quite common in finance and aim at easing some statistical estimation issues when a large number of assets is analyzed.

## 3.8.1 PRINCIPAL COMPONENT ANALYSIS (PCA)

To introduce principal component analysis (PCA) in the most intuitive way, let us have a look at Fig. 3.42, which shows a scatterplot of observations of two random variables _X_ 1 and _X_ 2. It is clear that the two random variables are positively correlated. Now imagine that, for some reason, we can only afford dealing with a single variable. In order to streamline our model of uncertainty, one possibility would be to get rid of one of the two variables, but which one? Since the variables are related and both display some non-negligible variability, discarding one the two variables looks like a very crude approach to dimensionality reduction. We should try to come up with a single variable trying to capture the most information. Now, consider the two axes in the figure, labeled with variables _Z_ 1 and _Z_ 2. Of course, we may represent the very same set of observations in terms of _Z_ 1 and _Z_ 2. This is just a change of coordinates, accomplished in two steps:

**FIGURE 3.42** **Visualization of PCA.**

  Data have been centered, which is obtained by subtracting the sample mean from the observations: **X − **. This is just a shift in the origin of reference axes, but it is often recommended to improve numerical stability in data analysis algorithms.

  The coordinates _Z_ 1 and _Z_ 2 are obtained by a suitable rotation of the axes.

Vector rotation is accomplished by multiplication by an orthogonal matrix. A square matrix **A** is orthogonal if its columns are a set of orthogonal unit vectors,39 which implies

This also shows that this matrix can be inverted by transposition. To see that multiplication by such a matrix is a rotation, we may observe that it does not change the norm of vectors:

Since orthogonal matrices enjoy plenty of nice properties, we might be able to find a linear transformation of variables with some interesting structure, Indeed, in Fig. 3.42 we observe two important facts about the new variables _Z_ 1 and _Z_ 2:

**1.** They are uncorrected.

**2.** The first variable is associated with the largest variance.

Now, if we had to get rid of one variable, we would have no doubt: **Z** 1 conveys the most information, and the residual one is not only less relevant, but also uncorrelated.

Now that we have a feeling for what PCA is and why it may be useful, we may investigate it in more depth following two different routes to introduce PCA:

**1.** As a means of taking linear combinations of variables, in such a way that they are uncorrelated.

**2.** As a way to combine variables so that the first component has maximal variance, the second one has the next maximal variance, and so forth.

Let us pursue both viewpoints, which actually lead us to same conclusions, as we will see.

### 3.8.1.1 A geometric view of PCA

Let _p_ be the dimension of the random vector **X** and _n_ the number of available joint observations. Such observations are collected into the following matrix:

where the element [ _X_ ] _kj_ in row _k_ and column _j_ is the component _j_ of observation _k_ , i.e., _X_ ( _k_ ) _j_.

A linear data transformation, including centering, can be written as

where **A**  . We assume that data have already been centered, in order to ease notation. Hence,

The _Z_ _i_ variables are called _principal components_ : _Z_ 1 is the first principal component. Now, let us consider the sample covariance matrix of **X** , denoted by **S** **x** , with entries

Since we assume centered data, this matrix can be expressed as follows:

Now, we may also find the corresponding sample covariance matrix for _Z_ , **S z**. However, we would like to find a matrix **A** such that the resulting principal components are uncorrelated; in other words, **S z** should be diagonal:40

where _S_ _Z_ _i_ 2. is the sample variance of each principal component. The matrix **A** should diagonalize the sample covariance matrix **S x**. To diagonalize **S x**, we should consider the product

where matrix **P** is orthogonal and its columns consist of the normalized eigenvectors of **S x**; since this is symmetric, its eigenvectors are indeed orthogonal. The diagonalized matrix consists of the eigenvalues λ _i_ , _i_ = 1,..., _p_ , of **S x**. Putting everything together, we see that the rows **a** _i_  , _i_ = 1,..., _p_ , of matrix **A** should be the normalized eigenvectors:

We also observe that the sample variances of the principal components _Z_ _i_ are the eigenvalues of **S x**:

If we sort eigenvalues in decreasing order, λ1 ≥ λ2 ≥... ≥ λ _p_ , we can see that indeed _Z_ 1 is the first principal component, accounting for most variability. Then, the second principal component _Z_ 2 is uncorrected with _Z_ 1 and is the second in rank. The fraction of variance explained by the first _q_ components is

Taking the first few components, we can account for most variability and reduce the problem dimension by replacing the original variables by the principal components.

### 3.8.1.2 Another view of PCA

Another view of PCA is obtained by interpreting the first principal component in terms of orthogonal projection. Consider a unit vector **u**    _p_ , and imagine projecting the observed vector **X** on **u**. This yields a vector parallel to **u** , of length **u  X**. Since **u** has unit length, the projection of observation **X** ( _k_ ) on **u** is

We are projecting _p_ -dimensional observations on just one axis, and, of course, we would like to have an approximation that is as good as possible. More precisely, we should find u in such a way that the distance between the originally observed vector **X** ( _k_ ) and its projection is as small as possible. If we have a sample of _n_ observations, we should minimize the average distance

which looks much like a least-squares problem. This amounts to an orthogonal projection of the original vectors on **u** , where we know that the original and the projected vectors are orthogonal. Hence, we can apply the Pythagorean theorem to rewrite the problem:

In order to minimize the left-hand side of this equality, on the average, we have to maximize

subject to the condition || **u** || = 1. The problem can be restated as

But we know that, assuming that data are centered, the sample covariance matrix is  ; hence, the problem is equivalent to

(3.96)

(3.97)

In plain English, what we want is finding one dimension on which multidimensional data should be projected, in such a way that the variance of the projected data is maximized. This makes sense from a least-squares perspective, but it also has an intuitive appeal: The dimension along which we maximize variance is the one providing the most information.

To solve the problem above, we may associate the constraint (3.97) with a Lagrange multiplier λ and augment the objective function (3.96) to obtain the Lagrangian function:41

The gradient of the Lagrangian function with respect to **u** is

and setting it to zero yields the first-order optimality condition

This amounts to saying that λ must be an eigenvalue of the sample covariance matrix, but which one? We can rewrite the objective function (3.96) as follows:

Hence, we see that λ should be the largest eigenvalue of **S X, u** is the corresponding normalized eigenvector, and we obtain the same result as in the previous section. Furthermore, we should continue on the same route, by asking for another direction in which variance is maximized, subject to the constraint that it is orthogonal to the first direction we found. Since eigenvectors of a symmetric matrix are orthogonal, we see that indeed we will find all of them, in decreasing order of the corresponding eigenvalues.

### 3.8.1.3 A small numerical example

Principal component analysis in practice is carried out on sampled data, but it may be instructive to consider an example where both the probabilistic and the statistical sides are dealt with.42 Consider first a random variable with bivariate normal distribution, **X** ~ N( **0** , Σ), where

and ρ > 0. Thus, _X_ 1 and _X_ 2 are standard normal variables with positive correlation ρ. To find the eigenvalues of Σ, we must find its characteristic polynomial and solve the corresponding equation

This yields the two eigenvalues λ1 = 1 + ρ and λ2 = 1 − ρ. Note that both eigenvalues are non-negative, since ρ   [−1, 1] is a correlation coefficient. To find the first eigenvector, associated with λ1, we consider the system of linear equations

Clearly, the two equations are linearly dependent and any vector such that _u_ 1 = _u_ 2 is an eigenvector. By a similar token, any vector such that _u_ 1 = − _u_ 2 is an eigenvector corresponding to λ2. Two normalized eigenvectors are

These are the rows of the transformation matrix

Since we are dealing with standard normals, μ = **0** and the first principal component is

The second principal component is

As a further check, let us compute the variance of the first principal component:

Figure 3.43 shows the level curves of the joint density of **X** when ρ = 0.85. Since correlation is positive, the main axis of the ellipses has positive slope. It is easy to see that along that direction we have the largest variability.

**FIGURE 3.43** **Level curves of a multivariate normal with ρ = 0.85.**

Practical PCA is carried out on sampled data, and R provides us with a suitable function called prcomp. The R code in Fig. 3.44 samples 2000 points from the above bivariate normal distribution, with ρ = 0.85, carries out PCA, and produces the following output:

**FIGURE 3.44** **Code to carry out PCA on sampled data.**

    > summary(PCA)
    Importance of components:
    PC1 PC2
    Standard deviation 1.3574 0.38496
    Proportion of Variance 0.9256 0.07444
    Cumulative Proportion 0.9256 1.00000
    > print(PCA)
    Standard deviations:
    [1] 1.3573917 0.3849576

    Rotation:
    PC1 PC2
    [1,] −0.7133313 0.7008270
    [2,] −0.7008270 −0.7133313

As usual in R, we may get the essential information about an object by using functions print or summary; here we are applying them on an object, PCA, which is a rich data structure including all of the relevant information about the results of the analysis. Let us check these results against the theory, starting with sample statistics, which are fairly close to the theoretical values (we also compute centered data along the way):

    > m <- colMeans(X);m
    [1] −0.013842466 −0.009333299
    > Xc <- X − (matrix(1,NN,1) %*% m)
    > S <- cov(X);S
    [,1] [,2]
    [1,] 1.0103328 0.8470275
    [2,] 0.8470275 0.9803719

Then, let us compute eigenvalues and eigenvectors of the sample covariance matrix, and check them against the output of PCA:

    > vv <- eigen(S)
    > eigval <- vv$values; eigval
    [1] 1.8425123 0.1481923
    > sqrt(eigval)
    [1] 1.3573917 0.3849576
    > PCA$sdev
    [1] 1.3573917 0.3849576
    > eigvet <- vv$vectors;eigvet
    [,1] [,2]
    [1,] −0.7133313 0.7008270
    [2,] −0.7008270 −0.7133313

Note that if we do not feel comfortable with eigenvalues and eigenvectors, we may also obtain the principal components directly.

    > A <- PCA$rotation;A
    PC1 PC2
    [1,] −0.7133313 0.7008270
    [2,] −0.7008270 −0.7133313

We clearly see that the first principal component accounts for most variability, precisely:

We may also find the transformed variables directly:

    > z <- PCA$x
    > cov(z)
    PC1 PC2
    PC1 1.842512e+00 2.636065e-16
    PC2 2.636065e-16 1.481923e-01
    > head(z)
    PC1 PC2
    [1,] −2.3881699 −0.83075134
    [2,] 0.6764307 −1.07220758
    [3,] 1.5929255 −0.33190052
    [4,] 2.3407987 0.06721413
    [5,] 1.6718914 −0.10795592
    [6,] −0.8683767 0.56002701
    > head(Xc %*% A)
    PC1 PC2
    [1,] −2.3881699 −0.83075134
    [2,] 0.6764307 −1.07220758
    [3,] 1.5929255 −0.33190052
    [4,] 2.3407987 0.06721413
    [5,] 1.6718914 −0.10795592
    [6,] −0.8683767 0.56002701

Finally, let us draw the scatterplots of the original variables and the principal components, as well as the boxplots of the principal components:

    > par(mfrow=c(1,2))
    > plot(X[,1], X[,2])
    > plot(z[,1], z[,2])
    > par(mfrow=c(1,1))
    > boxplot(z[,1], z[,2], names=c("Z1","Z2"))

The first plot, displayed in Fig. 3.45, shows the positive correlation of the original variables and the lack of correlation of the principal components. Note that the two scales here are different, however, and this hides the relative amounts of variability; the difference in variability is pretty evident in the two boxplots of Fig. 3.46.

**FIGURE 3.45** **Scatterplots of original variables and principal components.**

**FIGURE 3.46** **Boxplots of principal components.**

### 3.8.1.4 Applications of PCA

PCA can be applied in a Monte Carlo setting by reducing the number of random variables we have to sample from. A less obvious role is played in the contest of low-discrepancy sequences. As we shall see in Chapter 9, these sequences can replace random sampling in intermediate-sized problems, but may not be as effective in truly high-dimensional settings. Then, one can use low-discrepancy sequences to sample from the principal components, which have the most impact on the output, possibly using random sampling for the remaining ones. We should mention, however, that in order to apply such ideas we have to figure out the relationship between the distribution of the original variables and the principal components, which need not to be an easy task in general.

Another typical application of PCA is in reducing the number of risk factors in financial modeling. Consider, for instance, the _term structure of interest rates_ , which specifies a set of (annualized) interest rates applying on different time horizons. To use the most general notation, let _R_ (0, _t, t_ \+ τ) be continuously compounded forward interest rate applying on the time period [ _t, t_ \+ τ], as seen at time 0. This is a forward rate, as it is the rate that can be contracted now for a time interval starting in the future. If we consider the set of rates _R_ ( _t, t, t_ \+ τ) as a function of τ, we have the term structure of spot rates, i.e., the rates applying at time _t_ for several maturities τ. Usually, the term structure is increasing in τ, but it moves in time, changing its shape in various ways. Clearly, modeling the term structure requires the definition of a model for each maturity; furthermore, these movements must preserve some basic consistency, in order to prevent arbitrage opportunities. Measuring and managing interest rate risk is complicated by the presence of so many interrelated risk factors, and Monte Carlo simulation can be challenging as well. One possible approach to make the problem more manageable is to replace the term structure by a few key risk factors combining rates at different maturities, possibly obtained by PCA.43 In typical models, the first few principal components can be interpreted and related to the level, slope, and curvature of the term structure.

## 3.8.2 FACTOR MODELS

Now let us consider another quite practical question: If we have a large set of jointly distributed random variables, what is the required effort to estimate their covariance structure? Equivalently, how many correlations we need? The covariance matrix is symmetric; hence, if we have _n_ random variables, we have to estimate _n_ variances and _n_ ( _n_ − 1)/2 covariances. This amounts to

entries. Hence, if _n_ = 1000, we should estimate 500,500 entries in the covariance matrix. A daunting task, indeed! If you think that such a case will never occur in practice, please consider a practical portfolio management problem. You might well consider 1000 assets for inclusion in the portfolio. In such a case, can we estimate the covariance matrix? What you know about statistical inference tells that you might need a lot of data to come up with a reliable estimate of a parameter. If you have to estimate a huge number of parameters, you need a huge collection of historical data. Unfortunately, many of them would actually tell us nothing useful: Would you use data from the 1940s to characterize the distribution of returns for IBM stock shares now? We need a completely different approach to reduce our estimation requirements.

The returns of a stock share are influenced by many factors. Some are general economic factors, such as inflation and economic growth. Others are specific factors of a single firm, depending on its management strategy, product portfolio, etc. In between, we may have some factors that are quite relevant for a group of firms within a specific industrial sectors, much less for others; think of the impact of oil prices on energy or telecommunications.

Rather than modeling uncertain returns individually, we might try to take advantage of this structure of general and specific factors. Let us take the idea to an extreme and build a simple model whereby there is one systematic factor common to all of stock shares, and a specific factor for each single firm. Formally, we represent the random return for the stock share of firm _i_ as

where

  α _i_ and β _i_ are parameters to be estimated.

  _R_ _m_ is a random variable representing the systematic risk factor; the subscript _m_ stands for _market_ ; indeed, financial theory suggests that a suitable systematic factor could be the return of a market portfolio consisting of all stock shares, with a proportion depending on their relative capitalization with respect to the whole market.

   _i_ is a random variable representing specific risk, which in financial parlance is also referred to as _idiosyncratic_ risk; a natural assumption about these variables is that E[  _i_ ] = 0 (otherwise, we would include the expected value into α _i_ ). Another requirement is that the systematic factor really captures whatever the stock returns have in common, and that the specific factors are independent. Typically, we do not require independence, but only lack of correlation. These requirements can be formalized as

(3.98)

(3.99)

In the next chapter, where we deal with linear regression, we will see that condition (3.98) is actually ensured by model estimation procedures based on least squares. On the contrary, condition (3.99) is just an assumption, resulting in a so-called _diagonal model_ , since the covariance matrix of specific factors is diagonal.

The model above is called single-factor model for obvious reasons, but what are its advantages from a statistical perspective? Let us check how many unknown parameters we should estimate in order to evaluate expected return and variance of return for an arbitrary portfolio. To begin with, observe that for a portfolio with weights _w_ _i_ we have

Then,

where μ _m_ is the expected return of the market portfolio (more generally, the expected value of whatever systematic risk factor we choose). From this, we see that we need to estimate:

  _n_ parameters α _i_ , _i_ = 1,..., _n_

  _n_ parameters β _i_ , _i_ = 1,..., _n_

  The expected value μ _m_

These add up to 2 _n_ \+ 1 parameters. Variance is a bit trickier, but we may use the diagonality condition (3.99) to eliminate covariances and obtain

where σ _m_ 2 is the variance of the systematic risk factor and σ _i_ 2 is the variance of each idiosyncratic risk factor, _i_ = 1,..., _n_. They amount to _n_ \+ 1 additional parameters that we should estimate, bringing the total to 3 _n_ \+ 2 parameters. In the case of _n_ = 1000 assets, we have a grand total of 3,002 parameters; this is a large number, anyway, but it pales when compared with the 500,500 entries of the full covariance matrix.

In passing, we note that factor models also play a key role in financial economics. The _capital asset pricing model_ (CAPM) is essentially a single-factor model, based on a systematic risk factor related to the return of the market portfolio; if we add some suitable hypotheses to the model, we are led to an important, though controversial, equilibrium model. If more factors are included in the model, plus some hypotheses on the structure of the market and the behavior of investors, we obtain another equilibrium model, _arbitrage pricing theory_ (APT). The validity of these models is challenged by alternative views, as they are not just statistical but equilibrium models, relying on questionable assumptions. Nevertheless, the idea of factor models is widely used by practitioners who do not subscribe to CAPM or APT.

From a Monte Carlo point of view, factor models may simplify scenario generation. There might be some computational saving, but this is arguably not the really important point. What is more relevant is that we can provide the simulation model with more reliable parameter estimates and inputs, with a positive impact on output quality.

# 3.9 Risk-neutral derivative pricing

In this section we outline risk-neutral pricing for financial derivatives, most prominently options, for the sake of the unfamiliar reader. What we cover here is mostly relevant to Chapter 11. If a reader is not directly interested to this topic, but she wants nevertheless to see how Monte Carlo methods are used in this setting as an example, here are the essential results:

  Options (and other derivatives) can be priced by taking the expected value of the discounted payoff.

  The expectation must be taken under a risk-neutral (or risk-adjusted) probability measure.

This justifies the application of Monte Carlo methods to estimate the expected payoff and the option price as a consequence; however, we have to use a modified model for the dynamics of the underlying asset. Typically, this just means changing the drift in a stochastic differential equation.

Options are financial derivatives, i.e., assets whose value depends on some other variable. This can be illustrated by considering a call option, a contract between two counterparts, written on a non-dividend-paying stock: The option holder, i.e., the party buying the option, has the right, but not the obligation, to buy the underlying asset from the option writer, i.e., the party selling the option on the primary market, at a prescribed future time and at a price established now. This is called the _strike price_ , denoted by _K_. Let us denote the current price of the underlying asset by _S_ 0 (known when the option is written) and the future price at a time _t_ = _T_ by _S_ _T_ ; the latter is, at time _t_ = 0, a random variable. The time _t_ = _T_ corresponds to the option maturity. A _European-style_ option can only be exercised at maturity. On the contrary, _American-style options_ can be exercised at any time before their expiration date. The call option will be exercised only if _S_ _T_ ≥ _K_ , since there would be no point in using the option to buy at a price _K_ that is larger than the prevailing (spot) market price _S_ _T_ at time _T_. Then, the payoff of the option is, from the holder's viewpoint,

To interpret this, consider the possibility of exercising the option and buying at price _K_ an asset that can be immediately sold at _S_ _T_. Market structure is not really that simple, but some options are settled in cash, so the payoff really has a monetary nature; this occurs for options written on a nontraded market index or on interest rates. This payoff is a piecewise linear function and is illustrated in Fig. 3.47. Put options, unlike calls, grant the holder the right to sell the underlying asset at the strike price. Hence, the payoff of a put option is

**FIGURE 3.47** **Payoff of a call option.**

Put options have a positive payoff when the asset price drops, much like short selling strategies, whereby one borrows and sells an asset, hoping to buy the asset back at a lower price in order to give it back to the lender.

When the future asset price _S_ _T_ will be realized, at time _t_ = _T_ , the payoff will be clearly determined. What is not that clear is the fair price for the contract at time _t_ = 0. It is easy to see that a put or call option must have a positive price, as their payoff to the option holder cannot be negative; the option writer, on the contrary, can suffer a huge loss; hence, the latter requires an option premium, paid at time _t_ = 0, to enter into this risky contract.

## 3.9.1 OPTION PRICING IN THE BINOMIAL MODEL

The easiest way to get the essential message about option pricing is by referring to the binomial model of Section 3.2.1, based on multiplicative shocks and a generalization of the Bernoulli random variable. As shown in Fig. 3.48, the future price of the underlying asset can be either _uS_ 0 or _dS_ 0, with probabilities _p_ _u_ and _p_ _d_ , respectively. Conditional on the asset price at maturity, the option will provide its holder with a payoff _f_ _T_ , which can either be _f_ _u_ or _f_ _d_. These values are also illustrated in Fig. 3.48. For instance, in the case of a call option with strike price _K_ , we have

**FIGURE 3.48** **One-step binomial model for option pricing.**

The pricing problem consists of finding a fair price _f_ 0 for the option, i.e., its value at time _t_ = 0. Intuitively, one would guess that the two probabilities _p_ _u_ and _p_ _d_ play a major role in determining _f_ 0. What is not quite clear is the role that should be played by the overall risk aversion of market participants. However, a simple principle can be exploited in order to simplify our task. The key idea is to build a replicating portfolio, i.e., a portfolio consisting of elementary assets, such that its payoff replicates the option payoff. If we can do so and we know the current prices of the assets in the replicating portfolio, then it is easy to argue that the option value cannot be different from the value of the replicating portfolio. If this were not the case, the intuitive law of one price would be violated. We would have two equivalent portfolios with different prices, and we would just buy the cheaper one and sell the more expensive one to lock in a risk-free profit starting with no money. Such a money making machine is called an arbitrage opportunity, and option pricing relies on the assumption that arbitrage opportunities are ruled out. To be precise, what we need to assume is that they cannot exist in equilibrium, as investors would take immediate advantage of them, in such a way to move prices back to an equilibrium.

The replicating portfolio should certainly include the underlying asset, but we also need an additional one. Financial markets provide us with such an asset, in the form of a risk-free investment. We may consider a bank account yielding a risk-free interest rate _r_ ; we assume here that this rate is continuously compounded, so that if we invest $1 now, we will own $ _e_ _rt_ at a generic time instant _t_. Equivalently, we may think of a riskless zero-coupon bond, with initial price _B_ 0 = 1 and future price _B_ 1 = _e_ _rT_ at time _t_ = _T_ , when the option matures. In order to find the number of stock shares and riskless bonds that we should include in the replicating portfolio, we just need to set up a system of two linear equations. Let us denote the number of stock shares by Δ and the number of bonds by  . The initial value of this portfolio is

(3.100)

and its future value, depending on the realized state, will be either

Note that future value of the riskless bond does not depend on the realized state. If this portfolio has to replicate the option payoff, we should enforce the following two conditions:

(3.101)

Solving the system yields

Hence, by the law of one price, the option value now must be

(3.102)

A striking fact is that this relationship does _not_ involve the objective probabilities _p_ _u_ and _p_ _d_. However, let us consider the quantities

which happen to multiply the option payoffs in Eq. (3.102). We may notice two interesting features of π _u_ and π _d_ :

**1.** They add up to one, i.e., π _u_ \+ π _d_ = 1.

**2.** They are non-negative, i.e., π _u_ , π _d_ ≥ 0, provided that _d_ ≤ _e_ _rT_ ≤ _u_.

The last condition does make economic sense. If we had _e_ _rT_ < _d_ < _u_ , the risky stock share would perform better than the risk-free bond in any scenario, and we would just borrow money at the risk-free rate and buy a huge amount of the stock share. On the contrary, if the risk-free rate is always larger than the stock price in the future, one could make easy money by selling the stock share short and investing the proceeds in the risk-free asset. In fact, we can interpret π _u_ and π _d_ as probabilities, and Eq. (3.102) can be interpreted in turn as the discounted expected value of the option payoff,

(3.103)

provided that we do not use the "objective" probabilities _p_ _u_ and _p_ _d_ , but π _u_ and π _d_. Here the notation E [·] emphasizes this change in the probability measure. These are called _risk-neutral probabilities_ , a weird name that we can easily justify. In fact, if we use the probabilities π _u_ and π _d_ , the expected value of the underlying asset at maturity is

But how is it possible that the expected return of a risky asset is just the risk-free rate? This might happen only in a world where investors do not care about risk, but only about the expected return; in other words, they do not ask for any risk premium. In such a risk-neutral world, the expected return of all assets, at equilibrium, should be the same, and if there is a risk-free asset, its return will be the expected return for all risky assets as well. This observation justifies the name associated with the probabilities that we use in option pricing. Indeed, what we have just illustrated is the tip of a powerful iceberg, the risk-neutral pricing approach, which plays a pivotal role in financial engineering.

## 3.9.2 A CONTINUOUS-TIME MODEL FOR OPTION PRICING: THE BLACK–SCHOLES–MERTON FORMULA

The binomial model yields an intuitive and enlightening result. However, we cannot certainly represent uncertainty in stock prices by a Bernoulli random variable. If we introduce more states of nature, we have two ways to build a replicating portfolio: Either we introduce more assets, or we introduce a dynamic replicating strategy, i.e., we rebalance the portfolio several times along the option life. The second approach sounds much more promising, as it relies on the same elementary assets as before. In the limit, if we assume rebalancing in continuous time, we can rely on the machinery of stochastic calculus. Let us assume that the underlying asset price follows a geometric Brownian motion,

where to streamline notation a bit we use _S_ _t_ and _W_ _t_ rather than _S_ ( _t_ ) and _W_ ( _t_ ), and let us consider the viewpoint of the option writer, who holds a short position in the call option. Let us denote the fair option price at time _t_ , when the underlying asset price is _S_ _t_ , by _f_ ( _S_ _t_ , _t_ ). Note that we are taking for granted that the option price depends on these two variables only, which makes sense for a path-independent, vanilla option. In the case of Asian options, where the payoff depends on average prices, things are not that simple. Using Itô's lemma, we may write a stochastic differential equation for _f_ :

(3.104)

Just as in the binomial case, what we know is the option value at maturity,

and what we would like to know is _f_ ( _S_ 0, 0), the fair option price now. The option holder has a position of value − _f_ ( _S_ _t_ , _t_ ) and is subject to considerable risk: If the stock price rockets up, the option holder will exercise, and the writer will have to buy the underlying asset at a high price, unless some countermeasures are taken. One possibility is to buy some stock shares, say Δ, and build a hedged portfolio of value

Equation (3.104) does not suggest an immediate way to find the option price, but it would look a little bit nicer without the random term _dS_. Indeed, it is easy to see that, by choosing

we can hedge risk by eliminating the random term _dS_ , which includes the increment of the Wiener process. By differentiating Π, we find

(3.105)

With the above choice of Δ, our portfolio is riskless; hence, by no-arbitrage arguments, it must earn the risk-free interest rate _r_ :

(3.106)

Eliminating _d_ Π between Eqs. (3.105) and (3.106), we obtain

and finally

(3.107)

Now we have a deterministic partial differential equation (PDE for short) describing an option value _f_ ( _S, t_ ). This is the Black–Scholes–Merton PDE. Such a PDE must be solved with suitable boundary conditions, but before doing so, the following observations are in order about the choice of Δ:

  It is consistent with the value it takes in the binomial case, which is just an increment ratio and becomes a derivative in the continuous case.

  It eliminates the dependence with respect to the true drift μ, which is related to the expected return of the underlying asset; again, this is consistent with the binomial model, where the expected value of the future asset price, based on the objective probabilities, does not plat any role.

What does not quite look consistent with the binomial model is that there we obtain the price as an expected value, whereas here we have a PDE. Actually, they are two sides of the same coin, and the gap can be bridged by one version of the Feynman–Kač formula.

**THEOREM 3.17 Feynman–Kač representation theorem.** _Consider the partial differential equation_

_and let F_ = _F_ ( _x, t_ ) _be a solution satisfying the boundary condition_

_Then, under technical conditions, F_ ( _x, t_ ) _can be represented as_

_where X_ _t_ _is a stochastic process satisfying the differential equation_

_with initial condition X_ _t_ = _x_.

The notation E _x,t_ [·] is used to point out that this is a _conditional_ expectation, given that at time _t_ the value of the stochastic process is _X_ _t_ = _x_. From a mathematical viewpoint, the theorem is a consequence of how the stochastic integral in the sense of Itô is defined (see [1] for a clear proof). From a physical point of view, it is a consequence of the connection between Brownian motion (which is a diffusion process) and a certain type of PDEs which can be transformed into the heat equation.

The application of the representation theorem to the Black–Scholes–Merton equation, for an option with payoff function  , immediately yields

which is consistent with Eq. (3.103). We point out once more that the expectation is taken under a risk-neutral measure, which essentially means that we work as if the stochastic differential equation for _S_ _t_ were

By recalling the result of Eq. (3.93), we see that under the risk-neutral measure the price _S_ _t_ is lognormal and may be written as

where   ~ N(0, 1). Then, it is a fairly easy, eve though a bit tedious, exercise in integration to prove the celebrated Black–Scholes–Merton formula (BSM formula for short), which gives the price _C_ of a vanilla European-style call option:

(3.108)

where

and Φ( _x_ ) is the cumulative distribution function for the standard normal distribution:

It is easy to use R to implement the above formula and plot the call price for various underlying asset prices and times to maturity. The code in Fig. 3.49 produces the plot in Fig. 3.50. Note that the implementation is a bit naive, in the sense that it does not really work with some input arguments. For instance, when time to maturity is zero and the asset price is the same as the strike price, the function returns NaN. This is the reason why in the last line of the script we plot the option value at maturity by computing the payoff. The reader is invited to improve the code, including some consistency check on the input (for instance, volatility cannot be negative).

**FIGURE 3.49** **Naive implementation of the Black–Scholes–Merton formula for a vanilla call.**

**FIGURE 3.50** **Call option prices.**

The same reasoning can be applied to price a put option, but there is a very useful theorem that solves the problem.

**THEOREM 3.18 (Put–call parity)** _Consider a call and a put options written on the same non-dividend-paying stock, with the same strike price K and time to maturity T. Then their prices C_ 0 _and P_ 0 _at time t_ = 0 _are related as follows:_

_where S_ 0 _is the current stock price and r is the continuously compounded risk-free interest rate_.

**PROOF** Consider a portfolio consisting of a call option and an amount of cash given by _Ke_ − _rT_. At maturity, the value of this portfolio is

(3.109)

By a similar token, at maturity the value of a portfolio consisting of a put option and one share of the underlying stock is

Since the two portfolios have the same value in any future scenario, they cannot have a different value now, which implies the statement of the theorem.

This proof is instructive and nontechnical, which is why we have included it: It does not rely on any model and is just based on a no-arbitrage consideration, in the form of the law of one price. When we plug the BSM formula for the call option price into Eq. (3.109), we find a similar formula for the put option price:

(3.110)

where _d_ 1 and _d_ 2 are the same expressions occurring in the call option price. A fundamental advantage of the BSM formula, apart from its computational appeal, is that we may evaluate option sensitivities as well. For instance, the number of shares Δ in the hedging portfolio for a call option is

(3.111)

This quantity is commonly referred to as "delta," and it may be interpreted as the sensitivity of the option price to changes in the underlying asset price. By differentiating the put–call parity relationship with respect to _S_ 0 we immediately find the corresponding delta for the put option:

(3.112)

Other sensitivities may be obtained, collectively nicknamed as the _greeks_ :

(3.113)

(3.114)

where ϕ(·) is the PDF of a standard normal, not to be confused with the CDF Φ(·). Equation (3.113) defines and evaluates the option _gamma_ , i.e., the second-order derivative of the option price with respect to the underlying asset price. Equation (3.114) defines and evaluates the option _vega_ , i.e., the first-order derivative of the option price with respect to volatility. From put–call parity we observe that these two greeks are the same for call and put options.

Another relevant greek is _theta_ , which is related to the passage of time:

(3.115)

(3.116)

We see that different formulas are required for call and put options, but there is a subtle point here: We have expressed call and option prices at time _t_ = 0, as a function of time to maturity _T_. A more general statement would be recast as a function of time to maturity _T − t_. This is easily accomplished by replacing _T_ with _T − t_ in Eqs. (3.108) and (3.110). It is important to realize that the derivatives defining the option thetas in Eqs. (3.115) and (3.116) are with respect to _t_ , which is then set to _t_ = 0 to find the current theta. Typically, option thetas are negative, which shows that option values are subject to a decay over time.

The option sensitivities, among other things, may be used to evaluate the risk involved in holding a portfolio of options, as we will see in Section 13.3. All of the above formulas are implemented in the R code of Fig. 3.51.

**FIGURE 3.51** **R code to evaluate the put option price and the essential greeks within the BSM framework.**

## 3.9.3 OPTION PRICING IN INCOMPLETE MARKETS

The BSM formula relies on the fact that we can replicate an option. A market in which any option (more generally, any contingent claim) can be replicated, is called a _complete_ market. But the obvious question arises: If the BSM formula were correct, why should one trade options? Indeed, in a complete market, one can replicate any derivative, which turns out to be a redundant and useless asset. A fundamental theorem states that if there are no arbitrage opportunities in the market, then there exists a risk-neutral measure that can be used for pricing. But it is market completeness that ensures its uniqueness. However, markets in real life are incomplete, and even if we rule out arbitrage opportunities, we fail to find a _unique_ risk-neutral measure for pricing purposes.

A market can be incomplete for many reasons:

  Transaction costs make continuous-time hedging not practical.

  Some risk factors, such as jumps and stochastic volatilities, cannot be hedged.

  The underlying factor of an option is not a tradeable asset.

The last case is quite relevant for interest rate derivatives. We can model uncertainty in interest rates using Itô stochastic differential equations, but we cannot use the interest rate itself in a replicating portfolio. Hence, we cannot build a replicating portfolio using the underlying risk factor. Nevertheless, we still can get rid of risk. In fact, the idea here is not to replicate a risky derivative, but to build a riskless portfolio using two assets depending on the same risk factor.

Assume that we consider the short rate _r_ _t_ as the underlying risk factor, and let us denote by _Z_ 1( _r_ _t_ , _t_ ) and _Z_ 2( _r_ _t_ , _t_ ) be the prices of two traded assets, whose value depends on the short rate _r_ _t_ at time _t_. These assets can be as simple as two zero-coupon bonds, which explains the notation, or more complex interest rate derivatives. Let us assume the following general model for the interest rate:44

(3.117)

where the drift rate may feature mean reversion. Indeed, the model includes the Vasicek and the Cox–Ingersoll–Ross models of Section 3.7.5. Using Itô's lemma as usual, we may find the stochastic differential equation for the derivative value _Z_ ( _r_ _t_ , _t_ ):

(3.118)

We may also get rid of risk by setting up a portfolio involving both _Z_ 1 and _Z_ 2 in the right proportions. Let us build a portfolio as follows:

for a suitable choice of Δ. We note that the above choice implies:

(3.119)

(3.120)

From the first condition, we see that we obtain ∂Π/∂ _r_ = 0 if we choose

Actually, we are using the same trick as in the BSM case, but now it involves two risky assets depending on the same risk factor. That choice of Δ eliminates the impact of both the underlying risk and the drift of the short rate:

(3.121)

Since this portfolio is risk-free, we can also write

(3.122)

By linking Eqs. (3.121) and (3.122), taking advantage of Eqs. (3.119) and (3.120), and rearranging, we find

(3.123)

Substituting Δ and rearranging again yields the fundamental pricing equation

(3.124)

We note that the drift term _m_ ( _r_ _t_ , _t_ ) does not play any role, and this is consistent with the BSM case. However, here we are relating two derivative prices. We can observe that the two ratios above refer to generic derivatives depending on the risk factor _r_ _t_ , which implies that for any such derivative they are equal to some function of time and the factor itself. Let us define a function _m_ * ( _r_ _t_ , _t_ ) as _minus_ the ratios in Eq. (3.124); we change the sign just for the sake convenience, as this allows to write the following pricing PDE:

(3.125)

We notice that this is quite close to the BSM partial differential equation, but it involves an unknown function _m_ * ( _r_ _t_ , _t_ ). Using the Feynman–Kač theorem, we can think of the derivative price as the expected value of the payoff, if the risk factor follows the risk-adjusted dynamics

(3.126)

Once again, we notice that there is a change in drift, which reflects a change in probability measure. However, due to market incompleteness, we do not know a priori which measure we should use, as uniqueness of the risk-neutral measure requires completeness. The difficulty is that the drift is related to a market price of risk, which does play a role here since we cannot hedge using the underlying asset directly. The way out of the dilemma requires calibrating the model using some traded derivatives. In the case of interest rates, we might use the quoted prices of bonds or other liquid assets to formulate a model calibration problem and solve the corresponding optimization model to estimate the function _m_ *( _r_ _t_ , _t_ ). Having pinned down the right risk-adjusted measure, we may price other derivatives by using analytical formulas, when available, or by using numerical methods to solve the PDE, like finite differences, or by estimating the price in expectation form by Monte Carlo methods, thanks to the Feynman–Kač representation theorem.

From our viewpoint, we will assume that the model has already been calibrated and that we may proceed to price the derivative asset by estimating the risk-neutral expectation directly. We will see some examples related to interest rates in Section 6.3 and in Chapter 11.

# For further reading

  Many of the topics that we have considered in this chapter are reviewed, e.g., in [11].

  There are many excellent textbooks in probability theory; [8] is an example of the top-level ones in terms of quality and readability.

  For some additional information on data dependence and copulas, see [13] or [18].

  A good reference for introductory econometrics in finance, including VAR, ARCH, and GARCH models, is [3]. A deeper treatment is offered in [7].

  As to time series models, you may refer to [2] or the extensive treatment in [9]. A practical introduction focusing on financial time series analysis with R is [22].

  Readers interested in stochastic calculus may refer to [17] for a gentle introduction or to [21] for a deeper treatment.

  Several methods in multivariate statistics, including PCA, are covered in [10] and [20].

  The role of factor models in finance is illustrated, e.g., in [14] and [19].

# References

1 T. Björk. _Arbitrage Theory in Continuous Time_ (2nd ed.). Oxford University Press, Oxford, 2004.

2 P.J. Brockwell and R.A. Davis. _Time Series: Theory and Methods_ (2nd ed.). Springer, New York, 1991.

3 C. Brooks. _Introductory Econometrics for Finance_ (2nd ed.). Cambridge University Press, Cambridge, 2008.

4 J.Y. Campbell and L.M. Viceira. _Strategic Asset Allocation._ Oxford University Press, Oxford, 2002.

5 P. Embrechts, C. Kluppelberg, and T. Mikosch. _Modelling Extremal Events for Insurance and Finance._ Springer, Berlin, 2000.

6 P. Embrechts, F. Lindskog, and A. McNeil. Modelling dependence with copulas and applications to risk management. In S.T. Rachev, editor, _Handbook of Heavy Tailed Distributions in Finance_ , pp. 329–384. Elsevier, Amsterdam, 2003.

7 W.H. Greene. _Econometric Analysis_ (7th ed.). Prentice Hall, Upper Saddle River, NJ, 2011.

8 G. Grimmett and D. Stirzaker. _Probability and Random Processes_ (3rd ed.). Oxford University Press, Oxford, 2003.

9 J.D. Hamilton. _Time Series Analysis._ Princeton University Press, Princeton, NJ, 1994.

10 W. Härdle and L. Simar. _Applied Multivariate Statistical Analysis_ (2nd ed.). Springer, Berlin, 2007.

11 S. Hubbert. _Essential Mathematics for Market Risk Management._ Wiley, Hoboken, NJ, 2012.

12 J.C. Hull. _Options, Futures, and Other Derivatives_ (8th ed.). Prentice Hall, Upper Saddle River, NJ, 2011.

13 H. Joe. _Multivariable Models and Dependence Concepts._ Chapman & Hall/CRC, Boca Raton, FL, 2001.

14 J. Knight and S. Satchell, editors. _Linear Factor Models in Finance._ Elsevier, Amsterdam, 2005.

15 D.P. Kroese, T. Taimre, and Z.I. Botev. _Handbook of Monte Carlo Methods._ Wiley, Hoboken, NJ, 2011.

16 L. Martellini, P. Priaulet, and S. Priaulet. _Fixed-Income Securities: Valuation, Risk Management, and Portfolio Strategies._ Wiley, Chichester, 2003.

17 T. Mikosch. _Elementary Stochastic Calculus with Finance in View._ World Scientific Publishing, Singapore, 1998.

18 R.B. Nelsen. _An Introduction to Copulas._ Springer, New York, 1999.

19 E.E. Qian, R.H. Hua, and E.H. Sorensen, editors. _Quantitative Equity Portfolio Management: Modern Techniques and Applications._ CRC Press, London, 2007.

20 A.C. Rencher. _Methods of Multivariate Analysis_ (2nd ed.). Wiley, New York, 2002.

21 S. Shreve. _Stochastic Calculus for Finance (vols. I & II)._ Springer, New York, 2003.

22 R.S. Tsay. _An Introduction to the Analysis of Financial Data with R._ Wiley, Hoboken, NJ, 2013.

23 P. Veronesi. _Fixed Income Securities: Valuation, Risk, and Risk Management._ Wiley, Hoboken, NJ, 2010.

 See Section 4.4.

See Section 8.3.

What we use here as return is referred to as _rate_ of return in some treatments, but we sometimes prefer to streamline terminology a bit.

For the unfamiliar reader, we formally introduce covariance later in Section 3.3.

To put it bluntly, relying on past returns is akin to driving by just looking at the rear-view mirror.

This is a streamlined version of the model described in [4, Chapter 7]. See also Chapter 10 to see how such problems may be tackled by stochastic dynamic programming.

 Throughout this book, we use log rather than ln, but we always mean _natural_ logarithms.

 Actually, the precise definition of Gaussian processes does not only involve normal marginals but also jointly normal distributions.

 Needless to say, Monte Carlo methods play a key role in more complicated cases; see Chapter 11.

 We have outlined how a simple binomial model may be calibrated by moment matching in Section 2.5.1.

 We recall the definition of the factorial of an integer number, _n_! ≡ _n_ · ( _n_ − 1) · ( _n_ − 2) ··· 2·1; by convention, 0! = 1.

 Such a mechanism is found, e.g., in rejection sampling, as we shall see in Section 5.4.

 One could wonder why we typically use _X_ ~ N(μ, σ2), referring to variance, and not _X_ ~ N(μ, σ), referring to standard deviation. The reason is that this notation easily generalizes to the multivariate case **X** ~ N(μ, Σ), where Σ is the covariance matrix; see Section 3.3.

 See Section 8.6.2.

 We do not deal with stable distributions in this introductory treatment, since they require more advanced concepts; see, e.g., [15, pp. 129–131].

See Section 6.3.2, where we deal with sample path generation for continuous-time stochastic processes.

See Section 4.6 and Chapter 14.

Actually, they are sampled from a joint normal distribution with μ1 = μ2 = 10, σ1 = σ2 = 5. The multivariate normal distribution is introduced in Section 3.3.4.

You are advised _not_ to try estimating Kendall's correlation with R on a very large data set, as this may require a lot of computation and is likely to block your computer.

See, e.g., [7] for a proof.

See, e.g., [6] or [18].

These counterexamples are borrowed from [18, p. 6].

See the inverse transform method in Chapter 5.

See, e.g., [6].

See [5] for details.

The denominator in Eq. (3.51) is related to the estimate of autocovariance. In the literature, sample autocovariance is typically obtained by dividing the sum by _T_ , even though this results in a biased estimator. If we divide by _T_ − _k_ , we might obtain an autocovariance matrix which is not positive semidefinite; see Brockwell and Davis [2], pp. 220–221.

 Actually, expression is much more powerful; please refer to the online help.

 To be precise, this representation requires that the two polynomials ϕ( _B_ ) and Θ( _B_ ) satisfy some additional technical conditions that are beyond the scope of this book.

 The acronym VAR is a bit unfortunate, as it may be confused with variance or value-at-risk. To avoid confusion, variance is typically written as Var and value-at-risk as VaR; some authors use V@R for the latter, and we will follow that practice in this book.

 The square-root rule, which we consider in Section 3.7.1, suggests that on very short time spans volatility dominates drift.

 The parameters used in the simulation are calibrated on real data in [22, p. 202].

 Sample path generation for stochastic differential equations is the subject of Chapter 6.

 This has some implications on the evaluation of risk measures like value-at-risk; see Chapter 13.

We have already made a quite similar point in Example 3.3.

See Example 8.9.

We have already met this function in the first chapter; please refer to Fig. 1.5 for programming notes. Later, we will describe a more efficient version.

See Section 3.2.3.

One might well wonder whether an increasing exponential trend makes sense for the prices of stock shares, as we do not observe this in real life. One reason is that dividends are paid, bringing down the share price.

39We should maybe talk about orthonormal vectors and matrices, but we will follow the more common parlance.

40See Property 3.6 in Section 3.3.4.1.

41 Strictly speaking, we are in trouble here, since we are maximizing a _convex_ quadratic form. However, we may replace the equality constraint by an inequality constraint, which results in a concave problem, i.e., a problem in which the optimal solution is on the boundary of the feasible solution. It turns out that the solution we pinpoint using the Lagrange multiplier method is the right one.

42 This example has been adapted from [10, Chapter 9].

43 See, e.g., Chapters 3 and 6 of [16].

44The treatment below is related to [23].

# Chapter Four

# Estimation and Fitting

Before embarking in a computationally extensive Monte Carlo study, we should be reasonably confident that we are feeding our simulation tool with sensible input. In the previous chapter we have considered different families of models that can be used as the key ingredient to build a Monte Carlo simulator. Here we consider the quantitative side of the coin: After selecting a model _structure_ , how should we estimate its _parameters_? Hence, we step into the domain of inferential statistics, whereby, given a sample of observed data, we engage in increasingly difficult tasks:

  Finding point and interval estimates of basic moments like expected value and variance

  Estimating the parameters of a possibly complicated probability distribution

  Estimating the parameters of a time series model

The first task is typically associated with elementary concepts like confidence intervals, which may be somewhat dangerous and misleading for the newcomer. Computing a confidence interval for the mean of a normal population looks so easy that it is tempting to disregard the pitfalls involved. We will have more to say on this in Chapter 7, where we deal with output analysis. In fact, simulation input and output analysis do share some statistical background.1 If the output of a Monte Carlo run is used to estimate a probability or a quantile, it may be tempting to apply the same recipes that we adopt to estimate means, which may lead to unreasonable results. Moreover, a probability distribution is not really characterized by expected value and variance. This is the case with a normal distribution N(μ, σ2), as its parameters have that obvious interpretation, but it is not, e.g., with a gamma distribution. Hence, we need more refined strategies, like moment matching or maximum-likelihood estimation. Maximum likelihood is a powerful principle, which can also be used to estimate the parameters of complicated models, including time series.

We must also be aware that parameter estimation should not be our only concern. Apart from fitting parameters, we should also try to check the fit of the overall uncertainty model, which involves nonparametric issues. If we assume that a variable of interest is normally distributed, we should check if this is a defensible, even though debatable, assumption or sheer nonsense. Testing for normality is an example of a nonparametric test. More generally, we may need to compare the fit of alternative distributions, measuring the overall goodness of fit. What is relevant or not depends on the kind of application. For instance, it is possible to build two very different probability distributions having the same expected value, variance, skewness, and kurtosis. Thus, the first few moments do not tell the whole story. This may be not quite relevant if we are just dealing with mean–variance portfolio optimization, but it may be extremely relevant in dealing with extreme risk.

Most books on Monte Carlo methods for finance and economics do not include a chapter on these topics, which is fine if one assumes that readers have a suitable background in econometrics. Other books, such as [10], are aimed at industrial engineers and include an extensive chapter on input analysis. Since this is an introductory book also aimed at engineers and mathematicians, I have deliberately chosen to include such a chapter as well.2 To make the following treatment as useful as possible to a wide and diversified audience, we will start with really elementary topics, and then we will move on to possibly less standard ones, such as estimation of ARMA and GARCH models, as well as Bayesian parameter estimation. Furthermore, we will use R throughout, so that readers familiar with the underlying concepts, but unfamiliar with R functionalities, can take advantage of the examples. Since it is my firm conviction that understanding the underlying unity of concepts is fundamental for sound and deep learning, it is also important to stress the links of parameter estimation and model fitting with other topics dealt with in this book:

  Some nonstandard optimization methods, which we deal with in Chapter 10, may be needed to estimate parameters or calibrate models, when the resulting optimization model is nonconvex and badly behaved.

  The Kolmogorov–Smirnov test is related to discrepancy concepts, which are at the basis of the low-discrepancy sequences that are discussed in Chapter 9.

  Bayesian estimation is related to Markov chain Monte Carlo methods, illustrated in Chapter 14.

  Linear regression is used in some computational methods for dynamic stochastic optimization, which is in turn fundamental to price American-style options with high dimensionality, as we show in Chapter 11.

  A sound understanding of how confidence intervals arise is important to avoid subtle mistakes in output analysis, as we will stress in Chapter 7.

We start in Section 4.1 with a brief refresher on elementary inferential statistics, i.e., confidence intervals and hypothesis testing for the expected value; there, we also outline correlation testing. Then we move on to systematic strategies for parameter estimation in Section 4.2, where we consider moment matching and maximum likelihood. We outline nonparametric tests and checking goodness-of-fit in Section 4.3. We offer a brief refresher on ordinary least-squares methods to estimate linear regression models in Section 4.4, followed by the estimation of time series models in Section 4.5. We close the chapter with Section 4.6, where we compare the orthodox approach to inferential statistics and parameter estimation with the Bayesian view.

The newcomers will certainly realize that model estimation is a huge body of statistical knowledge, which we cannot cover in any satisfactory way in a short chapter. In particular, estimating time series models may require some deep knowledge, as is also the case when estimating a copula. We do not deal with copula estimation here, but later in Section 5.7; there we illustrate how sampling from a copula may be achieved by the R package copula, which also includes a function fitCopula for fitting purposes. We do not go into details, but we stress that the principle of parameter estimation by maximum likelihood, which we do outline in this chapter, is the theoretical underpinning of many of these estimation approaches.

# 4.1 Basic inferential statistics in R

In this section we deal with two very basic and related topics: confidence intervals for the mean and elementary hypothesis testing. Our aim is to review the foundations of these statistical tools, in order to get acquainted with the issues involved in more complicated cases, such as testing the significance of correlations, etc. Most readers could probably skip this section, or possibly just skim through it to see which functions R offers to accomplish these tasks. Not surprisingly, given their conceptual links, the relevant R functions boil down to one, t.test.

## 4.1.1 CONFIDENCE INTERVALS

Most of us are introduced to inferential statistics through the calculation of a confidence interval for an expected value μ. This concept is relevant on both ends of a Monte Carlo simulation, as it used to define the input random variables as well as to analyze the output. Given a sample _X_ _i_ , _i_ = 1,..., _n_ of i.i.d. (independent and identically distributed) random variables, the drill is as follows:

**1.** Compute sample statistics such as sample mean and sample variance:

**2.** Choose a confidence level (1 − α) and pick the corresponding quantile _t_ _n_ −1, 1−α/2 from a _t_ distribution with _n_ − 1 degrees of freedom.

**3.** Calculate the confidence interval

(4.1)

This procedure is so easy to perform that one tends to forget that it relies on some important assumptions. The following observations are in order:

  Strictly speaking, the above procedure is correct for normal random variables only. In fact, if variables _X_ _i_ ~ N(μ, σ2) _and_ they are independent, then it is true that the following standardized statistic is normal:

(4.2)

If we replace σ by its sample counterpart _S_ we find a Student _t_ distribution:

(4.3)

which implies

By rearranging this relationship we obtain the confidence interval of Eq. (4.1). A large part of inferential statistics relies on similar distributional results. If we apply the procedure to a different distribution, what we find is at best a good approximation for a suitably large sample; with a small sample and a skewed distribution, we should repeat the drill accounting for the specific features of that distribution.

  It is also very important to stress the role of independence. It is independence in the sample that allows us to write3

  When estimating sample variance _S_ 2, we recall that division by _n_ − 1 is required in order to obtain an unbiased estimator, i.e., to make sure that

Unbiasedness, as we insist upon later, is a basic requirement of a sensible estimator.

  When analyzing the output of a Monte Carlo simulation, the sample size is usually rather large. Hence, we usually replace quantiles of the _t_ distribution with quantiles _z_ 1−α/2 from the standard normal. However, this need not apply to input analysis.

Another danger is a common misinterpretation of confidence intervals. After calculating a confidence interval ( _l, u_ ), we _cannot_ say that

(4.4)

This statement does not make any sense, as all of the involved quantities are _numbers._ In the orthodox framework of statistics, μ is an unknown number; the extreme points of the confidence interval, _l_ and _u_ , are numerical realizations of random variables _L_ and _U_. Hence, the above probabilistic statement makes no sense. We should rephrase the claim of Eq. (4.4) with reference to the random variables _L_ and _U_ , i.e., the lower and upper bound of the interval _before sampling._ Once again: We have to realize that parameters are unknown numbers in orthodox statistics, not random variables.4 This is illustrated empirically and clarified below by a simple Monte Carlo experiment.

### Confidence intervals in R

Building standard confidence intervals in R is accomplished by the function t.test (x, conf. level = 0.95). The function name may look weird at first, but it is justified by the fact that the calculations needed to compute the confidence interval are related to those needed to run a hypothesis test, as we show later. The function receives a vector containing the sample and an optional parameter specifying the confidence level, which is 95% by default:

    > set.seed(55555)
    > X = rnorm(20,mean=10,sd=20)
    > t.test(X)
    One Sample t-test
    data: X
    t = 1.1709, df = 19, p-value = 0.2561
    alternative hypothesis: true mean is not equal to 0
    95 percent confidence interval:
    -3.960321 14.017022
    sample estimates:
    mean of x
    5.02835

The above output is a bit confusing, as it is cluttered by irrelevant information about a hypothesis test. It might be better to collect the output in a variable and then inspect what we are really interested in:

    > ci <- t.test(X,conf.level = 0.99)
    > ci$estimate
    mean of x
    5.02835
    > ci$conf.int
    [1] −7.258179 17.314880
    attr(,"conf.level")
    [1] 0.99

As expected, the 99% interval is larger than the 95% interval, and both are rather large due to the small sample size and the large standard deviation with respect to the expected value.

Armed with the above function, we may check the empirical _coverage_ of a confidence interval, i.e., the probability that it contains the true expected value. To this aim, it is instructive to compare the result with a normal and an exponential distribution of similar variability.5 By running the code in Fig. 4.1, we obtain the following results:

**FIGURE 4.1** **Checking the coverage of confidence intervals empirically.**

    > coverageExp
    [1] 0.9208
    > coverageNorm
    [1] 0.95

As expected, coverage with a skewed distribution is not exactly as specified.

In this section we have only considered confidence intervals for the expected value of a random variable. Different confidence intervals should be used, e.g., when we are faced with the task of estimating variances or probabilities. Another relevant and quite nontrivial case is the estimation of quantiles. Since this kind of application, as far as simulation is concerned, is more related to output analysis, we defer an extended treatement of confidence intervals to Section 7.3.

## 4.1.2 HYPOTHESIS TESTING

The basic hypothesis test one may wish to run concerns the expected value:

  We test the _null hypothesis H_ 0: μ = μ0, for a given μ0,

  against the _alternative hypothesis H_ a: μ ≠ μ0.

In the normal case, we rely on the distributional result of Eq. (4.3), where the unknown expected value μ is replaced by the hypothesized value μ0. This shows that, _if_ the null hypothesis is indeed true, then

In other words, the standardized test statistic

if the null hypothesis is true, has Student's _t_ distribution with _n_ − 1 degrees of freedom, and it should fall within bounds corresponding to quantiles. If _T_ falls outside that interval, there are two possible explanations: It may just be bad luck, or maybe the null hypothesis is wrong. We cannot be sure of either, and we may make two types of error: We may reject a true hypothesis, or we may accept a false one. The elementary approach is conservative and keeps the probability of rejecting a true null hypothesis under control. Therefore, we form a rejection region consisting of two tails,

and we reject the null hypothesis if the test statistic _T_   RJ. Here α plays the role of a significance level or, better said, the probability of rejecting the null hypothesis if it is true.6 The rejection region consists of a single tail if the alternative hypothesis is one-sided:

Note that if we increase α we restrict the rejection region, which implies that, all other factors being equal, it is easier to reject, possibly making a mistake. Statistical folklore dictates that 5% is a sensible choice for the significance level, but this is at best a rule of thumb.

Hypothesis testing is easily accomplished by the t.test function:

    > set.seed(55555)
    > X <- rnorm(10,10,20)
    > t.test(X,mu=20)

            One Sample t-test

    data: X
    t = −1.4468, df = 9, p-value = 0.1819
    alternative hypothesis: true mean is not equal to 20
    95 percent confidence interval:
    -6.881385 25.909259
    sample estimates:
    mean of x
    9.513937

In the above snapshot we sample a normal distribution with expected value 10 and test the hypothesis that the expected value is 20. We notice that the confidence interval is large, reflecting considerable variability due to the large standard deviation. By sheer luck, the sample mean is 9.513937, rather close to the true value, but the confidence interval is pretty large. Now, should we accept the null hypothesis or not? One trick is to observe that 20 is within the confidence interval, thus we cannot rule out this value. This way of thinking works here, but it should be discouraged as it does not work well in general, especially with one-tailed testing.

The fundamental output of the procedure is the _p_ -value, which is 0.1819. This is the probability of getting a statistic like that sample mean, or a larger one in absolute value, by pure randomness, assuming that the true expected value is 20. We may compare this _p_ -value against a significance level of 5%: Since the _p_ -value is larger, we do not feel at all confident in rejecting the null hypothesis. In this case, we do know that the null hypothesis is wrong since the true expected value is 10, but from a practical point of view we do not have evidence strong enough. Also note that we cannot really say that we "accept" the null hypothesis, as the sample statistic is far from 20; what we can say is that we "fail to reject it."

We may easily change the alternative hypothesis and run one-tailed tests:

    > t.test(X,mu=20,alternative='less')
            One Sample t-test
    data: X
    t = −1.4468, df = 9, p-value = 0.09093
    alternative hypothesis: true mean is less than 20
    95 percent confidence interval:
        -Inf 22.7997
    sample estimates:
    mean of x
    9.513937
    > t.test(X,mu=20,alternative='greater')
            One Sample t-test
    data: X
    t = −1.4468, df = 9, p-value = 0.9091
    alternative hypothesis: true mean is greater than 20
    95 percent confidence interval:
    -3.771823 Inf
    sample estimates:
    mean of x
    9.513937

The newcomer is strongly invited to compare the two _p_ -values and to interpret their difference.

We may run a host of hypothesis tests in R, which include:

  Testing the difference in the expected values of two large independent samples: t.test(X,Y)

  Testing the difference in the expected values of two paired samples: t.test(X,Y,paired=TRUE)

  Testing the significance of correlation: cor.test (X), discussed later in Section 4.1.3

The second test is relevant, for instance, in evaluating the performance of two systems or two strategies on a set of simulated scenarios. In such a case, it is fair and makes sense to compare the results on the _same_ set of scenarios and to check the differences; hence, the observations in the sample are not independent and should be paired to check if the observed difference is statistically significant.

Unlike trivial tests on the mean of a normal population, other testing procedures may rely on sophisticated distributional results, but they more or less follow the same drill:

  We select some suitable statistic.

  We derive distributional results about that statistic, assuming the null hypothesis is true. These results may rely on some underlying probability distribution and are often approximations valid for large samples.

  Based on these distributional results, the rejection region and the corresponding _p_ -values are determined. Monte Carlo sampling is also used to this purpose, when an approximation is not available, or it is feared that it is a poor one for a limited sample size.

We need not be concerned too much with the theoretical underpinnings of each testing procedure, but we should be aware of the underlying assumptions and the resulting limitations in terms of applicability; then, what we have to look for is the _p_ -value.

## 4.1.3 CORRELATION TESTING

Testing the significance of correlation is an important task in statistics, and it may also be a useful tool in preparing and analyzing Monte Carlo experiments. A starting point is the following formula for sample covariance:

(4.5)

where _n_ is the sample size, i.e., the number of observed _pairs_ ( _X_ _i_ , _Y_ _i_ ). The above formula is a straightforward sample counterpart of covariance and can also be rewritten as follows:

In passing, we note that when estimating a large covariance matrix, plenty of data are needed to obtain reliable estimates. Such a luxury cannot be afforded quite often, and this motivates the factor models discussed in Section 3.8.2.

To estimate the correlation coefficient ρ _XY_ between _X_ and _Y_ , we may just plug sample covariance _S_ _XY_ and sample standard deviations _S_ _X_ , _S_ _Y_ into its definition, resulting in the _sample coefficient of correlation_ , or sample correlation for short:

(4.6)

The factors _n_ − 1 in _S_ _XY_ , _S_ _X_ , and _S_ _Y_ cancel each other, and it can be proved that −1 ≤ _r_ _XY_ ≤ + 1, just like its probabilistic counterpart ρ _XY_.

We should always keep in mind that we are referring to Pearson's correlation, whose limitations have been stressed in Section 3.4.1. Nevertheless, correlation analysis plays an important role in Monte Carlo models:

  In input analysis, we should check if some variables are correlated in order to model them correctly.

  In output analysis, it may be important to check the correlation (or lack thereof) between output estimates, e.g., to find the right batch length in the method of batches. See Section 7.1.

  When applying variance reduction by control variates, it may be important to check the strength of the correlation between the crude Monte Carlo estimator and the control variate that we considering. See Chapter 8.

Whatever the reason why we are estimating a correlation, we should always wonder whether it is statistically significant. To see the point, imagine estimating the correlation when _n_ = 2. Since it is possible to fit a straight line going through two points, the resulting correlation is always 1. It is clear that the magnitude of the correlation should be checked against the sample size, and a simple strategy is to test the null hypothesis

against the alternative hypothesis

However, we need a statistic whose distribution under the null hypothesis is fairly manageable. One useful result is that, if the sample is normal, the statistic

is approximately distributed as a _t_ variable with _n_ − 2 degrees of freedom, for a suitably large sample. This can be exploited to come up with correlation tests.

In R, testing correlations is accomplished by the function cor.test. The snapshot in Fig. 4.2 generates two vectors of independent normals, X1 and X2, to which a common factor is added. This has the effect of producing a correlation coefficient of 0.3262. However, the _p_ -value is 0.1605, which makes the correlation not quite significant, due to the small sample size. The reader is urged to repeat the experiment with _n_ = 2000 and check that now the correlation is significant.

**FIGURE 4.2** **Testing correlations.**

# 4.2 Parameter estimation

Introductory treatments of inferential statistics focus on normal populations. In that case, the two parameters characterizing the distribution, μ and σ2, coincide with expected value, the first-order moment, and variance, the second-order central moment. Hence, students might believe that parameter estimation is just about calculating sample means and variances. It is easy to see that this is not the case.

###   Example 4.1 Parameters of a uniform distribution

Consider a uniform random variable _X_  . We recall that

Clearly, the sample mean   and the sample variance _S_ 2 do not provide us with direct estimates of the parameters _a_ and _b._ However, we might consider the following way of transforming the sample statistics into estimates of parameters. If we substitute μ and σ2 with their estimates, we find

Note that, in taking the square root of variance, we should only consider the positive root, since standard deviation cannot be negative. Solving this system yields the following estimates:

This example suggests a general strategy to estimate parameters:

  Estimate moments on the basis of a random sample.

  Set up a suitable system of equations relating parameters and moments and solve it.

Indeed, this is the starting point of a general parameter estimation approach called _method of moments._ However, a more careful look at the example should raise an issue. Consider the order statistics of a random sample of _n_ observations of a uniform distribution:

If we take the above approach, _all_ of these observations play a role in estimating _a_ and _b._ Yet, this is a bit counterintuitive; in order to characterize a uniform distribution, we need a lower and an upper bound on its realizations. So, it seems that only _U_ (1) and _U_ ( _n_ ), i.e., the smallest and the largest observations, should play a role. We might suspect that there are alternative strategies for finding point estimators and then, possibly, confidence intervals. In this section we outline two approaches: the aforementioned method of moments and the method of maximum likelihood. But before doing so, since there are alternative ways to build estimators, it is just natural to wonder how we can compare them. Therefore, we should first list the desirable properties that make a good estimator.

## 4.2.1 FEATURES OF POINT ESTIMATORS

We list here a few desirable properties of a point estimator   for a parameter θ. When comparing alternative estimators, we may have to trade off one property for another. An estimator   is _unbiased_ if

It is easy to show that the sample mean is an unbiased estimator of the expected value. Biasedness is related to the expected value of an estimator, but what about its variance? Clearly, ceteris paribus, we would like to have an estimator with low variance.

**DEFINITION 4.1 (Efficient unbiased estimator)** _An unbiased estimator_  1 _is more efficient than another unbiased estimator_  2 _if_

Note that we must compare unbiased estimators in assessing efficiency. Otherwise, we could obtain a nice estimator with zero variance by just choosing an arbitrary constant. It can be shown that, under suitable hypotheses, the variance of certain estimators has a lower bound. This bound, known as the _Cramér–Rao bound_ , is definitely beyond the scope of this book, but the message is clear: We cannot go below a minimal variance. If the variance of an estimator attains that lower bound, then it is efficient. Another desirable property is consistency. An estimator is _consistent_ if   → θ, where the limit must be understood in some probabilistic sense, when the sample size increases. For instance, the statistic

is not an unbiased estimator of variance σ2, as we know that we should divide by _n_ − 1 rather than _n._ However, it is a consistent estimator, since when _n_ → ∞ there is no difference between dividing by _n_ − 1 or _n._ Incidentally, by dividing by _n_ we lose unbiasedness but we retain consistency while reducing variance. Sometimes, we may prefer a biased estimator, if its variance is low, provided that it is consistent.

## 4.2.2 THE METHOD OF MOMENTS

We have already sketched an application of this method in Example 4.1. To state the approach in more generality, let us introduce the sample moment of order _k_ :

The sample moment is the sample counterpart of moment _m_ _k_ = E[ _X_ _k_ ]. Let us assume that we need an estimate of _k_ parameters θ1, θ2,..., θ _k_. The method of moments relies on the solution of the following system of equations:

In general, this is a system of nonlinear equations that may be difficult to solve, and we cannot be sure that a unique solution, if any, exists. However, assuming that there is in fact a unique solution, we may just replace moments _m_ _k_ by sample moments _M_ _k_ , and solve the system to obtain estimators  1,  2,...,  _k_.

###   Example 4.2 Application to the normal distribution

Let ( _X_ 1, _X_ 2,... _X_ _n_ ) be a random sample from a normal distribution with unknown parameters (θ1, θ2) ≡ (μ, σ). The relationship between the parameters and the first two moments is _m_ 1 = μ, _m_ 2 = σ2 \+ μ2. Plugging sample moments and solving the system yields

These estimators look familiar enough, but note that we do not obtain an unbiased estimator of variance.

## 4.2.3 THE METHOD OF MAXIMUM LIKELIHOOD

The method of maximum likelihood is an alternative approach to find estimators in a systematic way. Imagine that a random variable _X_ has a PDF characterized by a single parameter θ, denoted by _f_ _X_ ( _x_ ; θ). If we draw a sample of _n_ i.i.d. variables from this distribution, the joint density is just the product of individual PDFs:

This is a function depending on the parameter θ. Also note that we use lowercase letters _x_ _i_ to point out that we are referring to a specific set of numerical realizations of the random variable _X_. If we are interested in estimating θ, _given_ a sample _X_ _i_ = _x_ _i_ , _i_ = 1,..., _n_ , we may swap the role of variables and parameters and build the _likelihood function_

(4.7)

The shorthand notation _L_ (θ) is used to emphasize that this is a function of the unknown parameter θ, for a given sample of observations. On the basis of this framework, intuition suggests that we should select the parameter yielding the largest value of the likelihood function. For a discrete random variable, the interpretation is more natural: We select the parameter maximizing the probability of what we have indeed observed. For a continuous random variable, we cannot really speak of probabilities, but the rationale behind the method of maximum likelihood should be apparent: We are just trying to find the best explanation of what we observe. The acronym MLE, for maximum-likelihood estimator or estimation, is used to refer to the approach.

###   Example 4.3 MLE for the exponential distribution

Let us consider the PDF of an exponential random variable with parameter λ:

Given observed values _x_ 1,..., _x_ _n_ , the likelihood function is

Quite often, rather than attempting direct maximization of the likelihood function, it is convenient to maximize its logarithm, i.e., the _log-likelihood function_

(4.8)

The rationale behind this function is easy to grasp, since by taking the logarithm of the likelihood function, we transform a product of functions into a sum of functions:

The first-order optimality condition yields

The result is rather intuitive, since E[ _X_ ] = 1/λ.

The idea of MLE is immediately extended to the case requiring the estimation of multiple parameters: We just solve a maximization problem involving multiple variables.

###   Example 4.4 MLE for the normal distribution

Let us consider maximum-likelihood estimation of the parameters θ1 = μ and θ2 = σ2 of a normal distribution. Straightforward manipulation of the PDF yields the log-likelihood function:

The first-order optimality condition with respect to μ is

Hence, the maximum-likelihood estimator of μ is just the sample mean. If we apply the first-order condition with respect to σ2, plugging the estimator  , we obtain

In the two examples above, maximum likelihood yields the same estimator as the method of moments. Then, one could well wonder whether there is really any difference between the two approaches. The next example provides us with a partial answer.

###   Example 4.5 MLE for the uniform distribution

Let us consider a uniform distribution on interval [0, θ]. On the basis of a random sample of _n_ observations, we build the likelihood function

This may look a bit weird at first sight, but it makes perfect sense, as the PDF is zero for _x_ > θ. Indeed, θ cannot be smaller than the largest observation:

In this case, since there is a constraint on θ, we cannot just take the derivative of _L_ (θ) and set it to 0 (first-order optimality condition). Nevertheless, it is easy to see that likelihood is maximized by choosing the smallest θ, subject to constraint above. Hence, using the notation of order statistics, we find

While the application of maximum likelihood to exponential and normal variables looks a bit dull, in the last case we start seeing something worth noting. The estimator is quite different from what we have obtained by using the method of moments. The most striking feature is that this estimator does _not_ use the whole sample, but just a single observation. We leave it as an exercise for the reader to check that, in the case of a uniform distribution on the interval [ _a, b_ ], maximum-likelihood estimation yields

Indeed, the smallest and largest observations are _sufficient statistics_ for the parameters _a_ and _b_ of a uniform distribution.7 We refrain from stating a formal definition of sufficient statistics, but the idea is rather intuitive. Given a random sample **X** , a sufficient statistic is a function _T_ ( **X** ) that captures all of the information we need from the sample in order to estimate a parameter. As a further example, sample mean is a sufficient statistic for the parameter μ of a normal distribution. This concept has far-reaching implications in the theory of inferential statistics.

A last point concerns unbiasedness. In the first two examples, we have obtained biased estimators for variance, even though they are consistent. It can be shown that an unbiased estimator of θ for the uniform distribution on [0, θ] is

(4.9)

rather than _X_ ( _n_ ). Again, we see that maximum likelihood yields a less than ideal estimator, even though for _n_ → ∞ there is no real issue. It turns out that maximum-likelihood estimators (MLEs) do have limitations, but a few significant advantages as well. Subject to some technical conditions, the following properties can be shown for MLEs:

  They are consistent.

  They are asymptotically normal.

  They are asymptotically efficient.

  They are invariant, in the sense that, given a function _g_ (·), the MLE of γ = _g_ (θ) is _g_ ( ), where   is the MLE of θ.

As a general rule, finding MLEs requires the solution of an optimization problem by numerical methods, but there is an opportunity here. Whatever constraint we want to enforce on the parameters, depending on domain-specific knowledge, can be easily added, resulting in an optimization problem that can be solved numerically. Plenty of numerical optimization methods are available to this purpose.

## 4.2.4 DISTRIBUTION FITTING IN R

R offers some tools to fit distributions by maximum likelihood, such as

  mle, contained in the stats4 package,

  fitdistr, contained in the MASS package.

The former function leaves the task of specifying minus the log-likelihood function to the user and produces an object of an S4 class. The second one is a bit more user friendly. The use of these functions is best understood by a simple example related to a gamma distribution. The code in Fig. 4.3 produces the following (abridged) output:

**FIGURE 4.3** **Fitting a gamma distribution.**

    > fit1@coef
    shape rate
    2.233724 11.062195
    > confint(fit1)
    Profiling...
    2.5 % 97.5 %
    shape 1.985328 2.503327
    rate 9.684267 12.559251
    > fit2$estimate
    shape rate
    2.233726 11.062211
    > fit2$sd
    shape rate
    0.1320868 0.7331218

We notice a slight difference in the estimates of shape and rate parameters, due to the optimization algorithms embedded in the two functions. In the case of mle, we obtain an S4 object; this is why we have to access its properties by fit1@coef rather than by fit1$coef. We also select an optimization method, L-BFGS-B, so that we may enforce lower bounds on the parameters (otherwise, warnings may be obtained if they get negative in the search process). Also notice the use of a named list to provide initial values for the numerical optimization algorithm, as usual in nonlinear programming. In both cases we have some information on the reliability of the estimate, either via a confidence interval or standard errors.

# 4.3 Checking the fit of hypothetical distributions

So far, we have been concerned with parameters of probability distributions. We never questioned the fit of the distribution itself against empirical data. For instance, we might assume that a population is normally distributed, and we may estimate and test its expected value and variance. However, normality should not be taken for granted, just like any other claim about the underlying distribution. Sometimes, specific knowledge suggests strong reasons that justify the assumption; otherwise, this should be tested in some way. When we test whether experimental data fit a given probability distribution, we are not really testing a hypothesis about a parameter or two; in fact, we are running a _nonparametric_ test.

In this section we illustrate three kinds of approach:

  The chi-square test, which is general purpose and, loosely speaking, checks the fit in terms of histograms and densities.

  The Kolmogorov–Smirnov test, which is is general purpose as well, but checks the fit in terms of the cumulative distribution.

  Two examples ad hoc tests, aimed at checking normality, namely, the Shapiro–Wilks and Jarque–Bera tests.

All of the above procedures will be illustrated using R.

Before describing formal approaches, let us comment on intuitive and graphical checks. Clearly, visual inspection of a histogram and a comparison with the theoretical shape of a PDF may help. However, sometimes the judgement is not so easy to make. To see the point, let us generate a normal sample and look at the resulting histograms in Fig. 4.4:

**FIGURE 4.4** **Histograms of a normal sample.**

    > set.seed(55555)
    > par(mfrow=c(1,2))
    > X=rnorm(200,10, 40)
    > hist(X)
    > hist(X,nclass=50)
    > par(mfrow=c(1,1))

In the plot on the left of Fig. 4.4, there are very few bins, and the distribution might look compatible with a normal, but other densities as well. If we increase resolution by using more bins, statistical sampling makes the histogram too bumpy to be of any use. By the way, these observations also apply to the chi-square test, which formalizes the intuition of comparing histograms. Sometimes, checking the CDF is a better idea. In fact, another graphical tool relies on plotting the sample quantiles against the theoretical ones, obtaining a Q-Q plot. Using the above sample, we produce the plot in Fig. 4.5 as follows:

**FIGURE 4.5** **Checking the fit of a normal distribution.**

    > qqnorm(X)
    > qqline (X)

In principle, the points should lie on a straight line, since the sample is in fact normally distributed. This is indeed the case, with the exception of tails, which is to be expected after all as they are always critical.

## 4.3.1 THE CHI-SQUARE TEST

The idea behind the chi-square test is fairly intuitive and basically relies on a relative frequency histogram, although the technicalities do require some care. The first step is to divide the range of possible observed values in _J_ disjoint intervals, corresponding to bins of a frequency histogram. Given a probability distribution, we can compute the probability _p_ _j_ , _j_ = 1,..., _J_ , that a random variable distributed according to that distribution falls in each bin. If we have _n_ observations, the number of observations that should fall in interval _j_ , if the assumed distribution is indeed the true one, should be _E_ _j_ = _np_ _j_. This number should be compared against the number _O_ _j_ of observations that actually fall in interval _j_ ; a large discrepancy would suggest that the hypothesis about the underlying distribution should be rejected. Just like any statistical test, the chi-square test relies on a distributional property of a statistic. It can be shown that, for a large sample size, the statistic

has (approximately) a chi-square distribution. We should reject the hypothesis if χ2 is too large, i.e., if χ2 > χ1−α, _m_ 2, where:

  χ1−α, _m_ 2 is a quantile of the chi-square distribution.

  α is the significance level of the test.

  _m_ is the number of degrees of freedom.

What we are missing here is _m_ , which depends on the number of parameters of the distribution that we have estimated using the data. If no parameter has been estimated, i.e., if we have assumed a specific parameterized distribution prior to observing data, the degrees of freedom are _J_ − 1; if we have estimated _p_ parameters, we should use _J − p_ − 1.

R offers a chisq.test function that, among other things, can be used to check the fit of a continuous distribution. To this aim, we have to bin data and compute observed and theoretical frequencies:

    set.seed(55555)
    shape <- 2
    rate <- 10
    X <- rgamma(500, shape, rate)
    breakPts <- c(seq(0,0.8,0.1), 2)
    probs <- pgamma(breakPts,shape,rate)
    binProbs <- diff(probs)
    f.obs <- hist(X, breakPts, plot = FALSE)$counts
    chisq.test(x=f.obs,p=binProbs/sum(binProbs))

Note that we divide the vector binProbs by its sum to make sure that probabilities add up to 1. We also use hist as a convenient tool to bin data. The above code results in the following output:

            Chi-squared test for given probabilities
    data: f.obs
    X-squared = 4.4634, df = 8, p-value = 0.8131
    Warning message:
    In chisq.test(x = f.obs, p = binProbs/sum(binProbs)):
    Chi-squared approximation may be incorrect

As expected, we cannot reject the null hypothesis, but we get a warning message. Indeed, the idea of the chi-square test is pretty intuitive; however, it relies on approximated distributional results that may be critical. Another tricky point is that the result of the test may depend on the number and placement of bins. Rules of thumb have been proposed and are typically embedded in good statistical software.

## 4.3.2 THE KOLMOGOROV–SMIRNOV TEST

Continuing the above example, where the variable x stores a sample from a gamma distribution with given shape and rate parameters, it is very easy to run the Kolmogorov–Smirnov test in R:

    > ks.test(X,pgamma, shape, rate)
    One-sample Kolmogorov–Smirnov test
    data: X
    D = 0.0447, p-value = 0.2704
    alternative hypothesis: two-sided

The Kolmogorov–Smirnov test, unlike the chi-square test, relies on the CDF. Indeed, as you may see above, we have to provide ks.test with a function computing cumulative probabilities, pgamma in our case, along with all of the relevant parameters.

The idea behind the test relies on a comparison between the assumed CDF _F_ _X_ ( _x_ ) and an empirical CDF defined as follows:

where #{...} denotes the cardinality of a set. The distance is measured in a worst-case sense:8

From a computational point of view, this distance can be calculated as

where

and _X_ ( _i_ ) is the _i_ th order statistic (obtained by sorting values _X_ _i_ in increasing order). It is clear that a large value _D_ does not support the assumed distribution. What is not so clear is how to devise a rejection region and find the critical values defining its boundary. This depends on the underlying distribution, and Monte Carlo sampling is often used to find such values.

## 4.3.3 TESTING NORMALITY

The R core includes the Shapiro–Wilk test, implemented in the shapiro. test function:

    > set.seed(55555)
    > X=rnorm(20,10,20)
    > shapiro.test(X)
            Shapiro-Wilk normality test
    data: X
    W = 0.9512, p-value = 0.385

In this case, the function returns a rather large _p_ -value, such that we cannot reject the null hypothesis that the distribution is normal. Given this structure, it is clear that we get a conservative test. Strong evidence is needed to reject normality. Let us try a normal and a gamma distribution:

    > X=rexp(20,10)
    > shapiro.test(X)
            Shapiro-Wilk normality test
    data: X
    W = 0.9006, p-value = 0.04237
    > X=rgamma(20,2, 10)
    > Shapiro.test(X)
    Shapiro-Wilk normality test
    data: X
    W = 0.623, p-value = 5.14e-06

Evidence is reasonably strong in the former case, but not as strong as in the latter. The test statistic takes values between 0 and 1, and it compares two estimates of variance; when the statistic is close to 1 we cannot reject normality. In the Jarque–Bera test we take advantage of third- and fourth-order moments, and of the fact that skewness is zero and kurtosis is 3 for a normal. The test statistic is

where   and   are estimates of skewness and kurtosis, respectively. Clearly, the test statistic is non-negative and should be close to zero, according to the null hypothesis that the underlying distribution is normal. For a large sample the test statistic is a chi-square variable, which can be used to define a rejection region for the test. However, since this is not quite robust for small and medium size samples, critical values can be obtained by Monte Carlo sampling.

# 4.4 Estimation of linear regression models by ordinary least squares

Under suitable assumptions, multiple linear regression models may be estimated by ordinary least squares (OLS), which is accomplished in R by the lm function. This is illustrated by the code in Fig. 4.6, which produces the following output:

**FIGURE 4.6** **Fitting a multiple linear regression model.**

    Call:
    lm(formula = Y ~ X1 + X2)
    Residuals:
        Min     1Q Median     3Q     Max
    −12.396 −9.859 −6.075 8.818 29.831
    Coefficients:
                Estimate Std.    Error    t value    Pr(>|t|)
    (Intercept)    21.6349        11.0946        1.950    0.092162 .
    X1            4.7902        1.7912        2.674    0.031802 *
    X2            2.7463        0.4732        5.803    0.000661 ***
    ---
    Signif.codes: 0 '**' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 '' 1
    Residual standard error: 16 on 7 degrees of freedom
    Multiple R-squared: 0.8727,Adjusted R-squared: 0.8363
    F-statistic: 23.98 on 2 and 7 DF, p-value: 0.000737

We obtain parameter estimates, along with an assessment of their significance. The _p_ -value of the coefficient for _X_ 1 is significant, despite the high level of noise. The coefficient of determination _R_ 2 is 0.8727, but the adjusted _R_ 2 is 0.8363. We also obtain an overall assessment of the model in the _p_ -value of the _F_ -statistic. The reader is invited to check the effect of reducing the coefficient multiplying _X_ 1 in the data generation process.

We obtain confidence intervals for the estimated coefficients, and we may also test normality and uncorrelation of residuals:

    > confint(lmod1)
                    2.5 % 97.5 %
    (Intercept) −4.5996144 47.869439
    X1           0.5546597 9.025779
    X2           1.6272928 3.865392
    > shapiro.test(lmod1$resid)
    Shapiro-Wilk normality test
    data: lmod1$resid
    W = 0.8294, p-value = 0.03287
    > library(lmtest)
    > dwtest(lmod1)
    Durbin-Watson test
    data: lmod1
    DW = 2.5181, p-value = 0.5605
    alternative hypothesis:
    true autocorrelation is greater than 0

Using lm requires the specification of a formula. In the case above, Y~X1+X2 is self-explanatory, but formulas in R are much more powerful than that. Say that we want to fit a polynomial regression model of order 2:

Of course, we can just evaluate each power of the explanatory variable and run a linear regression (the model is nonlinear in variables, but linear in parameters). However, the code in Fig. 4.7 shows a shortcut: by setting raw=TRUE, we force R to use familiar polynomials rather than orthogonal ones.9 We obtain the following output:

**FIGURE 4.7** **Fitting a polynomial regression model.**

    Call:
    lm(formula = Y ~ poly(X, 2, raw = TRUE))
    Residuals:
    Min    1Q    Median    3Q    Max
    -13.673    -9.653    -5.712    8.714    29.547
    Coefficients:
        Estimate    Std.    Error    t value Pr(>|t|)
    (Intercept)    290.82646    7.53560    38.59    2.04e-09 ***
    poly(X,2,raw=TRUE)1    4.73459    0.46672    10.14    1.95e-05 ***
    poly(X,2,raw=TRUE)2    0.49706    0.04724    10.52    1.53e-05 ***
    ---
    Signif.codes: 0 '**' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 '' 1
    Residual standard error: 16.02 on 7 degrees of freedom
    Multiple R-squared: 0.9664, Adjusted R-squared: 0.9568
    F-statistic: 100.7 on 2 and 7 DF, p-value: 6.951e-06

There are cases in which nonlinearity should be used to capture interaction between variables. Consider the model

where δ   {0,1} is a dummy variable modeling a categorical variable. It is worth noting that when δ = 0, the data generating process is

whereas, when δ = 1, _both_ intercept and slope change:

The code in Fig. 4.8 illustrates such a case. The input data are depicted in Fig. 4.9, along with the fitted lines resulting from setting δ = 0 and δ = 1, respectively. We can see clearly the difference in intercept and slope between the two groups.

**FIGURE 4.8** **Fitting a regression model with interactions.**

**FIGURE 4.9** **A regression model with interactions.**

The following model summary shows that, when using the formula Y~X*Z, interactions are automatically accounted for:

    Call:
    lm(formula = Y ~ X * Z)
    Residuals:
    Min    1Q    Median    3Q    Max
    -27.0281    -5.4642    -0.1196    5.5480    22.4519
    Coefficients:
        Estimate Std.    Error    t value    Pr(>|t|)
    (Intercept)    29.6160    3.5574    8.325    2.62e-12 ***
    X    4.8991    0.1135    43.147    < 2e-16 ***
    Z    38.6779    4.6058    8.398    1.90e-12 ***
    X:Z    -2.8403    0.1584    -17.928    < 2e-16 ***
    Signif.codes: 0 '**' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 '' 1
    Residual standard error: 10.04 on 76 degrees of freedom
    Multiple R-squared: 0.9732, Adjusted R-squared: 0.9722
    F-statistic: 921.3 on 3 and 76 DF, p-value: < 2.2e-16

# 4.5 Fitting time series models

Fitting time series models requires the use of possibly sophisticated procedures, whose first step is selecting the type and order of the model. This model identification step can be accomplished using the concepts that we have outlined in Section 3.6. Given a model structure, estimating its parameters calls for the application, e.g., of maximum-likelihood methods. In this section we just simulate a couple of models and check whether we are able to recover their underlying structure, on the basis of the resulting sample paths, by using R functions.

To simulate an ARIMA model, we may write our own code, as we did in Section 3.6. For the sake of convenience, here we use the function arima.sim, which simulates an ARIMA model and accepts, among other things, the following input arguments:

  model: a named list with vector-valued components, ar and ma, specifying the AR and the MA part of the model

  n: the time horizon for the simulation

  n. start: the number of time buckets that should be simulated and discarded to avoid issues with the transient phase resulting from the initial conditions

  rand.gen: a function to generate shocks; by default, rnorm is used, but we may use, e.g., a Student's _t_ distribution to allow for slightly fatter tails

Then, we estimate the model using two functions:

  arima, which needs a specification of the model order

  auto.arima, provided by the forecast package, which automates the search for the model order

The code in Fig. 4.10 produces the following (edited) output:

**FIGURE 4.10** **Simulating and fitting an ARMA(2,2) model.**

    > arima(x1,order=c(2,0,2))
    Series: x1
    ARIMA (2, 0, 2) with non-zero mean
    Coefficients:
        ar1    ar2    ma1    ma2    intercept
        0.6477    -0.3853    -0.0880    0.3118    -0.0105
    s.e.    0.1364    0.0652    0.1362    0.0517    0.0295
    sigma^2 estimated as 0.4749: log likelihood=-1570.18
    AIC=3152.35 AICc=3152.41 BIC=3184.23
    > auto.arima(x1)
    Series: x1
    ARIMA (2, 0, 2) with zero mean
    Coefficients:
        ar1    ar2    ma1    ma2
        0.6474    -0.3849    -0.0875    0.3117
    s.e.    0.1363    0.0652    0.1361    0.0517
    sigma^2 estimated as 0.475: log likelihood=-1570.24
    AIC=3150.48 AICc=3150.52 BIC=3177.05
    > arima(x2, order=c(2,0,2))
    Series: x2
    ARIMA (2, 0, 2) with non-zero mean
    Coefficients:
        ar1    ar2    ma1    ma2    intercept
        0.9599    -0.5477    -0.3572    0.2970    -0.0076
    s.e.    0.0738    0.0425    0.0772    0.0439    0.0282
    sigma^2 estimated as 0.4677: log likelihood=-1558.8
    AIC=3129.6 AICc=3129.66 BIC=3161.48
    > auto.arima(x2)
    Series: x2
    ARIMA (2, 0, 2) with zero mean
    Coefficients:
        ar1    ar2    ma1    ma2
        0.9599    -0.5477    -0.3573    0.2970
    s.e.    0.0738    0.0425    0.0772    0.0439
    sigma^2 estimated as 0.4678: log likelihood=-1558.84
    AIC=3127.68 AICc=3127.72 BIC=3154.24

Note that arima needs a specification of the model order; in this case we cheat and use the actual order used in the simulation, i.e., (2, 0, 2), where there is no integration component. The alternative function auto. arima is able to find the correct order in this example, but this is not the case in general. We also note that, despite the fairly long simulated horizon, different sample paths may lead to rather different estimates of the coefficients. This illustrates the difficulties in fitting time series models, but we should consider that the main purpose here is to capture some essential features of the original time series, in order to produce realistic scenarios for a Monte Carlo simulation. The code of Fig. 4.10 also uses the function ts.plot to plot the first sample path, along with a zoomed portion of the initial time interval; see the result in Fig. 4.11.

**FIGURE 4.11** **Sample path produced by arima.sim; the plot on the right zooms over the initial portion of the path.**

# 4.6 Subjective probability: The Bayesian view

So far, we have adopted a rather standard view of parameter estimation, as we have followed the orthodox approach: Parameters are unknown numbers, which we try to estimate by squeezing information out of a random sample, in the form of point estimators and confidence intervals. We also insisted on the fact that, given a computed confidence interval with some confidence level (1 − α), we cannot say that the true parameter is contained there with probability (1 − α). This statement makes no sense, since we are only comparing known numbers (the realized bounds of the confidence interval) and an unknown number (the parameter), but no random variable is involved. So, there is no such a thing as a "probabilistic knowledge" about parameters, and data are the only source of information; any other knowledge, objective or subjective, is disregarded. The following example illustrates the potential difficulties induced by this view.10

###   Example 4.6 A pathological case

Let _X_ be a uniformly distributed random variable, and let us assume that we do not know where the support of its distribution is located, but we know that its width is 1. Then, _X_ ~ U(μ − 0.5, μ + 0.5), where μ is the unknown expected value of _X_ , as well as the midpoint of the support. In order to estimate μ, let us take a sample of only _n_ = 2 independent realizations _X_ 1 and _X_ 2 of the random variable. Now, let us consider the order statistics

and the confidence interval

(4.10)

Given this way of building a confidence interval, what is the confidence level of _I_ , i.e., the probability P{μ   _I_ }? Once again, note that we are _not_ asking this question by referring to a specific interval after sampling, but to a random interval before sampling. Because of symmetry, the two observations have a probability 0.5 of falling to the left or to the right of μ. The confidence interval will not contain μ if both of them fall on the same half of the support. Then, since _X_ 1 and _X_ 2 are independent, we have

Therefore, the confidence level for   is the complement of this probability, i.e., 50%. Now, let us suppose that we observe _X_ 1 = 0 and _X_ 2 = 0.6. What is the probability that μ is included in the confidence interval   resulting from Eq. (4.10), i.e., P{0 ≤ μ, ≤ 0.6}? In general, this question does not make any sense, since μ is a number. But in this specific case, we have some additional knowledge leading to the conclusion that the expected value is certainly included in the interval [0, 0.6]. In fact, if _X_ (1) = 0, we may conclude μ ≤ 0.5; by the same token, if X(2) = 0.6, we may conclude μ ≤ 0.1. Since the confidence interval   includes the interval [0.1, 0.5], we would have good reasons to claim something like P{0 ≤ μ ≤ 0.6} = 1. But again, this makes no sense in the orthodox framework. By a similar token, if we get _X_ 1 = 0 and _X_ 2 = 0.001, we would be tempted to say that such a small interval is quite unlikely to include μ. However, within the framework of orthodox statistics, there is no way in which we can express this intuition properly.

On one hand, the example illustrates the need to make our background knowledge explicit: The Bayesian framework is an alternative to the orthodox approach, where it can be argued that unconditional probabilities do not exist, in the sense that probabilities are always conditional on some background knowledge and assumptions. On the other hand, we see the need of a way to express subjective views, which may be revised after collecting empirical data. Bayesian estimation has been proposed to cope with such issues as well.

## 4.6.1 BAYESIAN ESTIMATION

Consider the problem of estimating a parameter θ, which characterizes the probability distribution of a random available _X_. We have some prior information about θ that we would like to express in a sensible way. We might assume that the unknown parameter lies anywhere in the unit interval [0, 1], or we might assume that it is close to some number μ, but we are somewhat uncertain about it. Such a knowledge or subjective view may be expressed by a probability density _p_ (θ), which is called the _prior_ distribution of θ. In the first case, we might associate a uniform distribution with θ; in the second case the prior could be a normal distribution with expected value μ and variance σ2. Note that this is the variance that we associate with the parameter, which is itself a random variable, rather than a number, and not the variance of the random variable _X_ itself.

In Bayesian estimation, the prior is merged with experimental evidence by using Bayes' theorem. Experimental evidence consists of independent observations _X_ 1,..., _X_ _n_ from the unknown distribution. Here and in the following we mostly assume that random variable _X_ is continuous, and we speak of densities; the case of discrete random variables and probability mass functions is similar. We also assume that the values of the parameter θ are not restricted to a discrete set, so that the prior is a density as well. Hence, let us denote the density of _X_ by _f_ ( _x_ | θ), to emphasize its dependence on parameter θ. Since a random sample consists of a sequence of independent random variables, their joint distribution, _conditional_ on θ, is

The conditional density _f_ _n_ ( _x_ 1,..., _x_ _n_ | θ) is also called the _likelihood function_ , as it is related to the likelihood of observing the data values _x_ 1,..., _x_ _n_ , given the value of the parameter θ; also notice the similarity with the likelihood function in maximum-likelihood estimation.11 Note that what really matters here is that the observed random variables _X_ 1,..., _X_ _n_ are independent _conditionally_ on θ. Since we are speaking about _n_ \+ 1 random variables, we could also consider the joint density

but this will not be really necessary for what follows. Given the joint conditional distribution _f_ _n_ ( _x_ 1,..., _x_ _n_ | θ) and the prior _p_ (θ), we can find the marginal density of _X_ 1,..., _X_ _n_ by applying the total probability theorem:

where we integrate over the domain Ω on which θ is defined, i.e., the support of the prior distribution. Now what we need is to invert conditioning, i.e., we would like the distribution of θ conditional on the observed values _X_ _i_ = _x_ _i_ , _i_ = 1,..., _n_ , i.e.,

This _posterior_ density should merge the prior and the density of observed data conditional on the parameter. This is obtained by applying Bayes' theorem to densities, which yields

(4.11)

Note that the posterior density involves a denominator term _g_ _n_ ( _x_ 1,..., _x_ _n_ ) that does not depend on θ. Its role is only to normalize the posterior distribution, so that its integral is 1. Sometimes, it might be convenient to rewrite Eq. (4.11) as

(4.12)

where the symbol ∝ means "proportional to." The denominator term does not affect the _shape_ of the posterior, which encodes our uncertainty about the parameter θ, and it may be argued that it is not really essential.12 In plain English, Eq. (4.11) states that the posterior is _proportional_ to the product of the likelihood function _f_ _n_ ( _x_ 1,..., _x_ _n_ | θ) and the prior distribution _p_ (θ):

What we are saying is that, given some prior knowledge about the parameter and the distribution of observations conditional on the parameter, we obtain an updated distribution of the parameter, conditional on the actually observed data.

## 4.6.2 BAYESIAN LEARNING AND COIN FLIPPING

We tend to take for granted that coins are fair, and that the probability of getting heads is  . Let us consider flipping a possibly unfair coin, with an unknown probability θ of getting heads. In order to learn this unknown value, we flip the coin repeatedly, i.e., we run a sequence of independent Bernoulli trials with unknown parameter θ. If we do not know anything about the coin, we might just assume a uniform prior

If we flip the coin _N_ times, we know that the probability _f_ _N_ ( _H_ | θ) of getting _H_ heads, conditional on θ, is related to the binomial probability distribution:

(4.13)

This is our likelihood function. If we regard this expression as the probability of observing _H_ heads, given θ, this should actually be the probability mass function of a binomial variable with parameters θ and _N_ , but we are disregarding the binomial coefficient, which does not depend on θ and just normalizes the distribution. If we multiply this likelihood function by the prior, which is just 1, we obtain the posterior density for θ, given the number of observed heads:

(4.14)

Equations (4.13) and (4.14) look like the same thing, because we use a uniform prior, but they are very different in nature. Equation (4.14) gives the posterior density of θ, conditional on the fact that we observed _H_ heads and _N_ − _H_ tails. If we look at it this way, we recognize the shape of a beta distribution, which is the density of a continuous random variable, the parameter θ, rather than the mass function of a discrete random variable, the number of heads _H_ out of _N_ flips. To normalize the posterior, we should multiply it by the appropriate value of the beta function.13 Again, this normalization factor does not depend on θ and can be disregarded.

The R code in Fig. 4.12 generates a sequence of 1000 unfair coin flips, with θ = 0.2, and successively updates and plots the posterior density, resulting in the plots of Fig. 4.13. There, we display posterior densities normalized in such a way that their maximum is 1, after flipping the coin _N_ times and having observed _H_ heads. In Fig. 4.13(a) we just see the uniform prior before any flip of the coin. In the simulated history, the first flip lands heads. After observing the first heads, we know for sure that θ ≠ 0; indeed, if θ were zero, we could not observe any heads. The posterior is now proportional to a triangle:

**FIGURE 4.12** **Sampling a sequence of coin flips and updating the posterior.**

**FIGURE 4.13** **Updating the posterior density in coin flipping.**

This triangle is shown in Fig. 4.13(b). We observe another heads in the second flip, and so the updated posterior density is a portion of a parabola, as shown in Fig. 4.13(c):

With respect to the triangle, the portion of parabola is more tilted toward θ = 1, because we observed two heads out of two launches. However, when we get tails at the third flip, we rule out θ = 1 as well. Proceeding this way, we get beta distributions concentrating around the true (unknown) value of θ = 0.2.

Armed with the posterior density, how can we build a Bayes' estimator for θ? Figure 4.13 would suggest taking the mode of the posterior, which would spare us the work of normalizing it. However, this need not be the most sensible choice. If we consider the expected value for the posterior distribution, we find

(4.15)

There are different ways of framing the problem, which are a bit beyond the scope of this book,14 but one thing that we can immediately appreciate is the challenge we face. The estimator above involves what looks like an intimidating integral, but our task is even more difficult in practice, because finding the posterior density may be a challenging computational exercise as well. In fact, given a prior, there is no general way of finding a closed-form posterior; things can really get awkward when _multiple_ parameters are involved. Moreover, there is no guarantee that the posterior distribution _p_ _n_ (θ | _x_ 1,..., _x_ _n_ ) will belong to the same family as the prior _p_ (θ). In general, numerical methods are needed in Bayesian computational statistics and, needless to say, Monte Carlo approaches play a key role here. However, there are some exceptions. A family of distributions is called a _conjugate family of priors_ if, whenever the prior is in the family, the posterior is too. For instance, if we choose a beta density as the prior in coin flipping, the posterior is a beta density as well. The following example illustrates the idea further.

###   Example 4.7 The case of a normal prior

Consider a sample ( _X_ 1,..., _X_ _n_ ) from a normal distribution with unknown expected value θ and known variance σ02. Then, given our knowledge about the multivariate normal, and taking advantage of independence among observations, we have the following likelihood function:

Let us assume that the prior distribution of θ is normal, too, with expected value μ and σ:

To find the posterior, we may simplify our work by considering in each function only the part that involves θ, wrapping the rest within a proportionality constant. In more detail, the likelihood function can be written as

(4.16)

Then, we may simplify the expression further, by observing that

where   is the average of _x_ _i_ , _i_ = 1,..., _n_. Then, we may include terms not depending on θ into the proportionality constant and rewrite Eq. (4.16) as

(4.17)

By a similar token, we may rewrite the prior as

(4.18)

By multiplying Eqs. (4.17) and (4.18), we obtain the posterior

(4.19)

Again, we should try to include θ within one term; to this aim, we use a bit of tedious algebra and rewrite the argument of the exponential as follows:

where

(4.20)

(4.21)

Finally, this leads us to

(4.22)

Disregarding the normalization constant, we immediately recognize the familiar shape of a normal density, with expected value ν and variance ξ2. Then, given an observed sample mean   and a prior μ, Eq. (4.20) tells us that the Bayes' estimator of θ can be written as

(4.23)

Equation (4.23) has a particularly nice and intuitive interpretation: The posterior estimate is a weighted average of the sample mean   (the new evidence) and the prior μ, with weights that are inversely proportional to σ20/ _n_ , the variance of sample mean, and σ2, the variance of the prior. The more reliable a term, the larger its weight in the average.

What may sound a little weird in the example is the assumption that the variance σ02 of _X_ is known, but its expected value is not. Of course, one can extend the framework to cope with estimation of multiple parameters. Furthermore, we have considered two lucky cases, coin flipping and the normal prior for a normal distribution, where the posterior is very easy to compute as it preserves the original shape of the prior. In such cases we speak of conjugate priors. In other cases, we are not so lucky and we need to compute a possibly tough integral to find the normalization constant making the posterior a density. Indeed, Bayesian approaches were not quite practical until computational tools were devised to solve the difficulty by Markov chain Monte Carlo methods. These strategies are described in Chapter 14 and allow to run a simulation even if we do not know a density exactly, but only a likelihood proportional to the density.

# For further reading

  From the simulation practitioner's viewpoint, an excellent introduction to input analysis can be found in [10].

  If you are interested in a sound treatment of the theoretical background of inferential statistics and parameter estimation, [2] is recommended.

  We have given an extremely cursory and superficial look at linear regression models and their estimation. If you would like a treatment of linear regression from the statistical point of view, one possibility is [5].

  There are many books on econometrics, covering regression models and more, such as [1] and, at a more advanced level, [7].

  The above references on econometrics also cover time series models. If you want an extensive treatment, including estimation of time series models, you may refer to [8].

  A quite readable introduction to Bayesian estimation can be found in [9], which is replete with good R code as well.

# References

1 C. Brooks. _Introductory Econometrics for Finance_ (2nd ed.). Cambridge University Press, Cambridge, 2008.

2 G. Casella and R.L. Berger. _Statistical Inference_ (2nd ed.). Duxbury, Pacific Grove, CA, 2002.

3 M.H. DeGroot. _Probability and Statistics._ Addison-Wesley, Reading, MA, 1975.

4 M.H. DeGroot and M.J. Schervish. _Probability and Statistics_ (3rd ed.). Addison-Wesley, Boston, 2002.

5 N.R. Draper and H. Smith. _Applied Regression Analysis_ (3rd ed.). Wiley, New York, 1998.

6 I. Gilboa. _Theory of Decision under Uncertainty._ Cambridge University Press, New York, 2009.

7 W.H. Greene. _Econometric Analysis_ (7th ed.). Prentice Hall, Upper Saddle River, NJ, 2011.

8 J.D. Hamilton. _Time Series Analysis._ Princeton University Press, Princeton, NJ, 1994.

9 J.K. Kruschke. _Doing Bayesian Data Analysis: A Tutorial with R and BUGS._ Academic Press, Burlington, MA, 2011.

10 A.M. Law and W.D. Kelton. _Simulation Modeling and Analysis_ (3rd ed.). McGraw-Hill, New York, 1999.

1We defer some topics, such as the estimation of probabilities and quantiles to Chapter 7, as from a practical point of view they are more relevant there, even though, from a conceptual point of view, they are also related to the content of this chapter.

2Similar considerations apply to the previous chapter, of course.

3We illustrate the kind of mistakes that we are led to when forgetting the role of independence in Chapter 7.

4If we want to associate parameters with probabilistic statements, we have to take a Bayesian view; see Section 4.6.1.

5The coefficient of variation is the same in both cases. This is defined as σ/ |μ|, i.e., the ratio between standard deviation and the absolute value of the expected value.

6In fact, what is typically written in textbooks is a hybrid of different viewpoints about the nature of hypothesis testing, and this is why the terminology may sometimes sound weird.

7 See [4, p. 375] for a proof.

8 A similar approach is used in Chapter 9 to assess the star discrepancy of a numerical sequence.

9 For an application of polynomial regression to pricing American-style options by Monte Carlo methods, see Section 11.3.

10 This example is taken from [6, p. 45], which in turn refers back to [3].

11 See Section 4.2.3.

12This will play a major role in Chapter 14 on Markov chain Monte Carlo.

13See Section 3.2.4.

14One way to frame the problem is by introducing a _loss function_ that accounts for the cost of a wrong estimate. It can be shown that Eq. (4.15) yields the optimal estimator when the loss function has a certain quadratic form.

# Part Three

# Sampling and Path Generation

# Chapter Five

# Random Variate Generation

Armed with a suitable model of uncertainty, possibly one of those described in Chapter 3, whose parameters have been estimated as illustrated in Chapter 4, we are ready to run a set of Monte Carlo experiments. To do so, we must feed our simulation program with a stream of random variates mimicking uncertainty. Actually, the job has already been done, at least for R users trusting the rich library of random generators that is available. So, why should we bother with the internal working of these generators and learn how random variates are generated? Indeed, we will not dig too deep in sophisticated algorithms, and we will also refrain from treating generators for a too wide class of probability distributions. Still, there are quite good reasons to get a glimpse of the underpinnings of random variate generation. The aims of this chapter are:

  To gain an understanding of the different options that R offers: for instance, there are alternative generators that can be set and controlled with the RNGkind function.

  To understand how to manage seeds of random generators, which is essential for debugging and controlling experiments.

  To have an idea of what may go wrong with bad generators that have been proposed in the past and may still be around.

  To pave the way for some variance reduction strategies, to be described in Chapter 8.

  To better understand the link between Monte Carlo simulation and integration, which in turn is essential to apply alternative approaches based on the low-discrepancy sequences dealt with in Chapter 9.

  A bit of knowledge about random variate generation by acceptance–rejection is also useful to better appreciate certain methods for computational Bayesian statistics, like Markov chain Monte Carlo; see Chapter 14.

The overall structure of a Monte Carlo simulation is described in Section 5.1. Essentially, a Monte Carlo simulator is a function mapping a stream of (pseudo)-random numbers, i.e., a sequence of i.i.d. variables with uniform distribution on the unit interval (0, 1), into a set of estimates of interest. The generation of random numbers is described in Section 5.2. Then, random numbers are transformed in order to create a sample from another, more interesting, probability distribution. We describe a few standard techniques for random variate generation, like the inverse transform method (Section 5.3) and the acceptance–rejection method (Section 5.4). For certain specific distributions, it may be preferable to apply ad hoc methods. The ubiquitous normal distribution is dealt with in Section 5.5, where we cover both the univariate and the multivariate case; a few more ad hoc methods are outlined in Section 5.6. Finally, in Section 5.7 we discuss how to generate jointly distributed variables whose dependence structure is described by a copula. We defer to Chapter 6 the task of generating sample paths of stochastic processes, typically described by stochastic differential equations, since they involve additional issues about the discretization of continuous-time models.

**A note on terminology.** As we have already hinted at, there is nothing random in a Monte Carlo simulation carried out in R, as it typically involves the generation of pseudorandom numbers, rather than genuinely random ones. We will occasionally stress this point, but, more often than not, we will drop the "pseudo" for the sake of brevity. Indeed, in both output analysis and variance reduction methods, we behave as we were truly dealing with random variates. We will use the term random _variate_ in general but random _number_ to refer to uniform variates only, as they are actually the primary input of a Monte Carlo simulation. Later, we will mostly avoid the term _quasi–random_ numbers when referring to deterministic low-discrepancy sequences that are the subject of Chapter 9, as this name is misleading. Of course, pseudorandom numbers are by necessity deterministic as well, but low-discrepancy sequences are _conceived_ as deterministic.

# 5.1 The structure of a Monte Carlo simulation

A typical Monte Carlo simulation can be represented as the sequence of building blocks depicted in Fig. 5.1.

**FIGURE 5.1** **Structure of a Monte Carlo simulation.**

**1.** The starting point is a stream of random numbers { _U_ _i_ }, i.e., a sequence of i.i.d. variables uniformly distributed on the unit interval (0, 1). Clearly, unless we resort to not quite practical physical random number generators, we must rely on an algorithm that actually generates _pseudo_ -random numbers. The reasons for doing so are:

  Efficiency: To run a Monte Carlo simulation we may need a huge set of random numbers, and an efficient computer algorithm is the only way to achieve that, as physical generators are slower.

  Repeatability: We may need to repeat the experiment, and program debugging would turn into a nightmare in the case of true randomness. In principle, we could generate random numbers by a physical device and then store them, but this would not be practical for very extensive Monte Carlo runs.

**2.** We do have some good strategies to emulate randomness, which are limited to uniform variates. Hence, even if we need, say, normal random variables, we have to use uniform random numbers as the primary input and then resort to some transformation to obtain more relevant distributions. In the second step, using an array of such methods, we transform the stream of uniform random numbers into a sequence of random variates with the desired distribution, resulting in a stream { _X_ _j_ }. We use a different subscript, _j_ rather than _i_ , to stress the fact that more than one random number might be needed to generate one realization of the random variate of interest, as is the case with acceptance–rejection strategies described in the following. Furthermore, the random variate we need may be multidimensional, as in the case of a multivariate normal.

**3.** The third step is the actual simulation model, where domain-specific dynamics are represented and used to generate a sample path. It may be the case that a sample path consists of the draw of a single random variate, but in general we may need to generate a whole sample path of a possibly complicated stochastic process. Unlike the previous steps, this is highly problem dependent. For instance, we may generate a sample path of stock prices or interest rates and, finally, the payoff of an exotic option.

**4.** In the fourth and last step, all of the outcomes are collected in order to come up with an estimate, typically in the form of both a point estimate and a confidence interval. It is important, when applying standard inferential statistics methods, that the outcomes form a sample of i.i.d. random variables (see Chapter 7).

Now we may grasp the link between multidimensional integration and Monte Carlo simulation: The structure of Fig. 5.1 points out quite clearly that a typical Monte Carlo simulation is, at least conceptually, a function mapping a vector of _n_ uniform random numbers into a numeric output. More formally, it is a function _g_ : (0, 1) _n_ →  , where the domain is the _n_ -fold Cartesian product of the unit interval,

i.e., the _unit hypercube._ When estimating an expected value, we are calculating the following integral:

(5.1)

Actually, this view is valid whenever we know exactly the length of the input required to run one Monte Carlo replication, i.e., how many random numbers are needed to get one observation of the output random variable of interest. If so, we know the dimension of the space on which we are integrating. In other cases, as we shall see, we may not really know _n._ Then, the interpretation as an integral is still valid, but we cannot apply the methods of Chapter 9, which are based on _deterministic_ low-discrepancy sequences of numbers replacing the stream of random numbers. This approach requires an integration over a domain of given dimension, and this in turn enforces a few requirements on how the block sequence of Fig. 5.1 is implemented in detail.

# 5.2 Generating pseudorandom numbers

A pseudorandom number generator is a tool to generate a stream of numbers on the unit interval. To be more precise, some generators produce numbers in the [0, 1) interval, i.e., 0 is a possible output, but 1 is not. For some purposes, the value 0 can be troublesome; this happens, e.g., if we have to transform the stream of random numbers by taking a logarithm. In such cases, numbers in the open interval (0, 1) are preferred. In both cases, we will use the notation _U_ ~ U(0, 1) to refer to the uniform distribution on the unit interval.

There is a rich literature on the generation of random numbers, which is often technically challenging. Fortunately, the average user need not be an expert, but some background knowledge is useful in order to use the available generators properly. In Section 5.2.1 we describe the basic textbook approach, i.e., linear congruential generators, and we point out some of their limitations in Section 5.2.2. Then, we outline a fairly general structure for random number generators in Section 5.2.3, and finally, in Section 5.2.4, we illustrate the principles that are the basis of some generators available in R.

## 5.2.1 LINEAR CONGRUENTIAL GENERATORS

The standard textbook method to generate U(0, 1) random numbers is the _linear congruential generator_ (LCG). This approach was actually implemented in many generators included in commercial software, and it is still possible to choose an LCG. However, LCGs are not considered state-of-the-art anymore and should not be used for high-quality Monte Carlo experiments, especially if these are rather extensive and require a huge stream of input numbers. Nevertheless, LCGs are most useful as an introduction to the structure of more sophisticated generators and illustrate well the pitfalls of pseudorandom number generation.

An LCG actually generates a sequence of non-negative integer numbers _Z_ _i_ , which are transformed into uniform random numbers. The sequence of integer numbers starts from a user-selected _seed Z_ 0; then, given the previous integer number _Z_ _i_ −1, the next number in the sequence is generated as follows:

(5.2)

where _a_ (the multiplier), _c_ (the shift), and _m_ (the modulus) are properly chosen integer numbers and "mod" denotes the remainder of integer division (e.g., 15 mod 6 = 3). Finally, the integer number _Z_ _i_ is transformed into a uniform number

(5.3)

Since _Z_ _i_ ranges in the set {0, 1, 2, 3,..., _m_ − 1}, _U_ _i_ ranges in the unit interval [0, 1). Note that 0 is a possible output, but 1 is not. In Fig. 5.2 we display a function implementing an LCG. The function, given a choice of the parameters a, c, and m, as well as a seed, yields a list consisting of two vectors of length N, where USeq contains the uniform random numbers and ZSeq contains the integer numbers.

**FIGURE 5.2** **Function to generate random numbers by a linear congruential generator.**

In order to check the function LCG and learn a couple of fundamental points, we may run the script of Fig. 5.3 and get the following output:

**FIGURE 5.3** **Script to check the linear congruential generator.**

    1 6 0.3750
    2 1 0.0625
    3 8 0.5000
    4 11 0.6875
    5 10 0.6250
    6 5 0.3125
    7 12 0.7500
    8 15 0.9375
    9 14 0.8750
    10 9 0.5625
    11 0 0.0000
    12 3 0.1875
    13 2 0.1250
    14 13 0.8125
    15 4 0.2500
    16 7 0.4375
    17 6 0.3750
    18 1 0.0625
    19 8 0.5000
    20 11 0.6875

On the basis of this simple run, we may observe what follows:

  It is clear that there is nothing random in the sequence generated by an LCG. The sequence is built from an initial number _Z_ 0, the seed of the sequence, which is the initial state of the generator. By starting the sequence from the same seed, we will always get the same sequence. We see quite clearly why we should speak of _pseudo_ random numbers. We may also check this point in R, which generates random numbers with the function runif. Actually, runif does not use an LCG (by default), but we may control the seed (better said, the initial state of the generator) as follows:

    > set.seed(55555)
    > runif (5)
    > [1] 0.9598632 0.6586607 0.3076999 0.9489697 0.1188039
    > runif (5)
    [1] 0.3415344 0.0415116 0.4067968 0.10739569 0.6403764
    > set.seed(55555)
    > runif (5)
    [1] 0.9598632 0.6586607 0.3076999 0.9489697 0.1188039

The function set.seed receives an integer number and sets the initial state of the generator accordingly; we will see later what this state actually is when using more sophisticated generators.

  Since we take a ratio of integer numbers, an LCG actually generates rational numbers in the set

rather than real numbers. This may look like a significant limitation, but on second thought we should realize that whatever we do on a computer is based on rational numbers, as floating-point numbers are represented, and possibly truncated, by a finite number of bits. In practice, this is not a serious problem, provided that the modulus _m_ is large enough.

  A less obvious point is that the generator is necessarily _periodic_. As we have already observed, we may generate at most _m_ distinct integer numbers _Z_ _i_ in the range from 0 to _m_ − 1, and whenever we repeat a previously generated number, the sequence will be repeated as well (which is not very random at all!). A careful look at the previous output shows that, having started with the initial seed _Z_ 0 = 7, we get back there after 16 steps. Of course, an LCG with a period of 16 is not acceptable for any practical purpose; yet, this is not too bad, as 16 is the maximum possible period for a modulus _m_ = 16. We do much worse if we select _a_ = 11, _c_ = 5, and _m_ = 16. In this case, starting from _Z_ 0 = 3, we get the following sequence of integer numbers _Z_ _i_ :

which has half the maximal period. Thus, since the maximum possible period is _m_ , we should choose a large modulus in order to have a large period. Indeed, a common choice for generators proposed in the literature was

This may look like a very large period, but with faster and faster hardware we run longer and longer simulations, and the above period may not be good enough.

The last observation has been one of the reasons behind the development of alternative generators. Nevertheless, a proper choice of _a_ and _c_ ensures that the period is maximized and that the sequence looks random enough. For instance, a generator proposed in [4] features the following parameters:

This LCG has zero shift, which is a rather common choice, as _c_ does not provide too many advantages. Hence, we speak of a _multiplicative_ generator; it is also clear that in such a case we must steer away from zero, as the LCG would get stuck there. But what is "random enough" exactly? There are quite sophisticated testing procedures to check random number generators, which are beyond the scope of this book. Anyway, we point out a few important issues in the next section.

## 5.2.2 DESIRABLE PROPERTIES OF RANDOM NUMBER GENERATORS

The first requirement for a good uniform random number generator is that it is, rather unsurprisingly, _uniform._ A quick check of uniformity can be run by evaluating mean and variance of sample sequences. Since we know that, for a random variable _U_ ~ U(0, 1), we have

a comparison is easy to make. A less crude test would check if, by dividing the unit interval into equally spaced bins, the same number of observations falls into each one of them. In other words, we may draw a histogram or run something like a goodness-of-fit test. However, this is not quite enough to tell a good random number generator. To see the point, consider a sequence like

which is obtained by an LCG with _a_ = _c_ = 1. This is certainly uniform and has a maximum period. However, we want i.i.d. variables, where now we stress independence. From this point of view, the choice _a_ = 1 is a very poor one in general, as we get a sequence like

featuring long increasing subsequences.

Given the structure of LCGs, there is no hope of getting a genuinely independent stream, of course; yet, a good LCG should be able to trick statistical testing procedures into "believing" that they produce a sequence of independent observations from the uniform distribution. Without considering sophisticated tests, we can investigate how short sequences of random numbers are distributed in bidimensional or tridimensional spaces. More precisely, imagine that we draw points of the form

on the plane. These points, if the sequence is independent, should be uniformly distributed on the unit square. By the same token, points of the form

should be uniformly distributed on the unit cube. As an illustration, let us try the script in Fig. 5.4.1 The first part of the script uses an LCG with _a_ = 65, _c_ = 1, and _m_ = 2048. We generate the whole sequence of 2048 points, corresponding to the maximum period, and plot points of the form ( _U_ _i_ , _U_ _i_ +1) in the upper half of Fig. 5.5. If we look at the plot, we observe a fairly good filling of the unit square, which may be interpreted as a sign of good quality. However, if we only draw the first quarter of the sequence, as we do next, a different pattern emerges. The bottom half of Fig. 5.5 shows that the initial portion of the sequence does not cover the unit square uniformly, pointing out a deficiency of the generator.

**FIGURE 5.4** **Script to illustrate the lattice structure of LCGs.**

**FIGURE 5.5** **Plots obtained by running the script ofFig. 5.4.**

The second part of the script, where we set _a_ = 1365, yields much worse results, however. In Fig. 5.6 we observe that points are arranged on parallel lines. This kind of pattern is known as _lattice structure_ and is typical of LCGs. Actually, this also happens in Fig. 5.5. In general, similar structures can be observed in three dimensions when considering points with coordinates of the form ( _U_ _i_ , _U_ _i_ +1, _U_ _i_ +2). The problem is more evident here, as very few lines with large gaps are observed.

**FIGURE 5.6** **Lattice structure in LCGs.**

Let us try to understand where the problem comes from, with reference to Fig. 5.6.2 A quick look at the figure suggests that points ( _U_ _i_ , _U_ _i_ +1) are arranged in lines of the form

(5.4)

for just a few values of the intercept α _j_. In order to check this, let us investigate what happens in terms of the corresponding integer numbers _Z_ _i_ and _Z_ _i_ +1, by noting that

implies

for some integer _k_. Hence, with the choice _a_ = 1365, _c_ = 1, and _m_ = 2048, we find

Now, if we divide by _m_ = 2048 in order to get back to the uniform numbers, we notice that 3 × 1365 + 1 = 4096, an integer multiple of the modulus. Indeed, this is what makes this case peculiar: the right-hand side of the previous equation, when divided by 2048, turns out to be the intercept α _j_ :

where β _j_ is integer. Therefore, we find just a few lines of the form of Eq. (5.4). To see how many, notice that, since _U_ _i_ , _U_ _i_ +1   [0, 1), we have

suggesting that at most four parallel lines may intersect the unit square. Indeed, a look at Fig. 5.6 shows that there are four such lines, but one seems to boil down to an isolated point close to the origin. Using R, it is easy to check that we get back to the initial integer seed _Z_ 0 = 0 after _Z_ 2047 = 3, since (3 × 1365 + 1) mod 2048 = 0. Therefore, we obtain

which corresponds to a line with β _j_ = 0, almost passing through the origin.

Having pointed out some issues with LCGs, we should also mention that they do have some advantages as well:

  They are efficient.

  It is easy to come up with portable generators, i.e., algorithms that can work on different machines.

  They allow for reproducibility by managing a simple seed.

  Sequences may be partitioned into separate streams.

The last point deserves some comment. By "separate streams" we mean different sequences of "independent" random numbers that can either be assigned to model different sources of randomness in a simulation3 or to different CPUs when running parallel Monte Carlo. We could just assign a different seed to each stream, but with a naive choice there is a danger that they overlap, impairing the independence requirement. With a multiplicative LCG, i.e., when the increment _c_ = 0, is easy to skip ahead in the sequence in order to partition the whole sequence in nonoverlapping subsequences:

So, skipping ahead by _k_ steps just requires computing _a_ _k_ mod _m_ once. All of these requirements should also be met by alternative and more sophisticated generators.

## 5.2.3 GENERAL STRUCTURE OF RANDOM NUMBER GENERATORS

In the previous section we have shown that LCGs may have several limitations, and that careful testing is needed in order to select a suitable one. Indeed, LCGs were state of the art in the past, but they have been replaced by alternative generators. Nevertheless, all of these generators share the same conceptual structure as LCGs, in which we have:

  A finite set _S_ of states

  A way to assign the initial state in _S_

  A state transition function from _S_ to _S_

  An output space _U_

  An output function from _S_ to _U_

In the simple LCG case, the states are just integer numbers and the output space is the unit interval. More recent generators use more complex state spaces. We cannot cover all of them, but we do point out a couple of ideas that have actually been implemented in R.

One possibility is to devise _combined generators_. The idea is to combine multiple generators that, taken individually, would have quite some weaknesses, but improve a lot when combined. One such generator is the Wichman–Hill generator, which combines three LCGs. For instance:

We see that _S_ consists of three integer numbers. The output function is

This choice results in much longer periods, and other generators along this line have been proposed. It should be noted that an advantage of a combined generator over an LCG with large modulus _m_ is that we avoid numerical issues with potential overflows.

An alternative approach is to use a richer state space _S_ , as in _multiple recursive generators_ :

A class of generators, called _Tausworthe generators_ or _linear feedback shift registers_ , implements this idea with _m_ = 2. In other words, the state consists of bits, a choice that takes advantage of the binary operations of a computer. The output function maps a sequence of bits into a number in the unit interval:

where _s_ and _L_ are positive integers playing the role of a step size and a word length, respectively. A natural choice for _L_ is 32 or 64, depending on the word length of the machine. The advantage of exploiting the binary arithmetic of hardware, where operations modulo 2 correspond to a XOR (exclusive OR) operation is obvious.4 Actually, variants of this scheme are used. _Generalized feedback shift registers_ (GFSR) use _k_ vectors **X** _i_ = ( _x_ _i_ ,1,..., _x_ _i,L_ )  of _L_ bits, rather than single bits:

(5.5)

Then, we may use an output function like

One very common generator, the _Mersenne twister_ , is obtained by adding a further tweak, i.e., by interpreting a GFSR in matrix terms:

where **Y** _i_ is a vector of _kL_ bits and **A** is a ( _kL_ ) × ( _kL_ ) matrix. In the case of the GFSR of Eq. (5.5), we have

where **I** _L_ is the _L_ × _L_ identity matrix. Clearly, the upper part of the matrix implements a shift, and the new bit word is generated by the last line. Twisting is obtained by using a different last line in matrix **A**. A version of Mersenne twister is widely used and, at the time of writing, it is the default generator in R. This version uses _k_ = 624 and has the huge period

Incidentally, the name stems from the concept of Mersenne numbers, i.e., numbers of the form 2 _p_ − 1; for certain values of _p_ , a Mersenne number is also a prime number. Indeed, both _p_ = 19,937 and the corresponding Mersenne number are prime.

## 5.2.4 RANDOM NUMBER GENERATORS IN R

Random numbers can be generated in R by the function runif, which generates a vector of given size:

    > set.seed(55555)
    > runif(5)
    [1] 0.9598632 0.6586607 0.3076999 0.9489697 0.1188039

Notice the use of set.seed to set the seed (better said, the state) of the generator. The function .Random.seed can be used to inspect the current state of the generator. Its value depends on the kind of generator, which can be selected by the function RNGkind by setting the input parameter kind; possible values of this parameter are wichmann–Hill and Mersenne–Twister, the default.

The following snapshot shows that, with the default setting, the state of the generator is an integer vector consisting of 626 entries, which can be reset by passing a single number to set.seed:

    > mode(.Random.seed)
    [1] "numeric"
    > class(.Random.seed)
    [1] "integer"
    > length(.Random.seed)
    [1] 626
    > set.seed(55555)
    > cat(.Random.seed[1:6],"\n")
    403 624 −734508337 397372100 1252553461 −911245518
    > runif(3)
    [1] 0.9598632 0.6586607 0.3076999
    > cat(.Random.seed[1:6],"\n")
    403 3 −1423130422 −543017428 −426382801 −1354865947
    > set.seed(55555)
    > cat(.Random.seed[1:6],"\n")
    403 624 −734508337 397372100 1252553461 −911245518

This corresponds to the state of a Mersenne twister, plus some additional information. The next snapshot shows how to change the generator from Mersenne twister to Wichmann–Hill, as well as the effect on the state:

    > RNGkind(kind="Mersenne–Twister")
    > set.seed(55555)
    > cat(.Random.seed[1:6],"\n")
    403 624 −734508337 397372100 1252553461 −911245518
    > RNGkind(kind="Wichmann-Hill")
    > set.seed(55555)
    > cat(.Random.seed,"\n")
    400 15266 22906 19508
    > RNGkind(kind="default")
    > set.seed(55555)
    > cat(.Random.seed[1:6],"\n")
    403 624 −734508337 397372100 1252553461 −911245518

Indeed, the state of a combined Wichmann-Hill generator consists of three integer numbers.

# 5.3 The inverse transform method

Suppose that we are given the CDF _F_ _X_ ( _x_ ) = _P_ { _X_ ≤ _x_ }, and that we want to generate random variates distributed according to _F_ _X_ (·). If we are able to invert _F_ _X_ (·) easily, then we may apply the following inverse transform method:

**1.** Draw a random number _U_ ~ U(0, 1)

**2.** Return _X_ = _F_ _X_ −1 ( _U_ )

It is easy to see that the random variate _X_ generated by this method is actually characterized by the CDF _F_ _X_ (·):

where we have used the monotonicity of _F_ _X_ and the fact that _U_ is uniformly distributed.5

###   Example 5.1 Generating exponential variates

A typical distribution which can be easily simulated by the inverse transform method is the exponential distribution. If _X_ ~ exp(μ), where 1/μ, is the expected value of _X_ , its CDF is

A direct application of the inverse transform method yields

Since the distributions of _U_ and (1 − _U_ ) are actually the same, it is customary to generate exponential variates by drawing a random number _U_ and by returning − log( _U_ )/μ. Generating exponential random variates is also one way to simulate a Poisson process, as we shall see in Chapter 6.

The inverse transform method is quite simple conceptually; however, we may not apply it when _F_ _X_ is not invertible, as it happens with discrete distributions. In this case the cumulative distribution function is piecewise constant, with jumps located where the probability mass is concentrated, i.e., for possible values of the discrete random variable. Nevertheless, we may adapt the method with a little thought. Consider a discrete empirical distribution with a finite support:

Then we should generate a uniform random variate _U_ and return _X_ as

It may be instructive to see how this idea may be implemented in a simple way (not the most efficient one, however). Suppose that we have a distribution defined by probabilities

for the values 1, 2, 3, 4, 5. First, we find the cumulative probabilities,

and then we draw a uniform random number, say _U_ = 0.82. For each cumulative probability _P_ , we check if _U_ > _P_ , which yields the vector

where 1 corresponds to "true" and 0 to "false." To select the correct value to return, we must sum the ones in this vector (the total is 3 here) and add 1; in this case we should return the value 4. Using R, this may be accomplished by working directly on vectors, as shown in Fig. 5.7. The code also includes a script to check the results by plotting a histogram, shown in Fig. 5.8, for the specific numerical example that we are considering.

**FIGURE 5.7** **Sampling from an empirical discrete distribution.**

**FIGURE 5.8** **Histogram produced by calling** EmpiricalDrnd.

Sometimes, the CDF is invertible in principle, but this is quite costly computationally. In such a case, one possibility is to resort to the acceptance–rejection method we describe next.

# 5.4 The acceptance–rejection method

Suppose that we must generate random variates according to a PDF _f_ _X_ ( _x_ ), and that the difficulty in inverting the corresponding CDF makes the inverse transform method unattractive. To see a concrete example, consider the beta distribution with PDF

for parameters α1, α2 > 1, where

is the beta function needed for normalization.6 To obtain the CDF we should integrate the PDF, resulting in a function that requires a costly numerical inversion by solving a nonlinear equation.

Assume that we can find a function _t_ ( _x_ ) such that

where _I_ is the support of _f_ _X_ (·). The function _t_ (·) is not a probability density, but the related function _g_ ( _x_ ) = _t_ ( _x_ )/ _c_ is, provided that we choose

If the distribution _g_ (·) is easy to sample from, it can be shown that the following acceptance–rejection method generates a random variate _X_ distributed according to the density _f_ _X_ (·):

**1.** Generate _Y_ ~ _g_.

**2.** Generate _U_ ~ U(0, 1), independent of _Y_.

**3.** If _U_ ≤ _f_ _X_ ( _Y_ )/ _t_ ( _Y_ ), return _X_ = _Y_ ; otherwise, repeat the procedure.

The PDF _f_ _X_ (·) is called the _target_ density, and the "easy" density _g_ (·) is called the _instrumental_ or _candidate_ density. If the support _I_ is bounded, a natural choice of instrumental density is the uniform distribution on _I_ , and we may set

Before proving the correctness of the method, let us get an intuitive grasp from Fig. 5.9, where we show a target PDF much like a beta distribution, whose support is the unit interval (0, 1). The upper bounding function _t_ (·) is a constant corresponding to the maximum of the target density. Hence, the instrumental density is uniform, and the variables _Y_ spread evenly over the unit interval. Consider point _A_ ; since _f_ _X_ ( _A_ ) is close to _t_ ( _A_ ), _A_ is likely to be accepted, as the ratio _f_ _X_ ( _A_ )/ _t_ ( _A_ ) is close to 1. When we consider point _B_ , where the value of the density _f_ _X_ (·) is small, we see that the ratio _f_ _X_ ( _B_ )/ _t_ ( _B_ ) is small; hence, _B_ is unlikely to be accepted, which is what we would expect. It can also be shown that the average number of iterations to terminate the procedure with an accepted value is _c_.

**FIGURE 5.9** **Graphical illustration of the acceptance–rejection method.**

###   Example 5.2 Generating a beta variate

Consider the density

The reader is urged to verify that this is indeed a density (actually, it is the beta density with α1 = α2 = 3). Since the PDF is a fourth-degree polynomial, the CDF is a fifth-degree polynomial. If we apply the inverse transform method, we have to invert such a polynomial at each generation; this inconvenience suggests the application of the acceptance–rejection method. By straightforward calculus we see that

for _x_ * = 0.5. We may check this result by running the first lines of the script in Fig. 5.10:

**FIGURE 5.10** **Script to check acceptance–rejection for Example 5.2.**

    > f <- function (x) dbeta(x,shape1=3,shape2=3)
    > out <- optimize(f,interval=c(0,1),maximum=TRUE)
    > out
    $maximum
    [1] 0.5
    $objective
    [1] 1.875

Using the uniform density as the easy instrumental density _g_ (·), we obtain the following algorithm:

**1.** Draw two independent and uniformly distributed random variables _U_ 1 and _U_ 2.

**2.** If _U_ 2 ≤ 16( _U_ 12 − 2 _U_ 13 \+ _U_ 14), accept _X_ = _U_ 1; otherwise, reject and go back to step 1.

In the script of Fig. 5.10 we also draw the points generated according to this scheme, with coordinates _U_ 1 and _U_ 2; this produces the plot in Fig. 5.11, where rejected points are depicted as empty circles, and accepted points as bullets. We observe that a larger fraction of points is accepted when the PDF is larger, as intuition requires. The average number of iterations to generate one random variate is  .

**FIGURE 5.11** **The plot produced by the R script ofFig. 5.10.**

Now, let us prove:

**1.** That the acceptance-rejection algorithm indeed is equivalent to sampling from the PDF _f_ _X_ (·).

**2.** That the expected number of iterations to accept _Y_ is _c_.

The plot of Fig. 5.11 is enlightening, as it shows that what we are doing is based on a common trick in mathematics: embedding a problem into a higher-dimensional space.7 In fact, our task is to generate a single random variable _X_ according to the PDF _f_ _X_ (·), but this density may also be interpreted as

i.e., as a marginal density within the joint distribution of two random variables _X_ and _U_ , umformly distributed on the set {( _x, u_ ) | 0 ≤ _u_ ≤ _f_ _X_ ( _x_ )}, i.e.,

Thus, sampling in the one-dimensional case is equivalent to sampling the bidimensional distribution. The argument applies to the case in which the upper bounding function _t_ (·) is a constant, as in Example 5.2, so that _g_ (·) boils down to a uniform density and the larger domain is just a box. To generalize the idea to an arbitrary instrumental density, let us calculate the probability  , for an arbitrary set8  , when _X_ is generated by the acceptance–rejection scheme:

  sample _Y_ ~ _g_ ;

  sample _U_ | _Y_ = _y_ ~ U{0, _cg_ ( _y_ ));

  accept _y_ if it satisfies the constraint _u_ ≤ _f_ _X_ ( _y_ ).

Note that _X_ is given by _Y_ conditional on the event "accept", which is equivalent to { _U_ ≤ _f_ _X_ ( _Y_ )}. Let _X_ be the support of _f_ _X_ (·). We can write:

This shows that the density of the accepted _Y_ is indeed _f_ _X_ (·). Furthermore, we also see that the probability of the event "accept" is

However, since the iterations in acceptance–rejection are independent and we stop at the first success, the number of iterations follows a geometric distribution with parameter _p_ = 1/ _c_. Since a geometric random variable with parameter _p_ has expected value 1/ _p_ ,9 it follows that the expected number of iterations to accept a proposed value is _c_.

We also observe that:

**1.** A tight upper bounding function, which is reflected in a small value of _c_ , is needed for efficiency.

**2.** A priori, we do not know how many random numbers are needed to generate a single variate by acceptance–rejection.

The second point implies that we do not know a priori the dimension of space on which we are integrating when carrying out Monte Carlo simulation. This does not prevent the application of Monte Carlo sampling, but it is a stumbling block when we use low-discrepancy sequences, as we shall see in Chapter 9. It is also worth mentioning that rejection sampling is not restricted to univariate distributions, and it is can also be applied to sample from a PDF which is not completely known and is given up to a normalization constant. This is quite relevant in computational Bayesian statistics, as we shall see in Chapter 14.

# 5.5 Generating normal variates

Sampling from a normal distribution is a common need in financial applications. Recall first that if _X_ ~ N(0, 1), then μ + σ _X_ ~ N(μ, σ2); hence, we just need a method for generating a stream of independent standard normal variables. If correlation needs to be induced, this can be easily accomplished as we shall explain. Over the years, a number of methods have been proposed to generate standard normal variates. Some are obsolete, like the idea of summing a suitably large number of uniform variates, taking advantage of the central limit theorem. The inverse transform method was not considered efficient in the past, but it is quite practical by now, as efficient approximations have been developed to invert the CDF of the standard normal. In between, there is a set of ad hoc methods, and some are implemented in the R function rnorm.

## 5.5.1 SAMPLING THE STANDARD NORMAL DISTRIBUTION

The general-purpose inverse transform methods may be applied by inverting the standard normal CDF. In R, this is accomplished, for a generic normal, by the function call

    x <- qnorm(p, mu, sigma)

that returns the quantile at probability level p of a variable with expected value mu and standard deviation sigma. The following snapshot shows that there is little difference between calling rnorm or generating uniform variates and then inverting the normal CDF:

    > system.time(qnorm(runif(50000000) ) )
    user system elapsed
    5.74 0.16 5.89
    > system.time(rnorm(50000000))
    user system elapsed
    5.80 0.05 5.85

Indeed, inversion of the CDF is currently the default method in R. Nevertheless, there are faster _ad hoc_ methods that can be selected as follows:

    > set.seed(55555,normal.kind="Ahrens-Dieter")
    > system.time(rnorm(50000000))
    user system elapsed
    3.43 0.05 3.48

In the following, we will not cover the generation of normal variates too extensively, but it is nevertheless instructive to get a glimpse of the ingenuity that may be employed to come up with ad hoc methods, and the dangers of bad random variate generators. A well-known approach is the Box–Muller method, which takes advantage of the representation of points on the plane by polar coordinates. Consider two independent variables _X, Y_ ~ N(0, 1), and let ( _R_ , θ) be the polar coordinates of the point of Cartesian coordinates ( _X, Y_ ) in the plane, so that

Given independence, the joint density of _X_ and _Y_ is

The last expression looks like a product of an exponential density for _d_ and a uniform distribution; the term 1/2π may be interpreted as the uniform distribution for the angle θ   (0, 2π). However, we are missing some constant term in order to obtain the exponential density. In fact, to properly express the density in terms of (d, θ), we should take the Jacobian of the transformation from ( _x, y_ ) to ( _d_ , θ) into account.10 Some calculations yield

and the correct density in the alternative coordinates is

Hence, we may generate _R_ 2 as an exponential variable with mean 2, θ as a uniformly distributed angle, and then transform back into Cartesian coordinates to obtain two independent standard normal variates. The Box–Muller algorithm may be implemented as follows:

**1.** Generate two independent uniform variates _U_ 1, _U_ 2 ~ U(0, 1).

**2.** Set _R_ 2 = −2log _U_ 1 and θ = 2π _U_ 2.

**3.** Return _X_ = _R_ cos θ, _Y_ = _R_ sin θ.

In practice, this algorithm may be improved by avoiding the costly evaluation of trigonometric functions and integrating the Box–Muller approach with the rejection approach. The idea results in the following polar rejection method:

**1.** Generate two independent uniform variates _U_ 1, _U_ 2 ~ U(0, 1).

**2.** Set _V_ 1 = 2 _U_ 1 − 1, _V_ 2 = 2 _U_ 2 − 1, _S_ = _V_ 12 \+ _V_ 22.

**3.** If _S_ > 1, return to step 1; otherwise, return the independent standard normal variates:

We refer the reader to [9, Section 5.3] for a justification of the polar rejection method.

###   Example 5.3 A bad interaction of strategies

We have seen that LCGs may exhibit a lattice structure. Since the Box–Muller transformation is nonlinear, one might wonder if the composition of these two features may yield weird effects. We may check this in a somewhat peculiar case (see [7]), using the R script in Fig. 5.12. The script generates 2046 uniform random numbers for a sequence with modulus _m_ = 2048; we discard the last pair, because the generator has maximum period and reverts back to the seed, which is 0 and causes trouble with the logarithm. Vectors U1 and U2 contain odd- and even-numbered random numbers in the sequence. The first part of the resulting plot, displayed in Fig. 5.13, shows poor coverage of the plane. The second part shows that swapping the pairs of random numbers may have a significant effect, whereas with truly random numbers the swap should be irrelevant. Of course, using better LCGs, or better random number generators altogether prevents pathological behavior like this. Nevertheless, it may be preferable to use the inverse transform method to avoid undesirable effects.

**FIGURE 5.12** **Script to check the Box–Muller approach.**

**FIGURE 5.13** **Effect of swapping random numbers in the Box–Muller transformation.**

## 5.5.2 SAMPLING A MULTIVARIATE NORMAL DISTRIBUTION

In many financial applications one has to generate variates according to a multivariate normal distribution with (vector) expected value μ and covariance matrix Σ. This task may be accomplished by finding the Cholesky factor for Σ, i.e., an upper triangular11 matrix **U** such that   Then we apply the following algorithm:

**1.** Generate _n_ independent standard normal variates _Z_ 1,..., _Z_ _n_ ~ N(0, 1).

**2.** Return  , where  .

###   Example 5.4 Sampling using Cholesky factors

A rough code to simulate multivariate normal variables is illustrated in Fig. 5.14. The code builds a matrix, where columns correspond to variables and rows to observations, as usual. Assume that we have the following parameters:

**FIGURE 5.14** **Code to sample from a multivariate normal distribution.**

    > Sigma <- matrix (c (4, 1, −2, 1, 3, 1, −2, 1, 5), nrow=3)
    > mu <- c(8,6,10)
    > eigen(Sigma)$values
    [1] 6.571201 4.143277 1.285521

Note that we make sure that the matrix Σ is positive definite, as it should be, by checking its eigenvalues with the function eigen. Now we may generate a sample and verify the results:

    > set.seed(55555)
    > Z <- MultiNormrnd(mu,Sigma,10000)
    > colMeans(Z)
    [1] 7.995686 5.981794 9.987316
    > cov(Z)
    [,1] [,2] [,3]
    [1,] 3.9930313 0.9789963 −2.003760
    [2,] 0.9789963 2.9525912 1.034064
    [3,] −2.0037602 1.0340636   5.021224

We should note that Cholesky decomposition is not the only one that we can use.12 An alternative is the square root matrix Σ1/2, i.e., a symmetric matrix such that

One way to find a symmetric square root is the following, taking advantage of matrix diagonalization:

    > eig <- eigen(Sigma)
    > V <- eig$vectors
    > V %*% diag(eig$values) %*% t(V)
          [,1] [,2] [,3]
    [1,]   4   1   −2
    [2,]   1   3   1
    [3,]   −2   1   5
    > sqrtSigma <- V %*% diag(sqrt(eig$values)) %*% t(V)
    > sqrtSigma %*% sqrtSigma
          [,1] [,2] [,3]
    [1,]   4   1   −2
    [2,]   1   3   1
    [3,]   −2   1   5

A handier alternative is the function sqrtm from package expm, as we have seen in Section 3.3.4. Anyway, with R there is no need to carry out that amount of work by ourselves, as we can rely on the mvrnorm function from the MASS package:

    > library(MASS)
    > set.seed(55555)
    > Z <- mvrnorm(n=10000,mu=mu,Sigma=Sigma)
    > colMeans(Z)
    [1] 7.967968 5.993851 10.001322
    > cov(Z)
    [,1] [,2] [,3]
    [1,] 3.9914023 0.9278669 −2.083500
    [2,] 0.9278669 2.9171899 1.014537
    [3,] −2.0835005 1.0145372 5.089220

The mvtnorm package offers an alternative function, rmvnorm, for this purpose.

# 5.6 Other ad hoc methods

Many ad hoc methods rely on relationships among distributions.13 For instance, we know that an Erlang variable Erl( _n_ , λ) is the sum of _n_ independent exponentials _X_ _i_ with rate λ. Hence, we may use the inverse transform method as follows:

By a similar token, we could take advantage of the fact that a chi-square variable χ2 _n_ with _n_ degrees of freedom is obtained by squaring and summing _n_ independent standard normals. However, we have also seen that these distributions may be regarded as specific subcases of the gamma distribution. We recall the PDF of a Gamma(α, λ) random variable:

(5.6)

where α is a shape parameter and λ is a scale parameter. Clearly, there is a notable difference between the cases α < 1 and α ≥ 1. Let us focus on the latter one. Different algorithms have been proposed for such a gamma variable. The following one, which we describe for illustrative purposes only, is based on acceptance–rejection:

**1.** Set _d_ = α −   and _c_ =  .

**2.** Generate two independent variates _U_ ~ U(0, 1) and _Z_ ~ N(0, 1), and set _V_ = (1 + _cZ_ )3.

**3.** If _Z_ > −1/ _c_ and log _U_ < 0.5 _Z_ 2 \+ _d − dV_ \+ _d_ log _V_ , then return _X_ = _dV_ ; otherwise go back to step 2.

The example also points out the fact that the boundary between ad hoc and general methods is a bit blurred, since we are using acceptance–rejection, but in a very peculiar way. Luckily, we may just rely on the rgamma function in R!

Another interesting case is the multivariate _t_ distribution _t_ ν(μ, Σ) (see Section 3.3.4). We may take advantage of the fact that a _t_ variable is related to the ratio of a standard normal and a chi-square, which in turn is a special case of the gamma. To induce correlation, we may use Cholesky factors of the covariance matrix, just like in the case of the normal.

**1.** Generate a column vector _Z_ ~ N( **0, I** ) of _n_ independent standard normals.

**2.** Generate _S_ ~ Gamma (ν/2, 1/2) ≡ χ2ν.

**3.** Compute **Y** = √ν/ _S_ **Z** and the Cholesky factor **U** of Σ.

**4.** Return **X** = μ + **U**  **Y**.

To deal with the multivariate _t_ distribution, we may use R functions from the package mvtnorm, such as rmvt, which has been introduced in Section 3.3.4.

# 5.7 Sampling from copulas

Given a copula _C_ ( _u_ 1,..., _u_ _n_ ) and marginal CDFs _F_ _i_ (·), _i_ = 1,..., _n_ , we may generate a vector **X** of _n_ dependent variables with those marginals by the following scheme:

**1.** Generate **U** ~ _C_ ( _u_ 1,..., _u_ _n_ ).

**2.** Return **X** = ( _F_ 1−1( _U_ 1),..., _F_ _n_ −1( _U_ _n_ )) .

As an instructive example, say that we want to generate a sample of _n_ = 10,000 points ( _X_ 1, _X_ 2), where _X_ 1 ~ gamma(2, 1) and _X_ 2 ~ N(0, 1), and the dependence is described by the Student's _t_ copula

where we denote by _T_ ν,Σ(···) the CDF of a multivariate _t_ distribution with _v_ degrees of freedom and by _T_ ν(·) the CDF of the corresponding univariate distribution; note that the covariance matrix Σ is actually a correlation matrix, in the sense that its diagonal entries are set to 1. To sample from the copula, we need not invert the multidimensional CDF. We rely on the extension of the following property for univariate distribution:14 If _X_ has CDF _F_ _X_ (·), then _F_ _X_ ( _X_ ) ~ U(0, 1). Thus, we sample _Y_ ~ _t_ ν( **0, Σ** ) from the multidimensional _t_ distribution, as we have seen in the previous section, and then we compute **U** = ( _T_ ν( _Y_ 1),..., _T_ ν( _Y_ _n_ )) . The vector **U** takes values in the unit hypercube and has uniform marginals, thanks to the aforementioned property; however, its components reflect the dependence structure associated with the copula. Finally, we generate **X** by inverting the marginals. The code in Fig. 5.15 accomplishes all of this when ν = 10 and

**FIGURE 5.15** **Code to sample from a multivariate _t_ copula with gamma and normal marginals.**

and it also produces the scatterplot and marginal histograms displayed in Fig. 5.16. Note the nonlinear association between the two variables, which cannot be accounted for by Pearson's correlation. We urge the reader to play with the code in order to see the effect of changing the correlation in the copula.

**FIGURE 5.16** **Scatterplot and histograms of the marginals for a bidimensional _t_ copula.**

All of the above can also be performed using the copula package. With the code in Fig. 5.17 we accomplish two tasks: First we sample from a given copula; then, given the sample, we fit a copula to see if we can get back the copula. The details are as follows:

**FIGURE 5.17** **Using the package** copula **to create, sample from, and fit a _t_ copula.**

**1.** We use tCopula to create a bidimensional _t_ copula, with degrees of freedom 10 and correlation parameter 0.8; the copula is specified as exchangeable by dispstr="ex", i.e., the two dimensions behave in the same way. Note that we could also use ellipCopula, which generates an elliptical copula. Such a copula family includes the normal and Student's _t_ , which can be specified by the family parameter.

**2.** We sample from the copula using rCopula; the output matrix of observations U is transformed into x by using the marginals we wish.

**3.** To go back and fit the copula, we need to transform the observations into pseudo-observations in the unit square; this may be accomplished by the function pobs. Alternatively, we might fit a marginal and use that. The observations and the pseudo-observations are plotted in Fig. 5.18.

**FIGURE 5.18** **Transforming observation to fit a copula.**

**4.** Finally, using the pseudo-observations Uhat, we fit a bidimensional _t_ copula.

Running the code, we obtain the following fitted copula, which is in good agreement with the one we have used to sample data:

    > fitted.copula
    fitCopula () estimation based on 'maximum pseudo-likelihood'
    and a sample of size 10000.
    Estimate Std. Error z value Pr(>|z|)
    rho. 1 0.805305 0.005384 149.6 <2e-16 ***
    df 9.161193 NA NA NA
    —
    Signif.codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
    The maximized loglikelihood is 5277
    Optimization converged
    Number of loglikelihood evaluations:
    function gradient
    34 11

# For further reading

  For an extensive treatment of random number generators, see [11].

  Testing procedures for random number generators are described in [5].

  Some books on simulation include extensive chapters on random variate generation. See, e.g., [3] or, at a more advanced level, [2].

  Advanced methods for the generation of random variates are treated at a research level in [1].

# References

1 J.H. Ahrens and U. Dieter. Computer methods for sampling from gamma, beta, Poisson and binomial distributions. _Computing_ , 12:223–246, 1974.

2 D.P. Kroese, T. Taimre, and Z.I. Botev. _Handbook of Monte Carlo Methods._ Wiley, Hoboken, NJ, 2011.

3 A.M. Law and W.D. Kelton. _Simulation Modeling and Analysis_ (3rd ed.). McGraw-Hill, New York, 1999.

4 P. L'Ecuyer. Efficient and portable combined random number generators. _Communications of the ACM_ , 31:742–749, 1988.

5 C. Lemieux. _Monte Carlo and Quasi–Monte Carlo Sampling._ Springer, New York, 2009.

6 B.D. Ripley. The lattice structure of pseudo-random number generators. _Proceedings of the Royal Society of London; Series A, Mathematical and Physical Sciences_ , 389:197–204, 1983.

7 B.D. Ripley. _Stochastic Simulation._ Wiley, New York, 1987.

8 C.P. Robert and G. Casella. _Monte Carlo Statistical Methods._ Springer, New York, 2004.

9 S. Ross. _Simulation_ (2nd ed.). Academic Press, San Diego, CA, 1997.

10 S. Ross. _Introduction to Probability Models_ (8th ed.). Academic Press, San Diego, CA, 2002.

11 S. Tezuka. _Uniform Random Numbers: Theory and Practice._ Kluwer Academic Publishers, Boston, 1995.

1These examples are taken from [7, pp. 22–25].

2See, e.g., [6] for a deeper investigation.

3 This may be useful to apply certain variance reduction strategies.

4 The exclusive OR, denoted as ⊕, is defined as follows: (1 ⊕ 0) = (0 ⊕ 1) = 1; (0 ⊕ 0) = (1 ⊕ 1) = 1. In the _inclusive_ OR case, denoted as V, we have (1 V 1) = 1.

5 The reader may appreciate the similarity between this line of reasoning and that of Eq. (3.40), where we showed that if _X_ has CDF _F_ _X_ (·), then _F_ _X_ ( _X_ ) is a uniform random variable.

6 See Section 3.2.4. The definition of the beta distribution, per se, does not require the above condition on its parameters; however, the PDF would have a vertical asymptote for either α1 < 1 or α2 < 1, which would prevent the application of acceptance–rejection.

7 The treatment here follows [8, pp. 47–51].

8 Strictly speaking, the set should be measurable.

9 See Section 3.2.1 and Eq. (3.14).

10 See, e.g., [10] for details.

11 In many numerical analysis textbooks, Cholesky decomposition is expressed as **A = LL**  , for a lower triangular matrix **L** ; R returns an upper triangular matrix **U** , but the two forms are clearly equivalent.

12 More on this in Section 13.4.

13 See Sections 3.2.2, 3.2.3, and 3.2.5.

14See Eq. 3.40.

# Chapter Six

# Sample Path Generation for Continuous-Time Models

Many models of interest in financial engineering can be represented by continuous-time stochastic differential equations. The application of Monte Carlo methods to this class of models requires sample path generation, which in turn requires a way to discretize them. Numerical analysis offers several ways to discretize deterministic differential equations for simulation purposes, including some very ingenious ones; here we will consider rather simple strategies, but we note that, due to the stochastic character of our applications, we have to tackle both numerical and statistical issues. To motivate the content of the chapter practically, we will mostly consider simple asset pricing problems, leaving more complex examples to Chapter 11 on option pricing, where we also apply variance reduction techniques, and Chapter 13 on risk management and hedging. The risk-neutral pricing approach that we discussed in Section 3.9 allows to price financial instruments by taking suitable expectations, which in turn requires Monte Carlo simulation, when the model is complicated enough to preclude an analytical solution. The following cases, listed in increasing order of complexity, may occur, depending on the nature of the underlying stochastic process and the kind of derivative contract we are evaluating:

**1.** We just need the value of a random variable at a single point in time and we are able to sample paths exactly. This is the lucky case where we may just need to sample a single random variable, and we do not need to discretize time. A relevant example is vanilla European-style call option in the Black–Scholes–Merton world.

**2.** We need the value of a random variable at multiple points in time, but we are able to sample paths exactly. Hence, we have to discretize time and the computational effort is increased. In some cases, we need not wonder about the most suitable time discretization, as this is written in the contract. Significant examples are a path-dependent European-style Asian option with discrete-time averaging and a Bermudan-style option, featuring early exercise opportunities but only at prespecified time instants. In both of these cases, the time instants at which we need to sample the value of the underlying asset are stated precisely in the contract. On the contrary, if we are dealing with an American-style derivative, the discretization step will have an impact on both accuracy and computational effort, as the option may be exercised anywhere before maturity.

**3.** We are not able to sample paths exactly. This may happen when we step out of the safe domain of geometric Brownian motion (GBM) and deal with mean reversion, stochastic volatility, non-Gaussian processes, etc. In such cases, we not only have to deal with the sampling and estimation errors that are common to all Monte Carlo applications, but with discretization issues as well.

Since here we are dealing with differential equations, the level of mathematical complexity is nontrivial, and we will take an informal view, keeping practical applications in mind. The necessary background on stochastic calculus was given in Section 3.7

We introduce the essential issues of path generation in Section 6.1, where we outline trivial and less trivial discretization strategies, like Euler and Milstein, as well as predictor-corrector methods. In Section 6.2 we investigate the easy case of geometric Brownian motion. There, we also cover the generation of sample paths for correlated processes and the Brownian bridge construction. Simple option pricing examples, concerning derivatives written on non-dividend-paying stocks, will help us along the way. We also apply these basic ideas to the simulation of rather simple interest rate models, like Vasicek and Cox–Ingersoll–Ross short-rate models in Section 6.3; such models involve mean reversion and may result or not in Gaussian processes, but they still feature a constant volatility. We move on and allow for stochastic volatility in Section 6.4. Finally, in Section 6.5 we also introduce jumps. We defer to Chapter 9 the use of low-discrepancy sequences for path generation within the framework of quasi–Monte Carlo simulation.

# 6.1 Issues in path generation

Let us consider the stochastic process _S_ _t_ describing, e.g., the price of a non-dividend-paying stock.1 A wide class of models can be cast into the form of an Itô stochastic differential equation:

(6.1)

where the function _a_ ( _S_ _t_ , _t_ ) defines the drift component, the function _b_ ( _S_ _t_ , _t_ ) defines the volatility component, and _dW_ _t_ is the differential of a standard Wiener process. We know that the standard Wiener process is a Gaussian process with stationary and independent increments such that

Therefore, for a small time interval δ _t_ , we have

which in practice means that we may simulate small increments as follows:

The notation  _t_ +δ _t_ may be preferred to  _t_ , since it points out clearly that the increment of the Wiener process is generated _after_ observing the state at time _t_ ; however, this is not quite essential, and in the following we will often just write  , since no confusion should arise. Sampling paths of a standard Wiener process with any time discretization involves no more than the generation of a stream of independent standard normals. When dealing with a simple generalized Wiener process like

(6.2)

where _a_ and _b_ are constant, we have just to integrate both sides:

(6.3)

This immediately yields the following discretization scheme:

(6.4)

which allows for easy sample path generation. When the discretization step is uniform, it may be better to introduce integer time stamps _k_ , _k_ = 0, 1, 2,..., corresponding to time instants _t_ _k_ = _k_ δ _t_ , and rewrite Eq. (6.4) as

(6.5)

where we may insist on writing  _k_ +1 to point out that this shock is not known when we observe _S_ _k_. We should note that if we generate sample paths of the generalized Wiener process and collect sample statistics, we are prone to sampling errors, in the sense that sample statistics will be subject to statistical fluctuations calling for the evaluation of confidence intervals. Sampling error is due to the random nature of Monte Carlo methods, and it can be mitigated using variance reduction strategies. However, there is no discretization error, in the sense that the marginal distributions of the random variables _S_ _k_ , both conditional and unconditional, are correct.

By analogy, when faced with the more general Itô process of Eq. (6.1), the simplest discretization approach that comes to mind is

(6.6)

Such a discretization is known as the Euler, or Euler–Maruyama, scheme. Note that we are assuming that functions _a_ ( _S_ _t_ , _t_ ) and _b_ ( _S_ _t_ , _t_ ) are constant over the time interval [ _t_ , _t_ \+ δ _t_ ). Since this is not really the case, it is clear that we are introducing a discretization error. The issue is faced also when dealing with an ordinary deterministic differential equation, and it can be kept under control by a wide array of numerical analysis techniques. Still, it might be argued that when δ _t_ → 0, the discretized solution should converge to the "true" solution of the stochastic differential equations. Unfortunately, convergence is a critical concept in stochastic differential equations, and a proper analysis would require a relatively deep mathematical machinery that is beyond the scope of this book.2 Nevertheless, it is quite easy to see that discretization errors are relevant and may even change the marginal distributions. For instance, consider the geometric Brownian motion described by

(6.7)

The Euler scheme yields

This is very easy to grasp and to implement, but the marginal distribution of each variable _S_ _t_ is normal, whereas we know from Section 3.7.3 that it should be lognormal. This is not quite a negligible issue when we model stock prices or interest rates that are supposed not to get negative. In this specific case, we show in Section 6.2 that we may use Itô's lemma and get rid of the discretization error altogether, but this ideal state of the matter cannot always be achieved. When we have to cope with inexact discretization, the Euler scheme is just the simplest choice and an array of more refined discretization schemes have been proposed in the literature. Below we outline a couple of alternatives, namely, the Milstein scheme and predictor-corrector methods.

A final, and possibly less obvious, question is also worth mentioning: Time discretization may induce another form of bias. To see this, imagine pricing an American-style option, which may be exercised at any time instant before and including its expiration date. When pricing it by Monte Carlo, we are actually pricing a Bermudan-style option that can be exercised at a _discrete_ set of time instants. Clearly, the value of the Bermudan-style option is a lower bound on the price of the corresponding American-style option; hence, we are introducing a low bias.3 One possible countermeasure is to adopt Richardson extrapolation, which is a rather common approach in numerical analysis to approximate a limit when δ _t_ → 0. Given a sequence of prices of Bermudan-style options that can be exercised with higher and higher frequencies, we may approximate a continuous-time exercise opportunity limit.

## 6.1.1 EULER VS. MILSTEIN SCHEMES

The Milstein scheme is based on a higher order Taylor expansion of Eq. (6.1):

(6.8)

We will justify this expansion later; for now, we should note that the formal rule ( _dW_ )2 = _dt_ shows that the last term disappears in the limit, so that we get back to the original differential equation.

###   Example 6.1 GBM processes by the Milstein scheme

As an immediate illustration, let us apply the scheme of Eq. (6.8) to a GBM process. Since _b_ ( _S_ , _t_ ) = σ _S_ for this kind of process, we have

and Eq. (6.8) becomes

Now, from a computational point of view, we rewrite  , which also implies

Finally, we obtain

(6.9)

It is interesting to compare Euler and Milstein approximations against the Taylor expansion of the exact formula based on an exponential:

where in the last line we have neglected terms of order higher than δ _t_. We understand that the Milstein scheme adds some missing term in the Euler approximation

so that now the formula is correct and yields an exact expectation and variance up to order δ _t_.

For the sake of simplicity, we provide a justification of the Milstein scheme in the scalar case, assuming that functions _a_ (·, ·) and _b_ (·, ·) only depend on _S_ , and not on time _t_. The starting point is recalling that a stochastic differential equation is actually a shorthand for an integral equation

The Euler scheme approximates integrals over the interval ( _t_ , _t_ \+ δ _t_ ) by "freezing" the integrand functions to their value at time _t_ :

(6.10)

We may try to improve on this by finding a better approximation of the diffusion term _b_ ( _S_ _t_ ) over the time interval ( _t_ , _t_ \+ δ _t_ ). In order to get a clue, let us consider Itô's lemma for the process _b_ ( _S_ _t_ ) (here we do not need to denote derivatives as partial ones, since we have ruled out dependence on time):

where we introduce μ _b_ ( _S_ _t_ ) and σ _b_ ( _X_ _t_ ) as shorthands for the drift and volatility functions in the differential equation of _b_ ( _S_ _t_ ). If we apply the Euler approximation to _b_ ( _S_ _t_ ) over the time interval ( _t_ , _u_ ), we find

where we have neglected the drift term, as it is of order [ _u_ − _t_ ], whereas the increment of the Wiener process is of order  . Rather than freezing the integrand in Eq. (6.10), we may substitute the last approximation:

(6.11)

The missing piece of the puzzle is the last integral, which we may compute by recalling Example 3.18. Applying that result we immediately find:

Plugging this into Eq. (6.11) yields

Substituting this refined approximation of the diffusion term into the Euler scheme finally yields the Milstein scheme:

Generalizations of this result are available, for which we refer to the chapter references.

## 6.1.2 PREDICTOR-CORRECTOR METHODS

When we apply the Euler scheme to the Itô differential equation

over the time interval ( _t_ , _t_ \+ δ _t_ ), we "freeze" the drift and volatility functions to their initial values and sample

The advantage of this scheme is that it is explicit, but we could try to account for the change in the functions _a_ (·, ·) and _b_ (·, ·) is some way. There is an array of ideas, originally developed for the numerical solution of deterministic (ordinary) differential equations, which can be adapted to the stochastic case, including implicit schemes. An implicit version of the Euler scheme could be

The scheme is implicit because now we have to solve an equation to find the new value of _S_ _t_ +δ _t_. Since this approach could be numerically troublesome, as it may require division by a Wiener increment close to zero, an alternative form is

(6.12)

where the diffusion term is kept explicit.

###   Example 6.2 An implicit Euler scheme

If we consider the stochastic differential equation

the scheme of Eq. (6.12) yields

which may be rewritten as

Needless to say, things are not always this easy, not to mention the open question of convergence. A possible alternative is to introduce a correction mechanism comparing the actual and the predicted value of the process in _t_ \+ δ _t_. For instance, the value obtained by straightforward application of the Euler scheme may be considered as a predictor

Since the value of the functions   and   at the end of the timestep are different from what was used in the prediction step, we may introduce a correction as follows:

(6.13)

where α and η are weights in the interval [0, 1] and

(6.14)

The role played by the weights α and η is rather intuitive, as they blend the initial and terminal values of functions _a_ (·, ·), and _b_ (·, ·), controlling the amount of correction; often, they are set close to 0.5. The further adjustment of Eq. (6.14) is less intuitive, but we refer the interested reader to [3] for further information, as well as a serious treatment of convergence issues. The aim of this short section was just to get readers acquainted with the richness of discretization approaches, but for the sake of simplicity we will refrain from using advanced path generation methods in what follows. We should also notice that, in the literature, the usefulness of some of these approaches with respect to a simpler Euler scheme with a suitably short time step has been sometimes questioned.

# 6.2 Simulating geometric Brownian motion

Sometimes, we may use suitable transformations of a differential equation to cast it into a more manageable form. From Section 3.7 we know that, by using Itô's lemma, we may rewrite Eq. (6.7), which defines GBM, as

(6.15)

Now this is a generalized Wiener process that we may easily integrate as we have seen in the previous section. Taking exponentials to get rid of the logarithm yields

where ν = μ − σ2/2. Since the integral of a standard Wiener process is normally distributed, from a computational point of view we may recast the last expression as

which shows lognormality of prices. From a path generation point of view, the last expression is used as

(6.16)

from which it is easy to generate sample paths of GBMs. A straightforward code is given in Fig. 6.1. Experienced R programmers will certainly notice that the function includes two dreaded nested for loops, which does not help efficiency; we will come back to that in a moment. The following snapshot produces the sample paths in Fig. 6.2, showing the impact of volatility:

**FIGURE 6.1** **R code to generate sample paths of a geometric Brownian motion.**

**FIGURE 6.2** **Sample paths of GBMs for different values of volatility.**

    set.seed(55555)
    paths1 <- simGBM(50,0.1,0.1,1,300,5)
    paths2 <- simGBM(50,0.1,0.4,1,300,5)
    yBounds <- c(min(min(paths1),min(paths2)),
    max(max(paths1),max(paths2)))
    par(mfrow=c(1,2))
    plot(1:301, paths1[1,], ylim=yBounds,type=′1′,
        xlab="sigma=10%",ylab=")
    for (j in 2:5)
    lines(1:301,paths1[j,]) plot(1:301,paths2[1,],ylim=yBounds,type=′1′,
        xlab="sigma=40%",ylab=")
    for (j in 2:5)
    lines(1:301,paths2[j,])
    par(mfrow=c(1,1))

The code in Fig. 6.1 is based on two nested for loops, contrary to common recommendations concerning R code efficiency. In order to vectorize the code, it is convenient to rewrite Eq. (6.16) as

We generate _differences_ in the logarithm of the asset prices and then use the cumsum function to obtain cumulative sums of log-increments along each sample path. The resulting function simvGBM is illustrated in Fig. 6.3. We may compare the two implementations in terms of speed:

**FIGURE 6.3** **Vectorized code to generate sample paths of GBM.**

    > system.time(simGBM(50,0.1,0.1,1,3000,100))
    user system elapsed
    2.54     0.00     2.54
    > system.time(simvGBM(50,0.1,0.1,1,3000,100))
    user system elapsed
    0.15     0.00     0.16

## 6.2.1 AN APPLICATION: PRICING A VANILLA CALL OPTION

As an immediate application of path generation for GBMs, we may price a vanilla call option on a non-dividend-paying stock. Clearly, there is no point in using Monte Carlo methods in this case, as we may apply the Black–Scholes–Merton formula that we discussed in Section 3.9.2. However, it is quite useful to verify the accuracy of numerical methods in easily solved case to check their viability and possible limitations. In Fig. 6.4 we recall for convenience the function giving the exact option price, and we also propose a very simple function to estimate the price by Monte Carlo. We just need:

**FIGURE 6.4** **Exact formula and a simple Monte Carlo pricer for a vanilla call option.**

**1.** To sample lognormal prices using Eq. (6.16).

**2.** To evaluate and discount the corresponding payoff (assuming a constant risk-free rate).

**3.** To return the estimated price along with a confidence interval, whose confidence level c.level is 95% by default. Note that we return a list with fields value ci containing the point and the interval estimate, respectively.4

The code also includes a little snapshot to verify accuracy. The relevant output is

    > EuVanillaCall(S0,K,T,r,sigma)
    [1] 5.234659
    > out$estimate
    [1] 5.164366
    > out$conf.int
    [1] 5.056212 5.272520
    > 100*(out$conf.int[2]-out$conf.int[1])/2/out$estimate
    [1] 2.09423

We observe that the point estimate is not quite precise, even though the true value is included in the 95% confidence interval. Taking the ratio between the half-length of the confidence interval and the point estimate suggests an error above 2%. One crude way to improve the estimate would be to increase the number of replications. We will see in Chapter 8 that variance reduction strategies may be a much better option.

## 6.2.2 MULTIDIMENSIONAL GBM

When dealing with options on multiple assets or managing multiple risk factors, we may need to generate a multidimensional geometric Brownian motion. The key issue is how to deal with correlation among risk factors, and there are two modeling choices, which have an impact on how simulation is carried out. For the sake of simplicity we illustrate the two approaches in the bidimensional case, where two Wiener processes drive prices _S_ 1( _t_ ) and _S_ 2( _t_ ). We may consider two independent standard Wiener processes that enter in both equations, thus inducing a correlation between prices:

An alternative is to consider instantaneously correlated Wiener processes and write the equations in the somewhat more familiar form:

The role of instantaneous correlation ρ can be understood by referring to the following formal rule of stochastic calculus:

We note that the analogous rule ( _dW_ )2 = _dt_ is actually a specific case of this rule, which may be justified heuristically along the lines of Section 3.7.3. We will take the second approach, which requires the generation of correlated standard normals. We know from Section 5.5.2 that this may be accomplished by taking the Cholesky factorization of the covariance matrix, which in this case is actually a correlation matrix, since we are dealing with standard normals. We illustrate the idea first in the bidimensional case, to reinforce intuition. In fact, when only two processes are involved, we may generate increments based on the correlated standard normal variables  1 and  2 obtained as a transformation of independent variables _Z_ 1, _Z_ 2 ~ N(0, 1):

The simple code in Fig. 6.5 generates a bidimensional sample path using the above transformation. The following R snapshot produces the plots in Fig. 6.6, which illustrate the impact of correlation:

**FIGURE 6.5** **Simple R code to generate sample paths of bidimensional GBM.**

**FIGURE 6.6** **Sample paths of a bidimensional GBM for different values of instantaneous correlation.**

    > set.seed(55555)
    > S0 <- c(30,50)
    > mus <- c(0.1, 0.1)
    > sigmas <- c(0.3,0.3)
    > paths1 <- sim2GBM(S0,mus,sigmas,0.3,1,300)
    > paths2 <- sim2GBM(S0,mus,sigmas,0.9,1,300)
    > yBounds <- c(min(min(paths1),min(paths2)),
    + max(max(paths1),max(paths2)))
    > par(mfrow=c(1,2))
    > plot(1:301,paths1[1,],ylim=yBounds,type=′1′,
    xlab="rho=0.4",ylab=")
    > lines (1:301,paths1[2,])
    > plot(1:301,paths2[1,],ylim=yBounds,type=′1′,
    xlab="rho=0.9",ylab=")
    > lines(1:301,paths2[2,])
    > par(mfrow=c(1,1))

The previous implementation is rather simple as it involves a single replication. The code in Fig. 6.7 is a bit more involved since, to store multiple replications, we need a tridimensional array; this is created by the function array, which requires a vector specifying the array size along each dimension.

**FIGURE 6.7** **Generating sample paths of multidimensional GBMs.**

## 6.2.3 THE BROWNIAN BRIDGE

In the previous sections we have generated asset paths according to a natural scheme, which proceeds forward in time. Actually, the Wiener process enjoys some peculiar properties which allow us to generate the sample paths in a different way. Consider a time interval with left and right end points _t_ _l_ and _t_ _r_ , respectively, and an intermediate time instant _s_ , such that _t_ _l_ < _s_ < _t_ _r_. In standard path generation, we would generate the Wiener process in the natural order: _W_ ( _t_ _l_ ), _W_ ( _s_ ), and finally _W_ ( _t_ _r_ ). Using the so-called Brownian bridge, we may generate _W_ ( _s_ ) as the last value, conditional on the values _W_ ( _t_ _l_ ) = _w_ _l_ and _W_ ( _t_ _r_ ) = _w_ _r_. The conditional value of _W_ ( _s_ ) is a normal variable with expected value

(6.17)

and variance

(6.18)

This is a consequence of the properties of the conditional distribution of a multivariate normal distribution, which we have stated in Theorem 3.7. Before proving Eqs. (6.17) and (6.18), it is important to realize that they are quite intuitive: The conditional expected value of _W_ ( _s_ ) is obtained by linear interpolation through _w_ _l_ and _w_ _r_ ; the variance is low near the two end points _t_ _l_ and _t_ _r_ and is at its maximum in the middle of the interval.

A preliminary step to prove Eqs. (6.17) and (6.18) is to find the covariance between _W_ ( _t_ ) and _W_ ( _u_ ), when _t_ < _u_ :

The statement can be generalized as Cov( _W_ ( _t_ ), _W_ ( _u_ )) = min{ _t, u_ }. Then, the joint distribution of the three values that we are considering is

Note that we have listed the time instant in view of the application of Theorem 3.7, since we need the value in _s_ conditional on the values in _t_ _l_ and _t_ _r_. Partitioning vector of expected values and the covariance matrix accordingly yields

and

By plugging the inverse matrix

into the above formulas we obtain Eqs. (6.17) and (6.18).

By using the Brownian bridge, we may generate sample paths by a sort of bisection strategy. Given _W_ (0) = 0, we sample _W_ ( _T_ ); then we sample _W_ ( _T_ /2). Given _W_ (0) and _W_ ( _T_ /2) we sample _W_ ( _T_ /4); given _W_ ( _T_ /2) and _W_ ( _T_ ) we sample _W_ (3 _T_ /4), etc. Actually, we may generate sample paths in any order we wish, with nonhomogeneous time steps. One could wonder why this complicated construction could be useful. There are at least two reasons:

**1.** It may help in using variance reduction by stratification, to be discussed in Chapter 8. It is difficult to use stratification in multiple dimensions, but we may use stratification just on the terminal value of the asset price, and maybe an intermediate point. Then we generate the remaining values along the sample path by using the bridge.

**2.** The Brownian bridge construction is also useful when used in conjunction with low-discrepancy sequences, as we shall see in Chapter 9. Such sequences may not work well in very high dimensional domains, but using Brownian bridge we may use high-quality sequences to outline the paths of the Wiener process, by sampling points acting as milestones; then we can fill the trajectory by Monte Carlo sampling.

In Fig. 6.8 we illustrate an R function to generate sample paths of the standard Wiener process using the Brownian bridge technique, but only in the specific case in which the time interval [0, _T_ ] is successively bisected (i.e., the number of intervals is a power of 2). In this case, we may simplify the formulas above to sample _W_ ( _s_ ): If we let δ _t_ = _t_ _l_ − _t_ _r_ , we obtain

**FIGURE 6.8** **Implementing path generation by a Brownian bridge for the standard Wiener process.**

where   is a standard normal variable. The function receives the length T of the time interval and the number numSteps of subintervals in which it must be partitioned, and it returns a vector containing one sample path. Assume that the number of intervals is 8 (a power of 2). Then we must carry out 3 bisections.

  Given the initial condition _W_ ( _t_ 0) = 0, we must first sample _W_ ( _t_ 8), which means "jumping" over an interval of length _T_ , which is the initial value TJump in the program. Since we store elements in a vector of nine elements (starting with index 1, and including _W_ ( _t_ 0)), we must jump eight places in this vector to store the new value. The number of places to jump is stored in IJump.

  Then we start the outermost for loop. In the first pass we must only sample _W_ ( _t_ 4), given _W_ ( _t_ 0) and _W_ ( _t_ 8). Given positions left<-1 and right<-IJump+1, we must generate a new value and store it in position i<-IJump/2 + 1, which is 4+1 = 5 in this case. Here we generate only one value, and we divide both time and index jumps by 2.

  In the second iteration we must sample _W_ ( _t_ 2), given _W_ ( _t_ 0) and _W_ ( _t_ 4), and _W_ ( _t_ 6), given _W_ ( _t_ 4) and _W_ ( _t_ 8). The nested loop will be executed twice, and indexes left, right, and i are incremented by 4.

  In the third and final iteration we generate the remaining four values.

The construction above is illustrated in Fig. 6.9. Figure 6.10 includes a script to check that the marginal distributions of the stochastic process that we generate are the correct ones. Expected values should be zero, and standard deviation should be the square root of time:

**FIGURE 6.9** **Illustrating the Brownian bridge construction by successive bisection.**

**FIGURE 6.10** **Checking the path generation for the standard Wiener process by a Brownian bridge.**

    > m <- colMeans(Wpaths[,-1]);m
    [1] 5.234951e-05 1.624087e-03 3.127389e-03 1.542460e-03
    > sdev <- apply(Wpaths[,-1],2,sd);sdev
    [1] 0.4993618 0.7080760 0.8653553 0.9971483
    > sqrt((1:numSteps)*T/numSteps)
    [1] 0.5000000 0.7071068 0.8660254 1.0000000

We see that, within sampling errors, the result looks correct. Given a way to generate the standard Wiener process, it is easy to simulate geometric Brownian motion. The function is given in Fig. 6.11 and uses a similar approach as the vectorized version simvGBM of Fig. 6.3. One thing we should note is the use of the function diff to generate the vector of Increments in the logarithmic asset price. In fact, in standard Monte Carlo we generate the underlying Wiener process by successive increments; with the Brownian bridge construction we build directly the values of the process at different time instants, and we must use the function diff to obtain the relative differences. In some sense, diff works in the opposite way to ecumsum.

**FIGURE 6.11** **Simulating geometric Brownian motion by the Brownian bridge.**

# 6.3 Sample paths of short-term interest rates

Modeling interest rates is a quite difficult endeavor, as it involves different rates applying to different maturities. The simplest class of models deals with only one interest rate applying to a very short time horizon. Let _r_ _t_ be the risk-free interest rate applying to the time interval ( _t,t_ \+ _dt_ ). This may be called the instantaneous interest rate, although it is often referred to as the _short rate._ There are different models for the short rate, but in this section we consider only the two that we have already mentioned in previous chapters:

**1.** The Vasicek model, characterized by a stochastic differential equation featuring mean reversion:

(6.19)

**2.** The Cox–Ingersoll–Ross (CIR) model, which is quite similar to the Vasicek model, but involves a slight change in the volatility term:

(6.20)

In the following, we show that this slight difference has a significant consequence:

  Short rates are normally distributed in the Vasicek model; the CIR model involves a more complicated noncentral chi-square distribution (see Section 3.2.3.2).

  The easier distribution of the Vasicek model results in better analytical tractability; we are able to price bonds and also some options analytically, whereas the CIR model involves more complicated formulas, when available.

  However, the Vasicek model can be criticized as the normal distribution allows for negative interest rates, whereas the volatility term in the CIR models avoids this difficulty, provided that

The intuition is that when the short rate _r_ _t_ gets close to zero, the volatility term in Eq. (6.20) vanishes; if the pull toward the long-term average rate is strong enough, the short rate will stay positive.

In both cases, generating sample paths is not too difficult, as we shall see, since we know the exact distribution of interest rates. More involved models call for more sophisticated path generation strategies, possibly involving discretization errors.

## 6.3.1 THE VASICEK SHORT-RATE MODEL

The process described in Eq. (6.19) is not a GBM; actually, it is an example of an Ornstein–Uhlenbeck process. To try and solve the equation, we may resort to a typical trick of the trade, related to the method of integrating factors in deterministic ordinary differential equations. The trick is to apply Itô's lemma to the process

The usual drill requires the calculation of partial derivatives:

Hence, the differential equation for _f_ ( _r_ _t_ , _t_ ) is

This equation does not look too bad and can be integrated over the interval (0, _t_ ):

which can be rewritten as

This expression includes a stochastic integral involving the Wiener process that, as we know, is normally distributed and has zero expected value. Thus, we may immediately conclude that the short rate _r_ _t_ under the Vasicek model is normally distributed with expected value:5

(6.21)

Finding variance is a bit trickier and requires the following result.

**THEOREM 6.1 (Itô's isometry)** _Let X_ _t_ _be a stochastic process adapted to the standard Wiener process W_ _t_. _Then_ ,

Actually, the result can be proven in a more general form, but it allows us to find the variance of the short rate:

(6.22)

It is important to get an intuitive feeling for Eqs. (6.21) and (6.22):

  At time _t_ = 0 the short rate is _r_ 0 and variance is zero.

  When _t_ → + ∞, the short rate tends to the long-term average value  ; the larger the speed of mean reversion γ, the faster the convergence.

  Long-term variance is given by σ2/(2γ); as expected, it is influenced by the volatility σ; however, also γ plays a role, and a strong mean reversion tends to kill volatility.

In order to use the above knowledge to generate sample paths with time step δ _t_ , we should express _r_ _t_ +δ _t_ conditional on _r_ _t_ :

where   ~ N(0,1).

Now we may generate sample paths by the code illustrated in Fig. 6.12. The code produces the plots in Fig. 6.13. We clearly see the impact of the mean reversion coefficient γ. In the plots for γ = 0.1 we also notice how short rates can indeed get negative in the Vasicek model.

**FIGURE 6.12** **Simulating short rates by the Vasicek model.**

**FIGURE 6.13** **Illustrating the effect of mean reversion on short rates in the Vasicek model.**

## 6.3.2 THE COX–INGERSOLL–ROSS SHORT-RATE MODEL

The CIR model relies on a square-root diffusion process, which is not as easy to deal with as the Ornstein–Uhlenbeck process underlying the Vasicek model. However, by studying the transition density of the square-root diffusion process, a link with the noncentral chi-square distribution6 can be established. We state the following result without proof.

**THEOREM 6.2 (CIR short-rate model)** _The transition law from r_ 0 _to r_ _t_ _for the CIR model can be expressed as_

(6.23)

_where the degrees of freedom are v_ = 4 γ/α _and the noncentrality parameter is_

By applying the theorem to the transition from _r_ _t_ to _r_ _t_ +δ _t_ , we immediately obtain a path generation mechanism. It is also useful to find the conditional expectation and variance of _r_ _t_ given _r_ 0, which is easily obtained by recalling Eq. (3.18). For the expected value we have

(6.24)

We notice that the expected value has the same intuitive form as in the Vasicek model. Variance is a bit trickier:

(6.25)

On the basis of Theorem 6.2 it is easy to build an R function to generate sample paths for the CIR short-rate model. It is also interesting to compare the exact sample paths against those generated by the straightforward Euler discretization

(6.26)

where   ~ N(0, 1) as usual. The code displayed in Fig. 6.14 includes two such functions. Figure 6.15 shows a script to compare the exact distribution against a one-step Euler approximation. For the cases _T_ = 0.25 and _T_ = 1 we sample the rates obtained in one step, with the following parameters:7

**FIGURE 6.14** **Simulating short rates by the CIR model: exact approach and Euler approximation.**

**FIGURE 6.15** **Comparing kernel estimates of CIR interest rates by an exact approach and the Euler approximation.**

Note that this setting of α corresponds to a volatility coefficient σ = 0.1 (see Eq. (6.20)). In Fig. 6.16 we compare the kernel densities estimated by sampling 100,000 replications; the densities estimated on the basis of the Euler scheme are drawn with dashed lines. We observe that for _T_ = 0.25 there is a close match, but for _T_ = 1 the Euler rates, which are in fact normally distributed, may get negative.

**FIGURE 6.16** **Kernel densities for exact CIR rates (continuous lines) and rates sampled by the Euler scheme (dashed lines).**

# 6.4 Dealing with stochastic volatility

In geometric Brownian motion models, the volatility σ is constant. On the contrary, an empirical finding is that volatility is stochastic, and clusters of high vs. low volatility can be observed over time. In discrete time, this may be modeled along the lines of GARCH time series models that we have seen in Chapter 3. We may also consider continuous-time models, and one possible approach relies on the square-root diffusion process that we have introduced in Section 6.3.2 for short-term interest rates. One such model is the Heston model:

(6.27)

(6.28)

Equation (6.27) is basically a generalization of GBM, and Eq. (6.28) models variance _V_ _t_ as a mean-reverting square-root diffusion. An easy way to generate sample paths for this process relies on a simple Euler discretization; we have already seen that this may result in negative values for quantities that are non-negative by definition. The more advanced path generation strategies that we have discussed earlier in the chapter may be exploited to remedy the situation. We defer details to Chapter 11, where we will see how to price a path-dependent, lookback option under stochastic volatility.

# 6.5 Dealing with jumps

Diffusion processes are continuous (with probability 1), and they are often criticized for this very reason, as sudden jumps are indeed observed in financial markets. A simple approach to incorporate jumps is to add the increments of a compound Poisson process, which is characterized by a rate λ and a distribution of the jump size. In practice, this means that jump epochs are separated by a stream of independent exponential variables with rate λ, and that at each such epoch we have to add a random jump size to the increment of the diffusion component. For instance, we may consider a generalized Wiener process, described by the differential equation

which can be easily solved:

How can we add normally distributed jumps N(ν, ξ)? The theoretical framework we need is provided by the theory of Lévy processes, which generalize both diffusions and Poisson (pure jump) processes. The essential features of Lévy processes are:

**1.** Increments are independent.

**2.** Increments are stationary.

**3.** The process is stochastically continuous.

**4.** The initial value is zero (with probability 1).

These are the same properties of the standard Wiener process, and the only point worth mentioning is property 3, which may seem at odds with discontinuous sample paths of Poisson processes. A correct statement of the property, in a one-dimensional case, is

Hence, jumps are not actually ruled out, but they cannot be "dense." For instance, in a Poisson process the probability that there is more than one jump in an infinitesimal time interval _dt_ is negligible.

A key result is the Lévy–Itô decomposition, whose proper statement is beyond the scope of this book.8 Loosely speaking, the theorem states that we may decompose a Lévy process into separate components, accounting for diffusion and jump subprocesses. Let us illustrate how we may take advantage of this theorem by adding a jump component to the above generalized Wiener process. To this aim, we shall take advantage of the following properties of the Poisson process:

  The number _N_ of jumps in the time interval [0, _T_ ] is a Poisson random variable with parameter λ _T_.

  Conditional on _N_ , the jump epochs are a sequence of _N_ i.i.d. uniform variables on [0, _T_ ].

Therefore, in order to generate a sample path of this jump–diffusion process, we may:

**1.** Generate the increments of the diffusion component as usual.

**2.** Sample the number of jumps as a Poisson variable.

**3.** Generate the time epochs as uniform random variables.

**4.** Sample the jump sizes and add them to the corresponding increments of the diffusion process.

The code in Fig. 6.17 applies the above ideas and produces the sample path displayed in Fig. 6.18. Following the traditional convention to plot discontinuous functions, we have added a bullet–circle pair corresponding to each point of discontinuity. Note that the process is continuous from the right.

**FIGURE 6.17** **Simulating a jump–diffusion process.**

**FIGURE 6.18** **Sample path of a jump–diffusion process.**

# For further reading

  For a sound theoretical treatment of numerical methods to solve stochastic differential equations, see [3].

  Several examples, at a more advanced level with respect to this chapter are described in [1] and [5]. See also [4], which requires some advanced measure theoretic background.

  An interesting discussion of transformation methods and the Milstein scheme is given in [2].

# References

1 P. Glasserman. _Monte Carlo Methods in Financial Engineering._ Springer, New York, 2004.

2 P. Jaeckel. _Monte Carlo Methods in Finance._ Wiley, Chichester, 2002.

3 P.E. Kloeden and E. Platen. _Numerical Solution of Stochastic Differential Equations._ Springer, Berlin, 1992.

4 R. Korn, E. Korn, and G. Kroisandt. _Monte Carlo Methods and Models in Finance and Insurance._ CRC Press, Boca Raton, FL, 2010.

5 D.P. Kroese, T. Taimre, and Z.I. Botev. _Handbook of Monte Carlo Methods._ Wiley, Hoboken, NJ, 2011.

1To avoid notational clutter, we will mostly, but not exclusively, use the style _S_ _t_ rather than _S_ ( _t_ ) in this chapter. The choice will be purely dictated by convenience.

2Suitable references are provided at the end of the chapter.

3We investigate the sources of bias in American-style option pricing in Section 11.3.1.

4We discuss confidence intervals and output analysis in Chapter 7.

5By the way, we can find this expectation by setting σ = 0 and solving the resulting deterministic equation.

6See Section 3.2.3.2.

7We are replicating the results discussed in [1, pp. 124–125].

8See, e.g., [5, p. 209].

# Part Four

# Output Analysis and Efficiency Improvement

# Chapter Seven

# Output Analysis

There are several reasons why we should carefully check the output of a Monte Carlo simulation:

**1.** Model validation, i.e., check that the model makes economic and/or financial sense, possibly with respect to empirical evidence.

**2.** Model verification, i.e., check that the computer code we have written implements the model, right or wrong, that we had in mind.

**3.** Statistical inference, i.e., check the accuracy of the estimates that we obtain by running the model implemented in the code.

In this chapter we deal with the last point. To motivate the development below, let us refer back to the queueing example that we introduced in Section 1.3.3.1, where Lindley's recursion is used to estimate the average waiting time in an _M/M_ /1 queue, i.e., a queue where the distributions of interarrival and service times are both memoryless, i.e., exponential.1 For the sake of convenience, we report the R code in Fig. 7.1; the function has been slightly modified, in order to return the whole vector of recorded waiting times, rather than just its mean.

**FIGURE 7.1** **Function to simulate a simple _M/M_ /1 queue.**

With the default input arguments, the average server utilization is

and 10,000 customers are simulated. Is this sample size enough for practical purposes? The following runs suggest some criticality, as the sample means show a considerable variability:

    > set.seed(55555)
    > mean(MM1_Queue_V())
    [1] 8.04929
    > mean(MM1_Queue_V())
    [1] 7.015979
    > mean(MM1_Queue_V())
    [1] 8.922044
    > mean(MM1_Queue_V())
    [1] 8.043148
    > mean(MM1_Queue_V())
    [1] 7.250367
    > mean(MM1_Queue_V())
    [1] 10.7721
    > mean(MM1_Queue_V())
    [1] 6.290644
    > mean(MM1_Queue_V())
    [1] 12.91079

Now we have to face two related questions:

  How can we quantify the accuracy of estimates for a given sample size?

  How can we find the sample size to obtain a given degree of accuracy?

Statistical inference provides us with a useful tool to answer the first question, namely, confidence intervals; it is fairly easy to reverse-engineer a confidence interval in order to estimate the sample size, i.e., the number of replications, that we need to achieve a selected degree of accuracy. Confidence intervals are clearly related to parameter estimation, which has already been dealt with in Chapter 4. However, we want to show a few potential pitfalls and mistakes that are incurred if one applies those deceptively simple formulas without due care; we provide a couple of enlightening examples in Section 7.1. Then, in Section 7.2, we illustrate how sample sizes may be related with measures of estimate precision. Following common practice, we have introduced confidence intervals with reference to a mean (better said, expected value). However, what we have to estimate is not always an expected value. For instance, we often need estimates of probabilities or quantiles. It is true that a probability is the expected value of an indicator function, but since its value is constrained in the interval [0, 1], some care may be needed. As to quantiles, when we estimate quantiles for small or large probability levels, as is common in risk management, we may face a difficult task. We address the related issues in Section 7.3. The essential message of this chapter, which we summarize in Section 7.4, is both good and bad news: Monte Carlo methods enjoy considerable flexibility and generality, but they are inherently inefficient. By pointing this out, we lay down the motivation for variance reduction strategies and low-discrepancy sequences, which are dealt with in Chapters 8 and 9, respectively.

# 7.1 Pitfalls in output analysis

We know from Chapter 4 that, given a sample { _X_ 1, _X_ 2,..., _X_ _n_ } of _n_ i.i.d. variables, we may estimate sample mean and sample variance:

Given the purpose of this chapter, here we point out the dependence of the above statistics on the sample size _n_. Then, elementary inferential statistics suggests the confidence interval

(7.1)

at confidence level 1 − α/2, where _z_ 1−α/2 is the quantile of the standard normal distribution corresponding to the selected confidence level. As we have already noted, we use the quantile of the normal distribution, since in Monte Carlo simulation the sample size is typically large enough to warrant its use rather than the quantile of the _t_ distribution.

Then, it should be an easy matter to use R to find a confidence interval for the average waiting time in the _M/M_ /1 queue. Given the variability that we have observed, we expect quite large confidence intervals:

    > set.seed(55555)
    > as.numeric(t.test(MM1_Queue_V())$conf.int)
    [1] 7.886045 8.212535
    > as.numeric (t. test (MM1_Queue_V ()) $conf.int)
    [1] 6.881556 7.150402
    > as.numeric(t.test(MM1_Queue_V())$conf.int)
    [1] 8.739323 9.104764
    > as.numeric(t.test(MM1_Queue_V())$conf.int)
    [1] 7.876257 8.210038
    > as.numeric(t.test(MM1_Queue_V())$conf.int)
    [1] 7.077575 7.423160
    > as.numeric(t.test(MM1_Queue_V())$conf.int)
    [1] 10.5591 10.9851
    > as.numeric(t.test(MM1_Queue_V())$conf.int)
    [1] 6.167307 6.413980

We clearly notice a disturbing pattern: The extreme points of the confidence intervals do jump around, as we expected, but the confidence intervals look overly "confident," as they are rather narrow and do not overlap with each other. In other words, it seems that we are overstating the precision of the estimate. In fact, what we are doing is _dead wrong_ , and we are actually making three mistakes:

  We are using a formula assuming independence of observations, whereas Lindley's recursion clearly shows that the waiting time _W_ _j_ depends on waiting time _W_ _j_ −1, which makes intuitive sense: If we wait in a queue for a long time, the customer behind us will wait a for a long time, too.

  The random variables are not independent, but they are not even identically distributed: The first waiting time is clearly zero, as the system is initially empty. There is an initial transient phase, and we should wait for the system to reach a steady state before starting to collect statistics. This is not too relevant for a single queue, but it is for a complex system of interacting queues.

  We are using a confidence interval that is meant for normal samples, but there is no reason to believe that waiting times are normally distributed.

By far, the most serious mistake is the first one. Indeed, writing the confidence interval as in Eq. (7.1) is somewhat misleading, as it obscures the fact that it depends on the standard error of estimate, i.e., the standard deviation of the sample mean. The standard deviation of the sample mean can be written as   only under the assumption of independence (actually, lack of correlation is enough). Unfortunately, there is a strong positive correlation between consecutive waiting times, especially when the utilization rate is high; as a consequence, the familiar formula severely understates variance, and this leads to small, but wrong confidence intervals. To clearly see the point, let us write the variance of the average waiting time explicitly:

where σ2 is the variance of each waiting time _W_ _i_ , and ρ _ij_ is the correlation between the waiting times of customers _i_ and _j_. We see that, when there is a significant positive correlation, the familiar formula σ2/ _n_ may grossly understate the variance of the sample mean. The strength of the correlation may be checked by the following snapshot:

    > set.seed(55555)
    > W1 <- MM1_Queue_V()
    > W2 <- MM1_Queue_V(lambda=1,mu=2)
    > par(mfrow=c(1,2))
    > acf(W1, main="utilization = 90.91%")
    > acf(W2, main="utilization = 50%")
    > par(mfrow=c(1, 1))

Running this code produces the two autocorrelation functions2 for waiting times, displayed in Fig. 7.2. The autorrelation is quite significant for a high utilization rate, whereas it fades away quicker when utilization rate is 50%. This explains why we cannot compute confidence intervals in a standard way. But then, what can we do? In queueing simulations the batch method is often used. The idea is to simulate batches of consecutive customers, collecting sample means. For instance, we may build batches of 10,000 customers and gather the batch means

**FIGURE 7.2** **Impact of server utilization on the autocorrelation of waiting times.**

The batch mean  1 includes customers from 1 to 10,000, the batch mean  2 includes customers from 10,001 to 20,000, etc. If batches are long enough, the batch means are practically uncorrelated; this can be checked by plotting the ACF. Furthermore, since they are sums of many waiting times, we can hope that their distribution will be closer to a normal, thus providing further justification for the use of standard confidence intervals. We should also mention that batching observations and calculating confidence intervals on the basis of batch means may also be useful in financial applications, when there are concerns about lack of normality in the observations.

## 7.1.1 BIAS AND DEPENDENCE ISSUES: A FINANCIAL EXAMPLE

The queueing example of the previous section is a nice and simple example to illustrate pitfalls in using standard confidence intervals without due care but, admittedly, is not quite interesting from a financial point of view. So, let us consider the pricing of an _as-you-like-it_ option (also known as _chooser_ option)3. The option is European-style and has maturity _T_ 2. At time _T_ 1 < _T_ 2 you may choose if the option is actually a call or a put; the strike price _K_ is fixed at time _t_ = 0. Clearly, at time _T_ 1 we should compare the values of the two options, put and call, and choose the more valuable one. This can be done by using the Black–Scholes–Merton (BSM) formula to evaluate the price of call and put options with initial underlying price _S_ ( _T_ 1) and time to maturity _T_ 2 − _T_ 1. This means that, conditional on _S_ 1 = _S_ ( _T_ 1), we may find the exact expected value of the payoff at time _T_ 2, under the risk-neutral probability. What's more, it turns out that it is actually possible to price a chooser option exactly, at least in the BSM world. To begin with, let us find the critical price _S_ 1* for which we are indifferent between a call and a put option; to this aim, we just need to consider put–call parity at time _t_ = _T_ 1:4

Now let us consider the strike price

and a portfolio consisting of:

**1.** A vanilla call option with maturity _T_ 2 and strike _K_

**2.** A vanilla put option with maturity _T_ 1 and strike _K_ *

Now, at time _T_ 1 there are two kinds of scenario:

  If _S_ 1 > _K_ *, then the put expires worthless and we keep the call. Therefore, at maturity, we will receive the payoff associated with the call, which indeed corresponds to the optimal choice if _S_ 1 > _K_ *.

  If _S_ 1 < _K_ *, then the put is exercised and at time _T_ 1 we have a position

At time _T_ 2 the position is worth

Therefore, at maturity, we receive the payoff associated with the put, which indeed corresponds to the optimal choice if _S_ 1 < _K_ *.

The code to price the chooser option exactly is given in Fig. 7.3 and relies on functions implementing the BSM formula for vanilla calls and puts. It is instructive to compare the price of the chooser option against the two vanilla options:

**FIGURE 7.3** **Code to price an as-you-like-it (chooser) option exactly in the BSM world.**

    > S0 <- 50
    > K <- 50
    > r <- 0.05
    > T1 <- 2/12
    > T2 <- 7/12
    > sigma <- 0.4
    > EuVanillaCall(S0,K,T2,r,sigma)
    [1] 6.728749
    > EuVanillaPut(S0,K,T2,r,sigma)
    [1] 5.291478
    > AYLI(S0,K,r,T1,T2,sigma)
    [1] 9.267982

As expected, the possibility of choosing may be quite valuable. Even though it is easy to find the price of a simple chooser option, it is extremely instructive to write a pure sampling-based Monte Carlo code. In the present case, this is not quite trivial, as we are not only estimating things: We must make a decision at time _T_ 1. This decision is similar to the early exercise decision we must make with American-style options. To get a feeling for the issues involved, let us consider the scenario tree in Fig. 7.4, where we should associate a price of the underlying asset with each node. Starting from the initial node, with price _S_ 0, we generate four observations of price _S_ ( _T_ 1), and for each of these, we sample three prices _S_ ( _T_ 2). We have 4 × 3 = 12 scenarios, but they are tree-structured. We need this structure, because the decision at time _T_ 1 (either we like the put or we like the call) must be the same for all scenarios departing from each node at time _T_ 1. Without this structure, our decisions would be based on perfect foresight about the future price at time _T_ 2. This nonanticipativity concept is fundamental in dynamic stochastic optimization and in pricing American-style options, as we shall see in Sections 10.5.2 and 11.3.

**FIGURE 7.4** **Scenario tree for the as-you-like-it option.**

A crude Monte Carlo code to price the option is displayed in Fig. 7.5. Here NRepl1 is the number of scenarios that we branch at time _T_ 1 and NRepl2 is the number of branches at time _T_ 2, for each node at time _T_ 1; hence, the overall number of scenarios is the product of NRepl1 and NRepl2. The vector DiscountedPayoffs has a size corresponding to the overall number of scenarios. For each node at _T_ 1, which is generated as usual with geometric Brownian motion, we generate nodes at time _T_ 2, and we compare the two estimates of expected payoff that we are going to earn if we choose the call or the put, respectively. Then, we select one of the two alternatives and we fill a corresponding block (of size NRepl2) in the vector of discounted payoffs. Finally, we compute sample mean and confidence intervals as usual.

**FIGURE 7.5** **Crude Monte Carlo code to price an as-you-like-it option.**

Now, let us run the simulator a few times, comparing the confidence intervals that we obtain against the exact price (9.267982):

    > NRepl1 <- 1000
    > NRepl2 <- 1000
    > set.seed(55555)
    > AYLIMC(S0,K,r,T1,T2,Sigma,NRepl1,NRepl2)$conf.int
    [1] 9.246557 9.289328
    > AYLIMC(S0,K,r,T1,T2,sigma,NRepl1,NRepl2)$conf.int
    [1] 9.025521 9.067313
    > AYLIMC(S0,K,r,T1,T2,sigma,NRepl1,NRepl2)$conf.int
    [1] 9.408449 9.452174
    > AYLIMC(S0,K,r,T1,T2,sigma,NRepl1,NRepl2)$conf.int
    [1] 9.260760 9.303662
    > AYLIMC(S0,K,r,T1,T2,sigma,NRepl1,NRepl2)$conf.int
    [1] 9.434346 9.478341
    > AYLIMC(S0,K,r,T1,T2,sigma,NRepl1,NRepl2)$conf.int
    [1] 9.144339 9.187482

Since we are computing a 95% confidence interval, we would expect to miss the correct value 1 time out of 20 on the average. However, it seems that this happens a tad too often and that we are overstating the reliability of Monte Carlo estimates. Indeed, it is not just a matter of sampling errors, as there are a few things that are dead wrong (again!) in the code above:

  The replications are not really independent: we use _all_ of the replications branching from a node at time _T_ 1 to decide for a call or a put.

  There is an unclear source of bias in the estimates. On the one hand, there is a low bias since we are not really deciding optimally between the call and the put; on the other hand we are relying on a limited set of scenarios, rather than the true continuous distribution; thus, we are possibly facing less uncertainty in the sample than we would do in the true world.

Once again, we see that independence among replications is not to be taken for granted. Furthermore, we also observe that when a Monte Carlo simulation involves some decision making, bias issues may arise. This is quite relevant, e.g., when pricing American-style options, as we will see in Section 11.3.1.

# 7.2 Setting the number of replications

As we have mentioned, we may try to quantify the quality of our estimator by considering the expected value of the squared error of estimate:

where σ2 may be estimated by the sample variance _S_ 2( _n_ ). Note that the above formula holds if we assume that the sample mean is an unbiased estimator of the parameter μ we are interested in. Clearly, increasing the number _n_ of replications improves the estimate, but how can we reasonably set the value of _n_?

Suppose that we are interested in controlling the _absolute_ error in such a way that, with probability (1 − α),

where β is the maximum acceptable tolerance. In fact, the confidence interval (7.1) is just built in such a way that

where we denote the half-length of the confidence interval by

This implies that, with probability 1 − α, we have

Hence, linking _H_ to β, we should simply run replications until _H_ is less than or equal to the tolerance β, so that the number _n_ must satisfy

(7.2)

Actually, we are chasing our tail a bit here, since we cannot estimate the sample variance _S_ 2( _n_ ) until the number _n_ has been set. One way out is to run a suitable number _k_ of pilot replications, in order to come up with an estimate _S_ 2( _k_ ). Then we may apply Eq. (7.2) using _S_ 2( _k_ ) to determine _n_. After running the _n_ replications, it is advisable to check that Eq. (7.2) holds with the new estimate _S_ 2( _n_ ). Alternatively, we may simply add replications, updating the sample variance, until the criterion is met; however, with this approach we do not control the amount of computation we are willing to spend.

If you are interested in controlling the _relative_ error, so that

holds with probability (1 − α), things are a little more involved. The difficulty is that we may run replications until the half-length _H_ satisfies

but in this inequality we are using the estimate  ( _n_ ) rather than the unknown parameter μ. Nevertheless, if the inequality above holds, we may write

(7.3)

(7.4)

where inequality (7.3) follows from the triangle inequality and Eq. (7.4) is obtained by a slight rearrangement. Therefore, we conclude that if we proceed without care, the actual relative error that we obtain is bounded by γ/(1 − γ), which is larger than the desired bound γ. Therefore, we should choose _n_ such that the following tighter criterion is met:

(7.5)

where

Again, we should run a few pilot replications in order to get a preliminary estimate of the sample variance _S_ 2( _n_ ).

# 7.3 A world beyond averages

The standard form of a confidence interval for the mean of a normal population is a cornerstone of inferential statistics. The unfortunate side effect is the tendency to use it in settings for which the recipe is not quite appropriate. Consider the problem of estimating the probability

where **X** is a possibly high-dimensional vector of random variables. It is natural to sample i.i.d. observations **X** _i_ , _i_ = 1,..., _n_ , and collect the random variables

These are i.i.d. observations of a Bernoulli random variable with parameter π, and it is natural to build an estimator

(7.6)

Then, _np_ is just a binomial variable with parameters _n_ and π, and

We see that there is an obvious link between expected value and variance, and it would make little sense to use the usual sample variance _S_ 2. In fact, building a confidence interval for π calls for same care. However, if the sample size _n_ is large enough, we may take advantage of the fact that the binomial distribution is a sum of i.i.d. variables and tends to a normal distribution, thanks to the central limit theorem. In other words, the statistic

is approximately N(0, 1), which justifies the approximate confidence interval

(7.7)

Hence, at the very least, we should use the standard confidence interval by using a sensible estimate of variance. More refined methods are available in the literature, but R solves the problem by offering an appropriate function, binom.test.

Another delicate estimation problem concerns quantiles. This is relevant, for instance, in the estimation of risk measures like value-at-risk, as we shall see in Chapter 13. In principle, the task is not too hard and can be carried out as follows:

**1.** Use Monte Carlo to generate the output sample { _X_ 1,..., _X_ _n_ }.

**2.** Build the empirical CDF

(7.8)

where **1** {·} is the indicator function for an event.

**3.** Find the generalized inverse (see below) of the empirical CDF and return the estimated quantile at probability level α

Since the empirical CDF is piecewise constant, with jumps of size 1/ _n_ , it cannot be inverted as usual, and we resort to the generalized inverse. In this case this is just accomplished by sorting the observations _X_ _i_ , which yields the order statistics _X_ ( _i_ ), and pick the observation in position

(7.9)

Alternative procedures can be devised, possibly based on linear interpolation, but the deep trouble is that if we are estimating extreme quantiles, for small or large values of α, we may get rather unreliable estimates. Hence, we need a way to build a confidence interval. The theory here is far from trivial (see the references at the end of the chapter), but there is a central limit theorem for quantiles, stating that the statistic

converges in distribution to a normal random variable N(0, σ2), where

and _f_ _X_ (·) is the PDF corresponding to the true CDF _F_ _X,n_ (·). This suggests the confidence interval

The problem is that, clearly, _f_ _X_ ( _q_ α) itself must be estimated, and it can be very small, resulting in large confidence intervals. In Section 13.4 we shall see an example of how the difficulty can be circumvented by recasting the problem a bit and using variance reduction methods.

# 7.4 Good and bad news

Equation (7.1) shows that the half-length of the confidence interval is determined by the ratio

i.e., by the variability in the output performance measure and the sample size. This is both good and bad news:

  On the one hand, we notice that no role is played by the problem dimensionality, i.e., by how many random variables are involved in the simulation or, if you prefer, by the dimension of the space in which we are computing an integral. Actually, the dimension does play a role in terms of computational effort, and it typically creeps into the variability σ itself. Nevertheless, this is certainly good news and it contributes to explain why Monte Carlo methods are widely used.

  On the other hand, the rate of improvement in the quality of our estimate, i.e., the rate of decrease of the error, is something like _O_ (1/√ _n_ ). Unfortunately, the square root is a concave function, and an economist would say that it features diminishing marginal returns. In practice, this means that the larger the sample size, the better, but the rate of improvement is slower and slower as we keep adding observations.

To appreciate the latter point, let us assume that with a sample size _n_ we have a certain precision, and that we want to improve that by one order of magnitude. For instance, we might find a confidence interval for an option price like

in which it seems that we got the first decimal digit right, but we also want to get the cents right. This means that we should divide the interval half-length by 10, which implies that we need 100 × _n_ replications. Thus, a brute-force Monte Carlo simulation may take quite some amount of computation to yield an acceptable estimate. One way to overcome this issue is to adopt a clever sampling strategy in order to reduce the variance σ2 of the observations themselves; the other one is to adopt a quasi–Monte Carlo approach based on low-discrepancy sequences. The next two chapters pursue these two strategies, respectively.

# For further reading

  An excellent, yet readable chapter on simulation output analysis can be found in [5].

  For a higher level and more general treatment, you may refer to [1, Chapter 3].

  For the estimation of probabilities, a standard reference is [2]; this topic and other things are covered in [3].

  See [4] for issues related to quantile estimation.

# References

1 S. Asmussen and P.W. Glynn. _Stochastic Simulation: Algorithms and Analysis_. Springer, New York, 2010.

2 C.J. Clopper and E.S. Pearson. The use of confidence or fiducial limits illustrated in the case of the binomial. _Biometrika_ , 26:404–413, 1934.

3 W.J. Conover. _Practical Nonparametric Statistics_ (3rd ed.). Wiley, Hoboken, NJ, 1999.

4 P.W. Glynn. Importance sampling for Monte Carlo estimation of quantiles. In _Proceedings of the 2nd International Workshop on Mathematical Methods in Stochastic Simulation and Experimental Design_ , pp. 180–185, 1996. See http://www.Stanford.edu/~glynn/papers/1996/G96.html.

5 A.M. Law and W.D. Kelton. _Simulation Modeling and Analysis_ (3rd ed.). McGraw-Hill, New York, 1999.

1 The memoryless property of the exponential distribution is discussed in Section 3.2.2.

2 We have introduced the autocorrelation function in Section 3.6.

3 The background on risk-neutral option pricing is given in Section 3.9.

4 See Theorem 3.18.

# Chapter Eight

# Variance Reduction Methods

In Chapter 7 we have discussed output analysis, and we have seen that an obvious way to improve the accuracy of an estimate  ( _n_ ) based on a sample of _n_ replications _X_ _i_ , _i_ = 1,..., _n_ , is to increase the sample size _n_ , since Var( ( _n_ )) = Var( _X_ _i_ )/ _n_. However, we have also seen that the width of a confidence interval, assuming that replications are genuinely i.i.d., decreases according to a square-root law involving  , which is rather bad news. Increasing the number of replications is less and less effective, and this brute force strategy may result in a remarkable computational burden. A less obvious strategy is to reduce Var( _X_ _i_ ). At first sight, this seems like cheating, as we have to change the estimator in some way, possibly introducing bias. The variance reduction strategies that we explore in this chapter aim at improving the efficiency of Monte Carlo methods, sometimes quite dramatically, without introducing any bias. This means that, given a required accuracy, we may reduce the computational burden needed to attain it; or, going the other way around, we may improve accuracy for a given computational budget. In Chapter 9 we also consider another strategy, which consists of using deterministic low-discrepancy sequences to drive sampling, rather than pseudorandom numbers.

Some variance reduction strategies are just technical tricks to improve sampling, and they can even be automated in software tools. Other strategies are based on general principles, but require a high degree of customization to work. In some cases, they replace part of the numerical integration process with a partially analytical integration. As the reader may imagine, the more sophisticated strategies require a lot of domain-specific knowledge, may be rather difficult to conceive, but are much more effective. We begin in Section 8.1 with antithetic sampling, which is arguably the simplest strategy and serves well to introduce the topic of this chapter. Antithetic sampling is related to common random numbers, the subject of Section 8.2, which is a bit different in nature, as it deals with differences of random variables; we will actually appreciate common random numbers in Chapter 12 on estimating sensitivities, but they are best introduced just after antithetic sampling. Antithetic sampling and common random numbers, just like low-discrepancy sequences, work on the input stream of numbers feeding the simulation. Control variates, on the contrary, are based on an augmented estimator, which incorporates some more analytical knowledge that we may have at our disposal. As we shall see in Section 8.3, control variates are quite akin to linear regression models, as they try to take advantage of correlation between random variables. Another helpful tool is the conditional variance formula; in Sections 8.4 and 8.5 we will show how it can be exploited in conditional Monte Carlo and stratified sampling, respectively. Finally, a change of probability measure is the principle behind the importance sampling approach introduced in Section 8.6. Intuitively, the idea is to make rare but significant events more likely.

It is our aim to introduce the above strategies by simple, yet concrete examples. Therefore, in this chapter we will use toy integration problems, estimating π and pricing a vanilla call option, for illustration purposes. In practice, we certainly need no Monte Carlo method for these problems, but it is useful to get a basic understanding of variance reduction strategies by trying them on elementary examples, where we may also afford the luxury of comparing numerical results against the exact ones. Later, we will tackle more interesting applications, mostly in Chapter 11 on option pricing; we will also see an application to risk measurement in Chapter 13. Extremely sophisticated customization may be needed to cope with these more realistic problems.

# 8.1 Antithetic sampling

The antithetic sampling approach is very easy to apply and does not rely on any deep knowledge about the specific problem we are tackling. In crude Monte Carlo, we generate a sample consisting of independent observations. However, inducing some correlation in a clever way may be helpful. Consider a sequence of paired replications ( _X_ 1( _i_ ), _X_ 2( _i_ )), _i_ = 1,..., _n_ :

We allow for correlation within each pair, but observations are otherwise "horizontally" independent. This means that if we pick any element of a pair, this is independent from any element of other pairs. Formally, _X_ _j_ ( _i_ 1) and _X_ _k_ ( _i_ 2) are independent however we choose _j, k_ = 1, 2, provided _i_ 1 ≠ _i_ 2. Thus, the pair-averaged observations

are independent and can be used to build a confidence interval as usual. Note that we cannot build a confidence interval without averaging within pairs, as we do not require "vertical" independence. Let us consider the sample mean  ( _n_ ) based on the pair-averaged observations _X_ ( _i_ ); its variance is given by

(8.1)

In the first line we use independence among pairs, in the second line we account for covariance within a pair, and finally we arrive at Eq. (8.1), which shows the potential role of correlation. In order to reduce the variance of the sample mean, we should take negatively correlated replications within each pair. Needless to say, it is not fair to claim that variance is reduced because of the factor 2 _n_ , since we have affectively doubled the sample size; we must compare Eq. (8.1) against the variance Var( _X_ _i_ )/(2 _n_ ) of an independent sample of size 2 _n_. Now the issue is how we can induce negative correlation. Each observation _X_ ( _i_ )1,2 is obtained by generating possibly many random variates, and all of them ultimately depend on a stream of uniform pseudorandom numbers. Hence, we may consider the idea of inducing a strong negative correlation between input streams of random numbers. This is easily achieved by using a random number sequence { _U_ _k_ } for the first replication in each pair, and then {1 − _U_ _k_ } in the second one. Since the input streams are negatively correlated, we hope that the output observations will be, too.

###   Example 8.1 Using antithetic sampling in integration

In Example 2.1 we used plain Monte Carlo integration to estimate

A small sample consisting of only 100 observations, as expected, does not provide us with a reliable estimate:

    > set.seed(55555)
    > X <- exp(runif(100))
    > T <- t.test(X)
    > T$estimate
    mean of x
    1.717044
    > CI <- as.numeric(T$conf.int)
    > CI
    [1] 1.621329 1.812759
    > CI [2]-CI[1]
    [1] 0.1914295
    > (CI[2]-CI[1])/2/(exp(1)-1)
    [1] 0.05570375

Antithetic sampling is easily accomplished here: We must store random numbers and take their complements to 1. In order to have a fair comparison, we consider 50 antithetic pairs, which entails sampling 100 function values as before:

    > set.seed(55555)
    > U1 <- runif(50)
    > U2 <- 1-U1
    > X <- 0.5*(exp(U1)+exp(U2))
    > T <- t.test(X)
    > T$estimate mean of x
    1.71449
    > CI <- as.numeric(T$conf.int)
    > CI
    [1] 1.699722 1.729258
    > CI[2]-CI[1]
    [1] 0.02953576
    > (CI[2]-CI[1])/2/(exp(1)-1)
    [1] 0.008594562

Now the confidence interval is much smaller and, despite the limited sample size, the estimate is fairly reliable.

The antithetic sampling method looks quite easy to apply and, in the example above, it works pretty well. May we always expect a similar pattern? Unfortunately, the answer is no. The reason why the approach works pretty well in the example is twofold:

**1.** There is a strong (positive) correlation between _U_ and _e_ _U_ over the interval [0, 1], because the function is almost linear there. This means that a strong correlation in the simulation input is preserved and turns into a strong correlation in the simulation output. We should not expect impressive results with more complicated and nonlinear functions.

**2.** Another reason is that the exponential function is monotonic increasing. As we shall see shortly, monotonicity is an important condition for antithetic sampling.

It is instructive to consider a simple example, showing that antithetic sampling may actually backfire and increase variance.

###   Example 8.2 A counterexample to antithetic sampling

Consider the function _h_ ( _x_ ), defined as

and suppose that we want to take a Monte Carlo approach to estimate

The function we want to integrate is obviously a triangle with both basis and height equal to 1; note that, unlike the exponential function of Example 8.1, this is not a monotone function with respect to _x_. It is easy to compute the integral as the area of a triangle:

Now let

be an estimator based on two independent uniform random variables _U_ 1 and _U_ 2, and let

be the pair-averaged sample built by antithetic sampling. We may compare the variances of the two estimators:

The difference between the two variances is

But in this case, due to the shape of _h_ , we have

and

Therefore,   and   0. In fact, antithetic sampling actually increases variance in this case, and there is a trivial explanation. The two antithetic observations within each pair have the same value _h_ ( _U_ ) = _h_ (1 − _U_ ), and so Cov[ _h_ ( _U_ ), _h_ (1 − _U_ )] = Cov[ _h_ ( _U_ ), _h_ ( _U_ )] = Var[ _h_ ( _U_ )]. In this (pathological) case, the variance of the single observation is doubled by applying antithetic sampling.

What is wrong with Example 8.2? The variance of the antithetic pair is actually increased due to the nonmonotonicity of _h_ ( _x_ ). In fact, while it is true that the random numbers { _U_ _i_ } and {1 − _U_ _i_ } are negatively correlated, there is no guarantee that the same holds for _X_ _i_ (1) and _X_ _i_ (2) in general. To be sure that the negative correlation in the input random numbers yields a negative correlation in the observed output, we must require a monotonic relationship between them. The exponential function is a monotonic function, but the triangle function of the second example is not.

Another side of the coin is related to the mechanism we use to generate random variates. The inverse transform method is based on the CDF, which is a monotonic function; hence, there is a monotonic relationship between the input random numbers and the random variates generated. This is not necessarily the case with the acceptance-rejection method or the Box–Muller method, which were discussed in Chapter 5. Luckily, when we need normal variates, we may simply generate a sequence  _i_ ~ N(0, 1) and use the sequence −  _i_ ; for the antithetic sample.

In order to illustrate antithetic sampling with normal variates, and to see a more concrete example, let us consider the use of antithetic sampling to price a simple vanilla call option. We have seen how such a simple option can be priced using Monte Carlo sampling. It is easy to modify the code in Fig. 6.4 to introduce antithetic sampling; we just need to store a vector of normal variates to use them twice. The resulting code is shown in Fig. 8.1. The following snapshot checks variance reduction in the case of an at-the-money option:

**FIGURE 8.1** **Pricing a vanilla call option with antithetic sampling.**

    > S0 <- 50
    > K <- 50
    > sigma <- 0.4
    > r <- 0.05
    > T <- 5/12
    > numRepl <- 50000
    > exact <- EuVanillaCall(S0,K,T,r,sigma); exact
    [1] 5.614995
    > set.seed(55555)
    > out1 <- BlsMCNaive(S0,K,T,r,sigma,numRepl); out1
    $estimate
    [1] 5.568167
    $conf.int
    [1] 5.488962 5.647371
    > set.seed(55555)
    > out2 <- BlsMCAV(S0,K,T,r,sigma,numRepl/2); out2
    $estimate
    [1] 5.596806
    $conf.int
    [1] 5.533952 5.659661

Note that, for the sake of a fair comparison, we divide the number of replications by two when using antithetic sampling. Indeed, there is a slight reduction in variance, but it is far from impressive (and, anyway, one run does not mean too much). The situation looks a bit better for a deeply in-the-money option:

    > K <- 25
    > exact <- EuVanillaCall(S0,K,T,r,sigma); exact
    [1] 25.52311
    > set.seed(55555)
    > out1 <- BlsMCNaive(S0,K,T,r,sigma,numRepl); out1
    $estimate
    [1] 25.4772
    $conf.int
    [1] 25.36299 25.59142
    > set.seed(55555)
    > out2 <- BlsMCAV(S0,K,T,r,sigma,numRepl/2); out2
    $estimate
    [1] 25.51441
    $conf.int
    [1] 25.48482 25.54401

A possible explanation is that in this second case we are sampling the payoff function where the nonlinearity plays a lesser role. We leave a serious computational study to the reader, but we should not be surprised by the fact that antithetic sampling, which is almost a simple programming trick, does not necessarily produce impressive improvements. Nevertheless, the idea does work well in some cases, and sometimes it can also be fruitfully integrated with the approaches that we describe in the following.

# 8.2 Common random numbers

The common random numbers technique is very similar to antithetic sampling, but it is applied in a different situation. Suppose that we use Monte Carlo simulation to estimate a value depending on a parameter α. In formulas, we are trying to estimate something like

where we have emphasized randomness by ω, which refers to random events underlying random variables. We could also be interested in evaluating the sensitivity of this value with respect to the parameter α:

This would be of interest when dealing with option sensitivities beyond the Black–Scholes–Merton model. Clearly, we cannot compute the derivative analytically; otherwise, we would not use simulation to estimate the function _h_ (·) in the first place. So, the simplest idea would be using simulation to estimate the value of the finite difference,

for a small value of the increment δα. However, what we can really do is to generate observations of the difference

and to estimate its expected value. Unfortunately, when the increment δα is small, it is difficult to tell if the difference we observe is due to random noise or to the variation in the parameter α. A similar problem arises when we want to compare two portfolio management policies on a set of scenarios; in this case, too, what we need is an estimate of the expected value of the difference between two random variables. In this setting, is it sensible to compare the two performances on the basis of different sets of scenarios?

Let us abstract a little and consider the difference of two random variables

where, in general, E[ _X_ 1] ≠ E[ _X_ 2], since they come from simulating two different systems, possibly differing only in the value of a single parameter. By Monte Carlo simulation we get a sequence of independent random variables,

and use statistical techniques to build a confidence interval for E[ _X_ 1 − _X_ 2]. To improve our estimate, it would be useful to reduce the variance of the sampled _Z_ _j_. If we do not assume independence between _X_ 1, _j_ and _X_ 2, _j_ (for the same _j_ ), we find something quite similar to antithetic sampling, with a change in sign:

Then, in order to reduce variance, we may try inducing a positive correlation between _X_ 1 _j_ and _X_ 2 _j_. This can be obtained by using the same stream of random numbers in simulating both _X_ 1 and _X_ 2. The technique works much like antithetic sampling, and the same monotonicity assumption is required to ensure that the technique does not backfire. We will see an application of these concepts in Section 12.1, where we apply Monte Carlo sampling to estimate option price sensitivities.

# 8.3 Control variates

Antithetic sampling and common random numbers are two very simple techniques that, provided the monotonicity assumption is valid, do not require much knowledge about the systems we are simulating. Better results might be obtained by taking advantage of deeper, domain-specific knowledge. Suppose that we want to estimate θ = E[ _X_ ], and that there is another random variable _Y_ , with a _known_ expected value ν, which is somehow correlated with _X._ Such a case occurs when we use Monte Carlo simulation to price an option for which an analytical formula is not known: θ is the unknown price of the option, and ν could be the price of a corresponding vanilla option. The variable _Y_ is called the _control variate_ and represents additional, domain-specific knowledge. The correlation _Y_ may be exploited by adopting the controlled estimator

where _c_ is a parameter that we must choose. Intuitively, when we run a simulation and observe that our estimate of _E_ [ _Y_ ] is too large:

Then, we may argue that the estimate   should be increased or reduced accordingly, depending on the sign of the correlation between _X_ and _Y_. For instance, if correlation is positive and  , we should reduce  , because if the sampled values for _Y_ are larger than their average, the sampled values for _X_ are probably too. The contrary applies when correlation is negative. Indeed, it is easy to see that

(8.2)

(8.3)

The first formula says that the controlled estimator is, for any choice of the control parameter _c_ , an unbiased estimator of θ. The second formula suggests that by a suitable choice of _c_ , we could reduce the variance of the estimator. We could even minimize the variance by choosing the optimal value for _c_ :

(8.4)

in which case we find

where ρ _XY_ is the correlation between _X_ and _Y_. Note that the sign of _c_ depends on the sign of this correlation, in a way that conforms to our intuition.

The form of the optimal control parameter in Eq. (8.4) looks suspiciously like the slope coefficient in a linear regression model.1 To see the connection, let us consider the estimation error

The variance of the random variable η may be large for difficult estimation problems. In linear regression we aim at explain variability of _X_ by introducing an explanatory variable _Y_ :

With respect to the usual notation, here we are swapping the roles of _X_ and _Y_. Furthermore, we use only the slope coefficient _b_ in the regression model, since we are centering random variables with respect to their expected values. Apart from these unsubstantial changes, we clearly see that the least-squares estimate of _b_ is what we need to maximize explained variability, which turns into Eq. (8.4) by a slight rearrangement.

In practice, the optimal value of _c_ must be estimated, since Cov( _X_ , _Y_ ) and possibly Var( _Y_ ) are not known. This may be accomplished by running a set of pilot replications to estimate them. It would be tempting to use these replications both to select _c_ * and to estimate θ, in order to save precious CPU time. However, doing so would induce some bias in the estimate of θ, since in this case _c_ * cannot be considered as a given number, but a random variable depending on _X_ itself. This invalidates the arguments leading to Eqs. (8.2) and (8.3). Therefore, unless suitable statistical techniques (e.g., jackknifing) are used, which are beyond the scope of this book, the pilot replications should be discarded.

###   Example 8.3 Using control variates in integration

We can illustrate control variates by referring again to the problem of Example 8.1, i.e., the integration of the exponential function on the unit interval,

A possible control variate in this case is the driving uniform variable _U_ itself, with an expected value of 0.5. We might even afford the luxury of computing _c_ * exactly, since the variance of the control variate is  , and with a little effort we may actually calculate the correlation between _U_ and _e_ _U_. Since this is not true in general, let us stick to statistical estimates only. The code in Fig. 8.2 produces the following output:

**FIGURE 8.2** **Using a control variate in a toy integration problem.**

    > as.numeric(out$estimate)
    [1] 1.713364
    > as.numeric(out$conf.int)
    [1] 1.646735 1.779992
    > as.numeric(outCV$estimate)
    [1] 1.713169
    > as.numeric(outCV$conf.int)
    [1] 1.703453 1.722885
    > cor(U1,exp1)
    [1] 0.9938869

A comparison against the naive estimator shows that the reduction in variance is quite remarkable, and it is easily explained by the strength in the correlation. We may not expect such a result in general, unless a strong control variate is available.

As a reality check, let us try the vanilla call example again. A natural control variate is the price of the underlying asset at maturity, whose expected value under the risk-neutral measure is just _S_ 0 _e_ _r_ _T_. The code is shown in Fig. 8.3. Let us try this approach and compare it against naive Monte Carlo:

**FIGURE 8.3** **Pricing a vanilla call option by control variates.**

    > S0 <- 50
    > K <- 50
    > sigma <- 0.4
    > r <- 0.05
    > T <- 5/12
    > numRepl <- 50000
    > numPilot <- 10000
    > exact <- EuVanillaCall(S0,K,T,r,sigma); exact
    [1] 5.614995
    > set.seed(55555)
    > out1 <- BlsMCNaive(S0,K,T,r,sigma,numRepl+numPilot); out1
    $estimate
    [1] 5.570385
    $conf.int
    [1] 5.498177 5.642593
    > set.seed(55555)
    > out2 <- BlsMCCV(S0,K,T,r,sigma,numRepl,numPilot); out2
    $estimate
    [1] 5.585363
    $conf.int
    [1] 5.552163 5.618563

With this problem instance, it seems that control variates is more effective than antithetic sampling. The control variates approach may be generalized to as many control variates as we wish, with a possible improvement in the quality of the estimates. Of course, this requires more domain-specific knowledge and more effort in setting the control parameters. For an impressive result obtained by a clever control variate, see the case of arithmetic and geometric Asian options in Section 11.2.

# 8.4 Conditional Monte Carlo

When faced with the task of calculating an expectation, a common technique in probability theory is the application of conditioning with respect to another random variable. Formally E[ _X_ ] can be expressed as

(8.5)

Equation (8.5) is known as the _law of iterated expectations_ , among other names, and it suggests the use of E[ _X_ | _Y_ ] as an estimator rather than _X_. It is important to realize that this conditional expectation is a random variable, and that this makes sense if, given _Y_ , we have a straightforward recipe to calculate the conditional expectation. What we should do is to sample _Y_ and apply this recipe, and Eq. (8.5) makes sure that we have an unbiased estimate. However, what makes sure that by doing so we can reduce variance? The answer lies in the related conditional variance formula:

(8.6)

We observe that all of the terms in Eq. (8.6) involve a variance and are non-negative. Therefore the conditional variance formula implies the following:

(8.7)

(8.8)

The first inequality states that E[ _X_ | _Y_ ] cannot have a larger variance than the crude estimator _X_ , and it justifies the idea of conditional Monte Carlo, The second inequality is the rationale behind variance reduction by stratification, which is discussed in the next section. Unlike antithetic sampling, variance reduction by conditioning requires some careful thinking and is strongly problem dependent.

To illustrate the approach, let us consider the expected value E[e− _X_ 2], where _X_ is an affine transformation of a Student's _t_ variable with _n_ degrees of freedom:2

In practice, the above transformation is similar to the destandardization of a standard normal, which is replaced by a heavier-tailed variable. If _X_ were normal, things would be much easier. Nevertheless, we may recall that a Student's _t_ variable is related to a normal:

Hence, we may rewrite _X_ in terms of a standard normal _Z_ and a chi-square variable:

Thus, conditional on _Y_ = _y_ , _X_ is normal, with a variance depending on _y_ : _X_ | _Y_ = _y_ ~ N(μ, σ2/ _y_ ). Now, let us consider the conditional expected value

If we introduce ξ2 = σ2/ _y_ , the conditional expectation can be written as

(8.9)

A typical trick of the trade to deal with this kind of integrand functions is completing the square in the argument of the exponential, in order to come up with something related to the PDF of a normal variable:

When we take the exponential of the last expression, we see that the second term does not depend on _x_ and can be taken outside the integral, leading to a multiplicative factor

The first term leads to an integral of a normal density with expected value

and variance

The integral of the density is 1, of course, but this change of variance requires some adjustment in the integral of Eq. (8.9), i.e., division by

Putting it all together, and substituting back ξ2 = σ2/ _y_ , the conditional expectation is

Now, what we have to do in order to apply conditional Monte Carlo is to sample a chi-square variable _Y_ with _n_ degrees of freedom and plug the value in the above conditional expectation. This is carried out by the R code of Fig. 8.4, for μ = 3, σ = 0.5, and _n_ = 5 degrees of freedom. We use rchisq to sample the chi-square distribution. In the code we also apply crude Monte Carlo, using rt to sample the Student's variable, to compare the results. Indeed, the comparison between the two confidence intervals shows the improvement obtained by conditioning:

**FIGURE 8.4** **Dealing with a function of a Student's _t_ variable by conditional Monte Carlo.**

    > as.numeric(out$estimate)
    [1] 0.007806775
    > as.numeric(out$conf.int)
    [1] 0.007472332 0.008141218
    > as.numeric(outC$estimate)
    [1] 0.007683399
    > as.numeric(outC$conf.int)
    [1] 0.007590393 0.007776404

It is also interesting to see how the two sample means evolve when adding replications successively. This may be obtained by cumulating and plotting sums, as is done in the last lines of the R script, which produces the plot in Fig. 8.5. The difference in convergence speed to the correct value is quite evident.

**FIGURE 8.5** **Showing the effect of conditioning on the convergence of estimates.**

# 8.5 Stratified sampling

Stratified sampling, like conditional Monte Carlo, relies on conditioning arguments, but it has a different perspective. We are interested in estimating θ = E[ _X_ ], and let us suppose that _X_ depends on another random variable _Y_ , which may take a finite set of values _y_ _j_ , _j_ = 1,..., _m_ , and has the known PMF:

Using conditioning, we see that

This suggests the idea of sampling _X_ conditional on _Y_ , and then to assemble a stratified estimator:

(8.10)

where _N_ _j_ is the number of independent observations allocated to stratum _j_ , and _X_ _jk_ is the _k_ th observation of _X_ in stratum _j_ , conditional on _Y_ = _y_ _j_. We speak of strata since the values _y_ _j_ induce a partitioning the sample space into a mutually exclusive and collectively exhaustive collection of subsets. It is a good idea to see a simple application of the idea immediately.

###   Example 8.4 A stratified integral

As a simple example of stratification, consider using simulation to compute

In crude Monte Carlo simulation we draw _N_ uniform random numbers _U_ _k_ ~ U(0, 1) and compute the sample mean

An improved estimator with respect to crude sampling may be obtained by partitioning the integration interval [0, 1] into a set of _m_ disjoint subintervals

(8.11)

Strictly speaking, these are not disjoint intervals, but since the probability of singletons is zero, this is not quite relevant. Each event _Y_ = _y_ _j_ corresponds to a random number falling in the _j_ th subinterval; in this case we might choose _p_ _j_ = 1/ _m_. Note that a uniform variable on the unit interval (0, 1) can be mapped into a uniform variable on the intervals of Eq. (8.11) by the affine transformation ( _U_ + _j_ − 1)/ _m_. Hence, for each stratum _j_ = 1,..., _m_ we generate _N_ _j_ random numbers _U_ _k_ ~ U(0, 1) and estimate

Then we build the overall estimator:

Clearly, we may apply the approach only if we know the probabilities _p_ _j_ for each stratum and are able to sample conditionally on _Y_. Assuming this, the following questions arise:

**1.** How can we allocate a budget of _N_ observations into _N_ _j_ observations for each stratum, such that  ?

**2.** How can we have the guarantee that, by doing so, we do reduce the overall variance?

We will give an answer to the first question later. For now, let us assume that we follow a simple proportional allocation policy:

subject to rounding, which we neglect. Let us denote by σ _j_ 2 the conditional variance

Given the independence among replication, we see that

(8.12)

If we assume proportional allocation of observation to strata, we find

where the last inequality follows from the inequality (8.8), which in turn is a consequence of the conditional variance formula (8.6). Hence, by using proportional allocation we may indeed reduce variance. A further improvement can be obtained, at least in principle, by an optimal allocation of observations to strata. This may be obtained by solving the nonlinear programming problem:

To be precise, we should require that the decision variables _N_ _j_ take only integer values. In practice, when _N_ is large enough, we may relax integer variables to continuous values, and then round the optimal solution. If we assume an interior optimal solution, i.e., _N_ _j_ * > 0, we may relax the non-negativity bounds and apply the Lagrange multiplier approach. First, we associate a multiplier λ with the budget constraint and build the Lagrangian function

and then write the stationarity conditions with respect to each decision variable:

This condition implies that _N_ _j_ should be proportional to _p_ _j_ σ _j_ :

Taking the budget constraint into account, we find the optimal budget allocation

and the corresponding minimal variance is

An obvious difficulty in this reasoning is that it relies on the knowledge of the conditional variances σ _j_ 2. Since it is quite unlikely that we know them, in order to apply optimal budget allocation we need some pilot runs to estimate the sample conditional variances:

where

is the sample mean for stratum _j_. Furthermore, we may also build an approximate confidence interval at confidence level (1 − α) as follows:

(8.13)

###   Example 8.5 Estimating π by stratification

In Section 1.1 we considered the use of Monte Carlo methods to estimate π, without much of a success. To improve things, let us consider the integral

Since _x_ 2 \+ _y_ 2 = 1 is the well-known equation describing a unit circle, we see that _I_ is the area of a quarter of the unit circle. Therefore, we have

In Fig. 8.6 we show three R functions to estimate π using the above integral. The first function uses crude Monte Carlo, the second one uses straightforward stratification, and the third one integrates stratification and antithetic sampling. By running the three functions with a sample size 5000 we notice that stratification improves the quality of the estimate:

**FIGURE 8.6** **Using stratification to estimate π.**

    > set.seed(55555)
    > estPiNaiveMC(5000)
    $estimate
    [1] 3.138237
    $conf.int
    [1] 3.113241 3.163233
    > estPiStratified(5000)
    $estimate
    [1] 3.141587
    $conf.int
    [1] 3.116832 3.166343
    > estPiStratifiedAnti(5000)
    $estimate
    [1] 3.141593
    $conf.int
    [1] 3.116838 3.166348
    > pi
    [1] 3.141593

In the functions using stratification, the number of strata is just the number of replications. To be fair, we are cheating a bit when integrating stratification with antithetic sampling, as we are doubling the number of observations. Nevertheless, we see that we get an estimate which is correct within at least five decimals. Actually, the natural objection is that with this approach we are reverting back to integration quadrature on a grid of points. Indeed, as we explain later, we cannot afford the luxury of stratifying with respect to several random variables; nevertheless, using some tricks we may still apply the approach to practically relevant cases.

###   Example 8.6 Option pricing by stratification

Let us apply stratification to option pricing in the simplest case, the vanilla call option. It is natural to stratify with respect to the asset price at maturity, which in turn requires stratification of a standard normal. To accomplish this task, we may stratify a uniform random number as we did in Example 8.4, and then obtain the standard normals by the inverse transform method. This is accomplished by the R code of Fig. 8.7, which produces the following (edited) output:

**FIGURE 8.7** **Pricing a vanilla call option by stratified sampling.**

    < EuVanillaCall(S0,K,T,r,sigma)
    [1] 5.614995
    < BlsMCNaive(S0,K,T,r,sigma,numRepl)
    $estimate
    [1] 5.74592
    $conf.int
    [1] 5.562113 5.929728
    > BsmStrat(S0,K,T,r,sigma,numRepl,numStrats)
    $estimate
    [1] 5.636611
    $conf.int
    [1] 5.614289 5.658932

In all of the above examples, the beneficial effect of stratification is rather evident. One may ask, however, how can we take advantage of stratification in a more relevant situation. After all, by straightforward stratification we are, in a sense, pushing back toward numerical integration on a grid, and it seems rather difficult to stratify with respect to several variables. Indeed, in pricing an option depending on several factors or the prices along a whole sample path, we cannot stratify with respect to all of them. However, we may use some tricks, such as the Brownian bridge of Section 6.2.3, to stratify with respect to some milestone prices, and then fill the gaps in each sample path using naive Monte Carlo.

# 8.6 Importance sampling

Unlike other variance reduction methods, importance sampling is based on the idea of "distorting" the underlying probability measure. Consider the problem of estimating

where **X** is a random vector with joint density _f_ (·). If we know another density _g_ (·) such that _f_ ( **x** ) = 0 whenever _g_ ( **x** ) = 0, we may write

(8.14)

where the notation E _g_ [·] is used to stress the fact that the last expected value is taken with respect to another measure. The ratio _f_ ( **x** )/ _g_ ( **x** ) is used to correct the change in probability measure, and it is typically called a _likelihood ratio_ : When sampling **X** randomly, this ratio will be a random variable.3 The requirement that _f_ ( **x** ) = 0 whenever _g_ ( **x** ) = 0 is needed to avoid trouble with division by zero, but it is just a technicality. The real issue is how to select a density _g_ (·) that results in a significant reduction in variance. Actually, there is no guarantee that variance will be reduced, as a poor choice may well increase variance.

To get a clue, let us introduce the notation

and assume for simplicity that _h_ ( **x** ) ≥ 0. As we have pointed out, there are two possible ways of estimating θ:

where _h_ *( **X** ) = _h_ ( **x** ) _f_ ( **x** )/ _g_ ( **x** ). The two estimators have the same expectation, but what about the variance? Using the well-known properties of the variance, we find

From the second equation, it is easy to see that the choice

(8.15)

leads to the ideal condition Var _g_ [ _h_ * ( **X** )] ≡ 0. We may find an exact estimate with _one_ observation! Clearly, there must be a fly somewhere in the ointment. Indeed, this density is _truly_ ideal, as it requires the knowledge of θ itself. Nevertheless, Eq. (8.15) does provide us with some guidance, as it suggests that we should use a density matching the product _h_ ( **x** ) _f_ ( **x** ) as far as possible. By the way, you may see why we have required the condition _h_ ( **x** ) ≥ 0, since a density cannot be negative.4 The problem is that _f_ (·) is a density, but its product with _h_ (·) is not; we miss a normalization constant, which is precisely what we want to compute. Not all hope is lost, however:

**1.** In Chapter 14 on Markov chain Monte Carlo we will see that there are methods to sample from a density which is known up to a constant.

**2.** We may try to come up with an approximation of the ideal density.

As a further hint, let us consider the difference between the two variances:

We would like to obtain ΔVar > 0. Unfortunately, while the factor _h_ 2( **x** ) _f_ ( **x** ) is non-negative, the term within brackets has to be negative somewhere, since both _f_ (·) and _g_ (·) are densities. We should select a new density _g_ such that

In fact, the name "importance sampling" derives from this observation.

###   Example 8.7 Using importance sampling to estimate π

Let us consider again the estimation problem of Example 8.5, where we have noted that

To improve our estimate, we may apply importance sampling, following an approach described in [1]. A possible idea to approximate the ideal probability distribution is to divide the integration interval [0, 1] into _L_ equally spaced subintervals of width 1/L. The extreme points of the _k_ th subinterval ( _k_ = 1,..., _L_ ) are ( _k_ − 1)/ _L_ and _k/L_ , and the midpoint of this subinterval is _s_ _k_ = ( _k_ − 1)/ _L_ \+ 1/(2 _L_ ). A rough estimate of the integral is obtained by computing

Then, to find an approximation of the ideal density _g_ ( _x_ ), we could use something like

where we have used the fact _f_ ( _x_ ) = 1 (uniform distribution). Unfortunately, this need not be a density integrating to 1 over the unit interval. In order to avoid this difficulty and to simplify sampling, we may define a probability of sampling from a subinterval and use a uniform density within each subinterval. To this end, let us consider the quantities

Clearly, Σ _k_ _q_ _k_ = 1 and _q_ _k_ ≥ 0, since our function _h_ is non-negative; hence, the numbers _q_ _k_ may be interpreted as probabilities. In our case, they may be used as the probabilities of selecting a sample point from the _k_ th subinterval. To summarize, and to cast the problem within the general framework, we have

Here, _g_ ( _x_ ) is a piecewise constant density; the _L_ factor multiplying the _q_ _k_ in _g_ ( _x_ ) is just needed to obtain the uniform density over an interval of length 1/ _L_.

The resulting code is illustrated in Fig. 8.8, where m is the sample size and L is the number of subintervals. The code is fairly simple, and subintervals are selected as described in the last part of Section 5.3, where we have seen how to sample discrete empirical distributions by the function EmpiricalDrnd of Fig. 5.7:

**FIGURE 8.8** **Using importance sampling to estimate π.**

    > pi
    [1] 3.141593
    > set.seed(55555)
    > estpiIS(1000,10)
    [1] 3.135357
    > estpiIS(1000,10)
    [1] 3.142377
    > estpiIS(1000,10)
    [1] 3.14807
    > estpilS(1000,10)
    [1] 3.148633
    > estpiIS(1000,10)
    [1] 3.140776

We see that the improved code, although not a very sensible way to compute π, yields a remarkable reduction in variance with respect to crude Monte Carlo.

The approach that we have just pursued looks suspiciously like stratified sampling. Actually, there is a subtle difference. In stratified sampling we define a set of strata, which correspond to events of known probability; here we have not used strata with known probability, as we have used sampling to estimate the probabilities _q_ _k_.

###   Example 8.8 Dancing on the tail of a normal

Let us consider the expected value E[ _h_ ( _X_ )], where _X_ ~ N(0, 1) and

We observe that the function _h_ (·) is a triangle with support on the interval (3, 5) and a fairly large maximum at _x_ = 4. However, since a standard normal has little density outside the interval (−3, 3), we have large function values where the probability is low. Crude Monte Carlo will generate almost all of the observations where the function value is zero, with a corresponding waste of effort. A huge sample size is needed to get an accurate estimate. Therefore, we may try to shift the mode of the density in order to match the maximum of the function. Hence, let us consider a distribution N(θ, 1), with θ = 4. Finding the likelihood ratio is fairly easy:

(8.16)

In Fig. 8.9 we show R code to evaluate the triangle function (please note how vectorization is achieved) and to compare the outcome of crude Monte Carlo against importance sampling, producing the following output:

**FIGURE 8.9** **Shifting the expected value of a normal distribution.**

    > as.numeric(out$estimate)
    [1] 0.4058854
    > as.numeric(out$conf.int)
    [1] 0.1120629 0.6997080
    > as.numeric(outC$estimate)
    [1] 0.3631287
    > as.numeric(outC$conf.int)
    [1] 0.3527247 0.3735328

We see how a very large confidence interval results from crude Monte Carlo, whereas shifting the mean of the normal produces a much more accurate estimate.

On the one hand, Example 8.8 shows a possible heuristic to find a distorted density, by matching it with the maximum of the integrand function. On the other hand, we see how rare events may be dealt with by importance sampling. We elaborate on the theme in the next section, using a more practically relevant example.

## 8.6.1 IMPORTANCE SAMPLING AND RARE EVENTS

Importance sampling is often used when small probabilities are involved. Consider, for instance, a random vector **X** with joint density _f_ , and suppose that we want to estimate

where { **X**   _A_ } is a rare event with a small but unknown probability **P** { **X**   _A_ }. Such an event could be the occurrence of a large loss, which is relevant in risk management. The conditional density is

for **x**   _A._ By defining the indicator function

we may rewrite θ as

If we use crude Monte Carlo simulation, many samples will be wasted, as the event { **X**   _A_ } will rarely occur. Now, assume that there is a density _g_ (·) such that this event is more likely under the corresponding probability measure. Then, we generate the observations **X** _i_ according to _g_ (·) and estimate

Let us illustrate the idea with a simple but quite instructive example: Pricing a deeply out-of-the-money vanilla call option. If _S_ 0 is the initial price of the underlying asset, we know that its expected value at maturity is, according to geometric Brownian motion model under the risk-neutral measure, _S_ 0 _e_ _rT_. If this expected value is small with respect to the strike price _K_ , it is unlikely that the option will be in-the-money at maturity, and with crude Monte Carlo many replications are wasted, because the payoff is zero in most of them. We should change the drift in order to increase the probability that the payoff is positive. One possible choice is a drift μ, such that the expected value of _S_ _T_ is the strike price:

While under the risk-neutral measure we sample _S_ _T_ = _S_ 0 _e_ _Z_ by generating normal variates

we should sample _S_ _T_ = _S_ 0 _e_ _Y_ by generating

This in turn requires sampling standard normal variates   and then using

Now the tricky part is to compute the likelihood ratio. For the sake of clarity, assume that we sample _Y_ from a normal distribution N(β, ξ) whereas the original distribution is N(α, ξ). Then, the ratio of the two densities can be found much like we did in Example 8.8:

(8.17)

Now it is easy to extend the naive Monte Carlo code of function BlsMCNaive, see Fig. 6.4, to the function BlsMCIS displayed in Fig. 8.10. We may check the efficiency gain of importance sampling by running the script in Fig. 8.11. In the script we estimate the price with both crude Monte Carlo and importance sampling, in order to compare the percentage errors with respect to the exact price. We price the same option 100 times, in order to collect the average percentage error and the worst-case percentage error. We reset the random variate generator twice in order to use exactly the same stream of standard normal variates. Running the script, we obtain

**FIGURE 8.10** **Using importance sampling to price a deeply out-of-the-money vanilla call option.**

**FIGURE 8.11** **Comparing the estimation errors obtained by crude Monte Carlo and importance sampling for a deeply out-of-the-money vanilla call option.**

    Average Percentage Error:
    MC = 2.248%
    MC+IS = 0.305%
    Worst Case Percentage Error:
    MC = 6.178%
    MC+IS = 1.328%

We notice that, as we hoped, importance sampling is able to reduce the average error. What is probably even more striking is that the worst-case error may be quite large in naive Monte Carlo, whereas it is more reasonable with importance sampling, which indeed seems more robust. We should note, however, that this improvement is not to be expected for at-the-money options.

## 8.6.2 A DIGRESSION: MOMENT AND CUMULANT GENERATING FUNCTIONS

In the previous section we have used a heuristic argument to come up with an importance sampling measure to improve pricing accuracy for a deeply out-of-the-money option. In the next section we will consider a systematic way to find distorted densities for use within importance sampling. Here we pause a little to introduce a couple of relevant concepts for the sake of the unfamiliar reader. As usual, we will not strive for full generality and mathematical rigor, as our aim is just to point out the practical relevance of some concepts from probability theory.

**DEFINITION 8.1 (Moment generating function)** _Given a continuous random variable X with density f_ _X_ (·), _its moment generating function is defined as_

(8.18)

The interest of the moment generating function lies in the fact that, as the name suggests, it can be used to compute moments of a random variable, since the following fact can be proved:

which means that to compute the moment of order _n_ of the random variable _X_ , denoted by _m_ _n_ , we can take the derivative of order _n_ of its moment generating function and evaluate it for θ = 0. To see this, let us consider the series expansion of an exponential:

If we plug a random variable _X_ into this expression and take an expectation, we find

By taking the first order derivative, it is easy to see that

By setting θ = 0 we obtain _m_ 1 = _M_ _X_ (1) (0). By the same token, if we take the second order derivative we obtain

which implies _m_ 2 = _M_ _X_ (2) (0), and so on. Now the problem is to find the moment generating function for the most common distributions. The next example outlines the case of a normal random variable.

###   Example 8.9 Moment generating function for normal variables

It is not too difficult to prove by an integration exercise that the moment generating function of a normal random variable _X_ ~ N(μ, σ2) is

(8.19)

The proof relies on typical tricks of the trade, like completing the square within an exponential, much like we have seen in Section 8.4. Then, it is a simple, even though a bit tedious, exercise to prove that for a normal variable _X_ we have

The latter result is a key step to prove that the kurtosis of a normal variable is 3.

**DEFINITION 8.2 (Cumulant generating function)** _The cumulant generating function of a random variable X with moment generating function M_ _X_ (θ) _is defined as_

(8.20)

For instance, given Eq. (8.19), we immediately see that the cumulant generating function of a normal variable _X_ ~ N(μ, σ2) is

(8.21)

We should also mention that the cumulant generating function, if it exists, is convex. The cumulant generating function plays a relevant role in exponential tilting for importance sampling.

## 8.6.3 EXPONENTIAL TILTING

Exponential tilting, also known as exponential twisting, is a common strategy to define distributions for use within importance sampling. The idea is to multiply the original density5 _f_ ( _x_ ) by an exponential function depending on a parameter θ. The result, _e_ θ _x_ _f_ ( _x_ ), is not a density and needs normalization to define a new density depending on the tilting parameter θ:

(8.22)

We immediately recognize the denominator of this ratio as the moment generating function _M_ _X_ (θ). By recalling the definition of the cumulant generating function, we may rewrite Eq. (8.22) as

(8.23)

This relationship provides us with the likelihood ratio that we need to apply importance sampling. In fact, from Eq. (8.14) we know that the expected value E[ _h_ ( _X_ )] under density _f_ (·) can be rewritten as

(8.24)

Thus, in order to apply importance sampling with exponential tilting, we have to:

**1.** Sample _X_ from the tilted density _f_ θ(·).

**2.** Use the estimator provided by Eq. (8.24), which involves the tilting parameter θ and the cumulant generating function  (θ).

In order to accomplish the first step above, we should wonder which kind of random variable we find, if we tilt its distribution exponentially. The following example illustrates yet another nice property of the normal distribution.

###   Example 8.10 Tilting a normal distribution

Let us find the density of a tilted normal variable _X_ ~ N(μ, σ2):

By putting the exponentials together, completing squares, and rearranging, we end up with

which shows that the tilted variable is again a normal variable, but with a shifted expected value: _X_ θ ~ N(μ + θσ2, σ2).

The next example deals with the case in which, in order to run a Monte Carlo simulation, we have to generate a sequence of normal variables.

###   Example 8.11 Likelihood ratios of tilted distributions

Consider a simulation which is driven buy a sequence of _n_ i.i.d. variables _X_ 1,..., _X_ _n_. Thanks to independence, the likelihood ratio is related to a product of densities, which can be expressed in terms of the cumulant generating function:

(8.25)

Now the open question is how to select a suitable tilting parameter θ. Typically, problem-dependent arguments are used. We have seen an example when dealing with a deeply out-of-the-money call option, where we argued that, under the importance sampling measure, the expected value of the underlying asset price at maturity should be the option strike price. Hence, let us say that, using such intuitive arguments, we conclude that the expected value of _X_ under the tilted distribution should be Eθ[ _X_ ] = β. To accomplish this objective, we rely on a useful result. If we take the derivative of the cumulant generating function with respect to θ, we find

(8.26)

where we have used Eq. (8.24) the other way around. Therefore, we may solve the equation

to find the corresponding tilting parameter. The material of this section will play a fundamental role in Section 13.4, where we use importance sampling to estimate a risk measure, value-at-risk, by Monte Carlo simulation.

# For further reading

  Every intermediate to advanced book on Monte Carlo methods has a chapter on variance reduction strategies; see, e.g., [4].

  More specific references, offering interesting examples, are [3] and [7].

  We did not cover all of the variance reduction strategies that have been proposed in the literature:

– For Latin hypercube sampling see, e.g., [4].

– Specific control variates for option pricing, based on hedging arguments, are discussed in [2].

– The cross-entropy method is discussed in [4] and [7].

– Moment matching is applied to option pricing in [3].

# References

1 I. Beichl and F. Sullivan. The importance of importance sampling. _Computing in Science and Engineering_ , 1:71–73, March–April 1999.

2 L. Clewlow and C. Strickland. _Implementing Derivatives Models._ Wiley, Chichester, West Sussex, England, 1998.

3 P. Glasserman. _Monte Carlo Methods in Financial Engineering._ Springer, New York, 2004.

4 D.P. Kroese, T. Taimre, and Z.I. Botev. _Handbook of Monte Carlo Methods._ Wiley, Hoboken, NJ, 2011.

5 C.P. Robert and G. Casella. _Introducing Monte Carlo Methods with R._ Springer, New York, 2011.

6 R. Y. Rubinstein. _Simulation and the Monte Carlo Method._ Wiley, Chichester, 1981.

7 H. Wang. _Monte Carlo Simulation with Applications in Finance._ CRC Press, Boca Raton, FL, 2012.

1See Section 3.5 for a probabilistic view of linear regression.

2This example is borrowed from [5, pp. 107–108].

3Readers with a background in stochastic calculus would probably use the term "Radon–Nikodym derivative."

4See, e.g., [6, p. 122] to see how to deal with a generic function _h._

5To avoid notational clutter, here we denote the density of a random variable _X_ by _f_ ( _x_ ) rather than _f_ _X_ ( _x_ ).

# Chapter Nine

# Low-Discrepancy Sequences

In the previous chapter we have reviewed the main variance reduction strategies that are used in financial and economic applications. Since those approaches are based on probabilistic concepts, they implicitly assume that random sampling in Monte Carlo methods is _really_ random. However, the pseudorandom numbers produced by an LCG or by more sophisticated algorithms are not random at all. Hence, one could take a philosophical view and wonder about the very validity of variance reduction methods, and even the Monte Carlo approach itself. Taking a more pragmatic view, and considering the fact that Monte Carlo methods have proven their value over the years, we should conclude that this shows that there are some deterministic number sequences that work well in generating samples. It is also useful to remember that the aim of a Monte Carlo simulation is actually to estimate a multidimensional integral on the unit hypercube:

The function _h_ (·) may be so complicated that we cannot express it analytically, but this is of no concern conceptually. We need a stream of i.i.d. random numbers to fill the integration domain in a satisfactory manner. When a regular grid derived from classical product rules for numerical integration (see Chapter 2) is not feasible, we may fill it by random numbers, but we could also resort to alternative deterministic sequences that are, in some sense, evenly distributed. This idea may be made more precise by defining the _discrepancy_ of a sequence of numbers. In a sense, low-discrepancy sequences are half-way between deterministic grid-based methods and Monte Carlo sampling, as they are _meant_ to be deterministic, whereas pseudorandom numbers are deterministic for the lack of a better alternative, yet they do not look regular at all, unlike grids. Low-discrepancy sequences are the foundation of quasi–Monte Carlo methods; sometimes, the term "quasi-random" sequence is used, but we will mostly avoid it as it is a bit misleading.

We first introduce the concept of discrepancy in Section 9.1. Then we cover two well-known low-discrepancy sequences: Halton sequences in Section 9.2 and Sobol sequences in Section 9.3. These two sequences do not exhaust the list of possible approaches, yet they are adequate to grasp the basic concepts and understand the pitfalls of quasi–Monte Carlo simulation. The deterministic nature of low-discrepancy sequences has one major drawback with respect to random sampling: There is no obvious way to assess the quality of estimates, since the usual confidence intervals do not make any sense. There are theoretical bounds on the integration error, but they may be very weak and of little practical use. A possible remedy is the randomization of low-discrepancy sequences. Furthermore, there are scrambling strategies to improve the quality of standard low-discrepancy sequences. We outline these matters in Section 9.4, without entering into theoretical details; some scrambling strategies are offered in R functions, and we just aim at providing the reader with the minimal background for their use. Low-discrepancy sequences play in quasi–Monte Carlo methods the same role as pseudorandom sequences do in Monte Carlo methods. When we need observations from another distribution, most notably the normal, we resort to some techniques that we have introduced in Chapter 5 for random variate generation. We illustrate such transformations and sample path generation by low-discrepancy sequences in Section 9.5. Finally, we should also mention that:

  Sampling by low-discrepancy sequences does not only play a role in simulation, but also in global optimization, i.e., the solution of hard optimization problems.

  Other deterministic sampling approaches are available, relying on good lattice rules; see Section 2.4.3.

# 9.1 Low-discrepancy sequences

In the first chapters of the book we have insisted on the fact that a Monte Carlo simulator is essentially a complicated function mapping a stream of independent uniform random variables into an estimator of a quantity of interest. Whenever the dimension _m_ of the integration domain is fixed, we are actually computing an integral over the unit hypercube _I_ _m_ = (0, 1) _m_ ⊂  _m_. This is the case when we use, for instance, the inverse transform method to obtain random variates from uniform pseudorandom numbers. However, this does not hold when we resort to acceptance–rejection, because with this kind of method we do not know a priori how many uniform random variables are needed to generate a single observation of the distribution of interest. Indeed, this is the reason why acceptance–rejection strategies are not compatible with the methods of this chapter. A good integration strategy should "cover" the unit hypercube in the most even and uniform way; note that, in this framework, we need not be concerned with independence of random variates, since this is actually guaranteed by a good coverage of _I_ _m_. However, we need to clarify what a "good coverage" is, which in turn requires a well-defined way to measure its quality.

Assume that we want to generate a sequence   of _N_ points in _I_ _m_ ,

in order to cover that domain in a satisfactory way. If the points are well distributed, the number of them included in any subset _G_ of _I_ _m_ should be roughly proportional to its volume vol( _G_ ). For instance, if _N_ = 100, _m_ = 2, and we consider the square _G_ = [0, 0.5] × [0, 0.5] ⊂ _I_ 2, we would expect to find 25 points in set _G_ , since its area is 0.5 × 0.5 = 0.25. Whatever _N_ , about 25% of the total number of points should fall there. However, this should apply to any subset of _I_ 2. The concept of discrepancy aims at formalizing this intuition in a mathematically tractable way. Since dealing with arbitrary subsets of the unit hypercube is not quite feasible, we should restrict their form. Given a vector **x** = [ _x_ 1, _x_ 2,..., _x_ _m_ ] , consider the rectangular subset _G_ x defined as

(9.1)

which has volume _x_ 1 _x_ 2 ··· _x_ _m_. If we denote by _S_ ( , _G_ ) a function counting the number of points in the sequence   that are contained in a subset _G_ ⊂ _I_ _m_ , a possible definition of discrepancy is

To be precise, this is called the _star discrepancy_ , and it measures the worst-case distance between the ideal and the actual number of points included in a rectangle of the type specified by Eq. (9.1). In Section 4.3.2 we have considered the Kolmogorov–Smirnov measure of fit for probability distributions: Star discrepancy can be interpreted as such a measure of fit for a multidimensional uniform distribution with independent marginals.

When computing a multidimensional integral on the unit hypercube, it is natural to look for low-discrepancy sequences. Some theoretical results suggest that low-discrepancy sequences may perform better than pseudorandom sequences obtained through an LCG or its variations. From the theory of confidence intervals, we know that the estimation error with Monte Carlo sampling is something like

where _N_ is the sample size. With certain point sequences, it can be shown that the error is something like

(9.2)

where _m_ is the dimension of the space in which we are integrating.1 It is conjectured that the expression in Eq. (9.2) is a lower bound on what can be achieved, and the term "low-discrepancy sequence" is used when referring to sequences achieving this bound. Various such sequences have been proposed in the literature, and some are available in R. In the following, we illustrate the basic ideas behind two of them, namely, Halton and Sobol sequences.

# 9.2 Halton sequences

The Halton sequence was the first sequence for which the attainment of the low-discrepancy bound of Eq. (9.2) was shown. This sequence relies on a generalization of the Van der Corput one-dimensional sequence, which is based on the following simple recipe:

  Represent an integer number _n_ in a base _b_ , where _b_ is a prime number:

  Reflect the digits and add a radix point to obtain a number within the unit interval:

More formally, if we represent an integer number _n_ as

the _n_ th number in the Van der Corput sequence with base _b_ is

###   Example 9.1 Van der Corput sequence in base 2

The generation of a sequence with base 2 is quite instructive. Let us start with _n_ = 1, which may be represented in base 2 as

Reflecting this number we find

Applying the same drill to _n_ = 2 yields

Now the pattern should be clear enough:

This sequence may be generated in R using the halton function from the randtoolbox package:

    > library(randtoolbox)
    > halton(10, dim = 1)
    [1] 0.5000 0.2500 0.7500 0.1250 0.6250 0.3750
    0.8750 0.0625 0.5625 0.3125

We see from the example how a Van der Corput sequence works: It fills the unit interval by jumping around in such a way to maintain a balanced coverage. A Halton sequence in multiple dimensions is obtained by associating a Van der Corput sequence with each dimension, using prime numbers for each base. For instance, the snapshot

    > halton(10, dim = 3)
    [,1] [,2] [,3]
    [1,] 0.5000 0.33333333 0.20
    [2,] 0.2500 0.66666667 0.40
    [3,] 0.7500 0.11111111 0.60
    [4,] 0.1250 0.44444444 0.80
    [5,] 0.6250 0.77777778 0.04
    [6,] 0.3750 0.22222222 0.24
    [7,] 0.8750 0.55555556 0.44
    [8,] 0.0625 0.88888889 0.64
    [9,] 0.5625 0.03703704 0.84
    [10,] 0.3125 0.37037037 0.08

shows that the base _b_ = 2 is associated with dimension 1, whereas bases _b_ = 3 and _b_ = 5 are associated with dimensions 2 and 3, respectively.

It is instructive to compare how a pseudorandom sample and a Halton sequence cover the unit square (0, 1) × (0, 1). The following snapshot produces the plots in Figs. 9.1 and 9.2:

**FIGURE 9.1** **Pseudorandom sample in two dimensions.**

**FIGURE 9.2** **Covering the bidimensional unit square with Halton sequences.**

    > set.seed(55555)
    > haltonSeq <- halton (1000, dim = 2)
    > windows()
    > plot(haltonSeq[,1],haltonSeq[,2],pch=20)
    > grid(nx=10,col='black')
    > windows()
    > plot(runif(1000),runif (1000),pch=20)
    > grid(nx=10,col='black')

The judgment is a bit subjective here, but it could be argued that the coverage of the Halton sequence is more even. What is certainly true is that using non-prime numbers as bases may result in quite unsatisfactory patterns, such as the one shown in Fig. 9.3. This plot has been obtained using Van der Corput sequences with bases 2 and 4.

**FIGURE 9.3** **Bad choice of bases in Halton sequences.**

###   Example 9.2 Numerical integration with Halton sequences

Let us explore the use of Halton low-discrepancy sequences in a bidimensional integration context. Suppose that we want to compute

The surface corresponding to this function is illustrated in Fig. 9.4 and features a nasty oscillatory behavior. In R, we first write a function to compute the integrand and then use the adapt Integrate function from the cubature package to get an estimate by traditional quadrature formulas:

**FIGURE 9.4** **Plot of the integrand function in Example 9.2.**

    > library(cubature)
    > f <- function(x) exp(−x[1]*x [2])*
    (sin(6*pi*x[1])+cos(8*pi*x[2]))
    > adaptIntegrate(f,rep(0,2),rep(1,2),tol=1e-4)
    $integral
    [1] 0.01986377
    $error
    [1] 1.983034e-06
    $functionEvaluations
    [1] 13957
    $returnCode
    [1] 0

It is easy to see that Monte Carlo estimates based on a sample of 10,000 pseudorandom points are not reliable:

    > set.seed(55555)
    > mean(apply(matrix(runif(20000),ncol=2),1,f))
    [1] 0.02368898
    > mean(apply(matrix(runif(20000),ncol=2), 1, f))
    [1] 0.01311604
    > mean(apply(matrix(runif(20000),ncol=2),1,f))
    [1] 0.01870101
    > mean(apply(matrix(runif(20000),ncol=2),1,f))
    [1] 0.01796109
    > mean(apply(matrix(runif(20000), ncol=2),1, f))
    [1] 0.01680492

In the above snapshot we sample 20,000 observations from the uniform distribution, but they are arranged into a two-column matrix and correspond to 10,000 points; also note the use of apply. On the contrary, the result obtained by the bidimensional Halton sequence seems fairly satisfactory:

    > library(randtoolbox)
    > mean(apply(halton(10000,dim=2),1,f))
    [1] 0.01987514

The example above looks quite promising. Nevertheless, there are issues in using Halton sequences for high-dimensional problems. To see why, let us plot the Van der Corput sequence for a large base:

    > X <- halton(n=1000,dim=30)
    > plot (X[,30],pch=20)
    > 1/X[1,30]
    [1] 113

Here we generate a Halton sequence with dimension 30 and plot the last column of the output matrix, which corresponds to a Van der Corput sequence with base _b_ = 113. The resulting plot, shown in Fig. 9.5, features long monotonic subsequences. This is not surprising, given the large base: For instance, we have to move from 1/113 to 112/113 before breaking the first subsequence. This results in a very bad coverage of the unit square when we use Halton sequences with high bases associated with high dimensions. The following command produces the plot in Fig. 9.6:

**FIGURE 9.5** **Long monotonic subsequences in a Van der Corput sequence with a large base.**

**FIGURE 9.6** **Poor coverage of the unit square when large bases are used in Halton sequences.**

    > plot (X[,30],X[,29],pch=20)

This is just a projection of the 30-dimensional sequence on two margins, but it is reasonable to expect that such a pattern may have a strong adverse effect on the results.

# 9.3 Sobol low-discrepancy sequences

With Halton sequences, we have to use Van der Corput sequences associated with large prime numbers for high-dimensional problems, and as we have seen this may result in poor performance. A way to overcome this issue is to generate only Van der Corput sequences associated with a suitably small base. This idea has been pursued and resulted in two more low-discrepancy sequences:

  In Faure sequences, only one base _b_ is used for all dimensions. It is required that the prime number _b_ is larger than the problem dimensionality _m_ ; then, different permutations of the resulting Van der Corput sequence are associated with each dimension.

  In Sobol sequences, only the smallest base, _b_ = 2, is used. Again, suitable permutations of the corresponding Van der Corput sequence are associated with each dimension.

Here we only outline the basic ideas behind Sobol sequences, which rely on more sophisticated concepts in the algebra of polynomials on a binary field. Before doing so, however, it is useful to get acquainted with this kind of sequence by using the function sobol provided by the package randtoolbox:

    > sobol(n=10,dim=4)
    [,1] [,2] [,3] [,4]
    [1,] 0.5000 0.5000 0.5000 0.5000
    [2,] 0.7500 0.2500 0.7500 0.2500
    [3,] 0.2500 0.7500 0.2500 0.7500
    [4,] 0.3750 0.3750 0.6250 0.1250
    [5,] 0.8750 0.8750 0.1250 0.6250
    [6,] 0.6250 0.1250 0.3750 0.3750
    [7,] 0.1250 0.6250 0.8750 0.8750
    [8,] 0.1875 0.3125 0.3125 0.6875
    [9,] 0.6875 0.8125 0.8125 0.1875
    [10,] 0.9375 0.0625 0.5625 0.9375

It is easy to see that the four columns are all permutations of the Van der Corput sequence with base _b_ = 2.

## 9.3.1 SOBOL SEQUENCES AND THE ALGEBRA OF POLYNOMIALS

The uninterested reader may safely skip this section, which is more technical and not needed in the rest of the book. We will only give some hints at the theory behind Sobol sequences; the section can also be seen as an excuse to illustrate some R functions oriented toward bit manipulation. Let us focus on the generation of a one-dimensional sequence

in the unit interval [0, 1]. A Sobol sequence is generated on the basis of a set of _direction numbers v_ 1, _v_ 2,...; we will see shortly how direction numbers are selected, but for now just think of them as numbers which are less than 1. To get the _n_ th number in the Sobol sequence, consider the binary representation of the integer _n_ :2

The corresponding number in the sequence is obtained by computing the bitwise exclusive OR of the direction numbers _v_ _i_ for which _b_ _i_ ≠ 0:

(9.3)

If direction numbers are chosen properly, a low-discrepancy sequence will be generated [8]. A direction number may be thought as a binary fraction:

or as

where _m_ _i_ < 2 _i_ is an odd integer. To generate direction numbers, we exploit primitive polynomials over the field  2, i.e., polynomials with binary coefficients:

Irreducible polynomials are those polynomials which cannot be factored; primitive polynomials are a subset of the irreducible polynomials and are strongly linked to the theory of error-correcting codes, which is beyond the scope of this book. Some irreducible polynomials over the field  2 are listed, e.g., in [7, Chapter 7], to which the reader is referred for further information. Given a primitive polynomial of degree _d_ , the procedure for generating direction numbers is based on the recurrence formula:

This is better implemented in integer arithmetic as

(9.4)

Some numbers _m_ 1,..., _m_ _d_ are needed to initialize the recursion. They may be chosen arbitrarily, provided that each _m_ _i_ is odd and _m_ _i_ < 2 _i_.

###   Example 9.3 Direction numbers

As an example, let us build the set of direction numbers on the basis of the primitive polynomial

In this polynomial we have coefficients _a_ 1 = 0 and _a_ 2 = 1. Therefore, the recursive scheme of Eq. (9.4) runs as follows:

which may be initialized with _m_ 1 = 1, _m_ 2 = 3, _m_ 3 = 7. The reasons why this may be a good choice are given in [2]. We may carry out the necessary computations step by step in R, using the bitXor function.

    > library(bitops)
    > m <- c(1, 3, 7)
    > i <- 4
    > m[i] <- bitXor(4*m[i-2],bitXor(8*m[i-3],m[i-3]))
    > i <- 5
    > m[i] <- bitXor(4*m[i-2],bitXor(8*m[i-3],m[i-3]))
    > i <- 6
    > m[i] <- bitXor(4*m[i-2],bitXor(8*m[i-3],m[i-3]))
    > m
    [1] 1 3 7 5 7 43

Given the integer numbers _m_ _i_ , we may build the direction numbers _v_ _i_. To implement the generation of direction numbers, we may use a function like GetDirNumbers, which is given in Fig. 9.7. The function requires a primitive polynomial p, a vector of initial numbers m, and the number n of direction numbers we want to generate. On exit, we obtain the direction numbers v and the integer numbers m.

**FIGURE 9.7** **R code to generate direction numbers for Sobol sequences.**

    > p <- c(1,0,1,1)
    > m0 <- c(1,3,7)
    > GetDirNumbers(p,m0,6)
    $v
    [1] 0.500000    0.750000 0.875000 0.312500
    0.218750 0.671875
    $m
    [1] 1 3 7 5 7 43

The code is not optimized; for instance, the first and last coefficients of the input polynomial should be 1 by default, and no check is done on the congruence in size of the input vectors.

After computing the direction numbers, we could generate a Sobol sequence according to Eq. (9.3). However, an improved method was proposed by Antonov and Saleev [1], who proved that the discrepancy is not changed by using the Gray code representation of _n_. Gray codes are discussed, e.g., in [7, Chapter 20]; all we need to know is the following:

**1.** A Gray code is a function mapping an integer _i_ to a corresponding binary representation _G_ ( _i_ ); the function, for a given integer _N_ , is one-to-one for 0 ≤ _i_ ≤ 2 _N_ − 1.

**2.** A Gray code representation for the integer _n_ is obtained from its binary representation by computing

**3.** The main feature of such a code is that the codes for consecutive numbers _n_ and _n_ \+ 1 differ only in one position.

###   Example 9.4 Gray code

Computing a Gray code is easily accomplished in R. For instance, we may define a gray function and compute the Gray codes for the numbers _i_ = 0, 1,..., 15 as follows:

    > gray <- function (x) bitXor(x,bitShiftR(x,b=1))
    > codes <- matrix(0,nrow=16,ncol=4)
    > for (i in 1:16) codes[i,]
    <- as.numeric(intToBits(gray(i-1)))[c(4,3,2,1)]
    > codes
          [,1]    [,2]    [,3]    [,4]
    [1,]    0    0    0    0
    [2,]    0    0    0    1
    [3,]    0    0    1    1
    [4,]    0    0    1    0
    [5,]    0    1    1    0
    [6,]    0    1    1    1
    [7,]    0    1    0    1
    [8,]    0    1    0    0
    [9,]    1    1    0    0
    [10,]    1    1    0    1
    [11,]    1    1    1    1
    [12,]    1    1    1    0
    [13,]    1    0    1    0
    [14,]    1    0    1    1
    [15,]    1    0    0    1
    [16,]    1    0    0    0

We have used the function bitShiftR of the package bitops to shift the binary representation of × one position to the right and the function intToBits to obtain the binary representation of a number. We see that indeed the Gray codes for consecutive numbers _i_ and _i_ \+ 1 differ in one position; that position corresponds to the rightmost zero bit in the binary representation of _i_ (adding leading zeros if necessary).

Using the feature of Gray codes, we may streamline the generation of a Sobol sequence. Given _x_ ( _n_ ), we have _x_ ( _n_ +1) = _x_ ( _n_ ) ⊕ _v_ _c_ , where _c_ is the index of the rightmost zero bit _b_ _c_ in the binary representation of _n_.

# 9.4 Randomized and scrambled low-discrepancy sequences

In Chapter 7 we have seen how confidence intervals may be used to measure the reliability of Monte Carlo estimates. With low-discrepancy sequences, we cannot apply that idea, as there is nothing random in quasi–Monte Carlo simulation. Actually, there is a result, known as the Koksma–Hlawka theorem, which provides us with a bound on the integration error we incur when approximating an integral on the unit hypercube:

where _D_ * ( **x** (1),..., **x** ( _N_ )) is the star discrepancy of the point sequence  _N_ = ( **x** (1),..., **x** ( _N_ )), and _V_ ( _f_ ) is a quantity related to the integrand function, known as the Hardy–Krause variation. We will refrain from defining this form of variation, but let us say that, in general, variations try to capture the variability of a function, as this clearly affects the difficulty in approximating its integral numerically. Unfortunately, this bound is not quite useful, unless we have an operational way to evaluate the variation _V_ ( _f_ ). Another issue with such bounds is that they can be very weak. We may compare the above bound with the corresponding Monte Carlo probabilistic bound obtained by a pseudorandom sequence { **U** ( _k_ )} in the unit hypercube:

with probability 1 − α/2, where σ _f_ 2 = Var[ _f_ ( **U** )]. It is true that the discrepancy bound in Eq. (9.2) is more appealing than the   factor in Monte Carlo sampling, but the latter is related to a quite operational measure of estimate quality.

An array of methods have been designed both to allow the construction of confidence intervals and to improve the quality of point sequences:

**Random shift.** This is simple idea that lends itself to application with Korobov point sets (see Section 2.4.3). Given a point set { **u** ( _i_ )}, a uniform random vector **V** in [0, 1) _m_ is sampled and used to shift the point set to yield a sequence consisting of

**Digital shift.** The digital shift differs from the random shift in how the shift is applied, i.e., in an arithmetic modulo _b_ , where _b_ is the base used to represent the numbers in the unit interval. Consider the expansion in base _b_ of the coordinate _j_ of a point **u** ( _i_ ) in [0, 1) _m_ :

A random shift **V** in [0, 1) _m_ is sampled, with coordinates

and shifted points are generated as

**Scrambling.** Scrambling is a deeper readjustment of points in the sequence. A scrambling procedure was originally proposed by Tezuka to improve Faure sequences, and is therefore known as Faure–Tezuka scrambling; an alternative approach is due to Owen. Both of these scrambling approaches are available in R and can be applied to Sobol sequences.

Given a randomized shifting, we may generate a sample of sequences and come up with a confidence interval. The approach is justified by distributional results that we skip here. We just want to illustrate how scrambling of Sobol sequences is applied in R by setting the optional parameter scrambling in the function sobol:

    > sobol(n=10,dim=4, scrambling=1)
    [,1] [,2] [,3] [,4]
    [1,] 0.8897523 0.13510869 0.99200249 0.06669691
    [2,] 0.5694336 0.58325994 0.73452467 0.99942088
    [3,] 0.2349515 0.27649581 0.07017102 0.36903149
    [4,] 0.0633796 0.68916714 0.76618928 0.52431285
    [5,] 0.7416083 0.39046729 0.41350642 0.15471716
    [6,] 0.8119298 0.82890844 0.15676318 0.78641582
    [7,] 0.3836831 0.02996351 0.52293140 0.40991741
    [8,] 0.4830195 0.50730205 0.05096463 0.24351555
    [9,] 0.8369830 0.32138658 0.62871468 0.62484527
    [10,] 0.6416652 0.89649355 0.87904912 0.44233483

By setting scrambling to 1, Owen scrambling is selected, whereas 2 corresponds to Faure–Tezuka scrambling. We may check the effect of scrambling on the integration problem of Example 9.2:

    > adaptIntegrate(f, rep(0,2),rep(1,2),tol = le-6)$integral
    [1] 0.01986377
    > mean(apply(sobol(10000,dim=2),1,f))
    [1] 0.01976074
    > mean(apply(sobol(10000, dim=2, scrambling=1),1,f))
    [1] 0.01981418
    > mean (apply(sobol(10000, dim=2, scrambling=2),1, f))
    [1] 0.01963135

In this specific example, Owen scrambling seems to work well.

# 9.5 Sample path generation with low-discrepancy sequences

Whatever low-discrepancy sequence we use, we need to transform the "uniform" numbers into something else. Since normal variables are ubiquitous in financial applications, let us consider how we may generate standard normal variables. To generate normal variates, we may either use the Box–Muller method, which we described in Section 5.5.1, or the inverse transform method. We _cannot_ apply polar rejection, because when using low-discrepancy sequences we must integrate over a space with a well-defined dimensionality. We must know exactly how many quasi-random numbers we need, whereas with rejection-based methods we cannot anticipate that.

We recall the Box–Muller algorithm here for the sake of convenience. To generate two independent standard normal variates, we should first generate two independent random numbers _U_ 1 and _U_ 2, and then set

Rather than generating pseudorandom numbers, we may use a bidimensional low-discrepancy sequence. Given the potentially weird effects of the Box–Muller transformation, which we have illustrated in Fig. 5.12, one could argue that the inverse transform method is a safer approach. Since fast and accurate methods to invert the CDF of the standard normal have been devised, we will only pursue the latter approach.

###   Example 9.5 Pricing a vanilla call option

As an illustration, let us price a vanilla call option using Halton and Sobol sequences. The code in Fig. 9.8 receives a sequence of uniform numbers, possibly obtained by a pseudorandom generator, and transforms them to standard normals by inversion. The snapshot below shows that low-discrepancy sequences yield much more accurate answers than naive Monte Carlo, for a relatively small sample size.

**FIGURE 9.8** **Pricing a vanilla call option with low-discrepancy sequences.**

    > S0 <- 50; K <- 50; sigma <- 0.4
    > r <- 0.05; T <- 5/12
    > EuVanillaCall(S0,K,T,r,sigma)
    [1] 5.614995
    > set.seed(55555)
    > BlsLowD(S0,K,T,r,sigma,runif(5000))
    [1] 5.673162
    > BlsLowD(S0,K,T,r,sigma, halton(5000,dim=1))
    [1] 5.592316
    > BlsLowD(S0,K,T,r,sigma,sobol(5000,dim=1))
    [1] 5.605094
    > BlsLowD(S0,K,T,r,sigma,
    sobol(5000,dim=1, scrambling=1))
    [1] 5.612801

In this case, too, it seems that scrambling a Sobol sequence works fairly well.

A final important note concerns the correct use of low-discrepancy sequences for sample path generation. Generating sample paths for geometric Brownian motion is rather easy, as we only need standard normal variables. Consider the following matrix of uniform numbers:

    > halton(10, dim=5)
    [,1] [,2] [,3] [,4] [,5]
    [1,] 0.5000 0.33333333 0.20 0.14285714 0.09090909
    [2,] 0.2500 0.66666667 0.40 0.28571429 0.18181818
    [3,] 0.7500 0.11111111 0.60 0.42857143 0.27272727
    [4,] 0.1250 0.44444444 0.80 0.57142857 0.36363636
    [5,] 0.6250 0.77777778 0.04 0.71428571 0.45454545
    [6,] 0.3750 0.22222222 0.24 0.85714286 0.54545455
    [7,] 0.8750 0.55555556 0.44 0.02040816 0.63636364
    [8,] 0.0625 0.88888889 0.64 0.16326531 0.72727273
    [9,] 0.5625 0.03703704 0.84 0.30612245 0.81818182
    [10,] 0.3125 0.37037037 0.08 0.44897959 0.90909091

It must be realized that with these numbers we may generate ten sample paths with five time instants along each path (not including the initial value of the process). Each dimension of the sequence (number of columns) must correspond to a time instant, and each row corresponds to a sample path. A common mistake is to associate a dimension with a sample path, but by doing so we lose the intertemporal independence property of the standard Wiener processes.

The function in Fig. 9.9 is quite similar to the vectorized code of Fig. 6.3. It can use either Halton or Sobol sequences and, for the latter ones, we may also select the type of scrambling. As a first check, let us generate and plot five sample paths, without any scrambling:

**FIGURE 9.9** **Generating sample paths of GBM by low-discrepancy sequences.**

    > S0 <- 50; mu <- 0.1; sigma <- 0.4;
    > T <- 1/12; numSteps <- 30; numRepl <- 5
    > pathsH <- lowDiscrGBM(S0,mu,sigma,T,numSteps,
                    numRepl,type="halton")
    > pathsS <- lowDiscrGBM(S0,mu,sigma,T,numSteps,numRepl)
    > par(mfrow=c(1,2))
    > plot(pathsH[1,],type='l',ylim=range(pathsH),
                    xlab="Halton",ylab="")
    > for (k in 2:numRepl) lines (pathsH[k,])
    > plot(pathsS[1,],type='l',ylim=range(pathsS),
                    xlab="Sobol",ylab="")
    > for (k in 2:numRepl) lines(pathsS[k,])
    > par(mfrow=c(1,1))

The sample paths, displayed in Fig. 9.10 do not look quite satisfactory. The case of Halton sequence shows decreasing paths for latter time periods; the case of Sobol sequence shows one sample path that looks like a straight horizontal line. A bit of reflection explains why we observe such weird patterns. We have already noted that, for large bases, Van der Corput sequences suffer from long monotonically increasing subsequences. Thus, they start with small numbers, which correspond to negative normal shocks and a monotonic decrease in the stochastic process, at least in the first steps. On the contrary, the first part of the sample paths seems less problematic, as there we are using small bases. As to the Sobol sequences, since the first point consists of a repetition of values 0.5, which correspond to zero normal shocks, the first part of the sample path may look like a straight line, even though it is actually the initial part of an exponential.

**FIGURE 9.10** **Five GBM paths by Halton and Sobol sequences.**

If we sample more paths, as shown in Fig. 9.11, the results look a bit more reasonable. Indeed, skipping the initial parts of a low-discrepancy sequence is a strategy that is sometimes adopted, as well as the Brownian bridge, in order to break some nasty patterns. Another possibility, of course, is to use scrambling. The ten sample paths of Fig. 9.12 have been produced by Sobol sequences with Owen scrambling, and they definitely look more realistic.

**FIGURE 9.11** **More GBM paths by Halton and Sobol sequences.**

**FIGURE 9.12** **GBM paths by scrambled Sobol sequences.**

# For further reading

  We have introduced low-discrepancy sequences in a rather informal way, without referring to fundamental concepts such as the digital nets. A rigorous account can be found in [6], whereas a less extensive but readable treatment can be found in [3, Chapter 5].

  You may also see [5], where Chapter 5, among other things, deals with quality measures, and Chapter 6 analyzes questions related to decomposition and effective dimensions.

  Another interesting reference is [4, Chapter 8], which is more specifically aimed at financial applications.

# References

1 I.A. Antonov and V.M. Saleev. An economic method of computing _lp_ τ sequences. _USSR Computational Mathematics and Mathematical Physics_ , 19:252–256, 1979.

2 P. Bratley and B.L. Fox. Algorithm 659: Implementing Sobol's quasirandom sequence generator. _ACM Transactions on Mathematical Software_ , 14:88–100, 1988.

3 P. Glasserman. _Monte Carlo Methods in Financial Engineering_. Springer, New York, 2004.

4 P. Jaeckel. _Monte Carlo Methods in Finance_. Wiley, Chichester, 2002.

5 C. Lemieux. _Monte Carlo and Quasi–Monte Carlo Sampling_. Springer, New York, 2009.

6 H. Niederreiter. _Random Number Generation and Quasi–Monte Carlo Methods_. Society for Industrial and Applied Mathematics, Philadelphia, PA, 1992.

7 W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery. _Numerical Recipes in C_ (2nd ed.). Cambridge University Press, Cambridge, 1992.

8 I.M. Sobol. On the distribution of points in a cube and the approximate evaluation of integrals. _USSR Computational Mathematics and Mathematical Physics_ , 7:86–112, 1967.

1This is stated in a slightly more precise form in Section 9.4. See [6] for a detailed and rigorous account of these results.

2 In this section, the rightmost digit in the (binary) representation of an integer number is denoted by _b_ 1 instead of _d_ 0; this is just a matter of convenience, as we associate bit positions with direction numbers rather than powers of a generic base.

# Part Five

# Miscellaneous Applications

# Chapter Ten

# Optimization

There is a multifaceted interaction between optimization and Monte Carlo simulation, which is illustrated by the rich connections between this and other chapters of this book. Monte Carlo methods may be used when dealing with stochastic optimization models, i.e., when some of the parameters characterizing a decision problem are uncertain. This is clearly relevant in finance, and there is an array of approaches depending on the problem structure and on our purpose:

  In stochastic programming (SP) models we generalize deterministic mathematical programming models in order to deal with uncertainty that is typically represented by a scenario tree. The scenario tree may be generated by crude Monte Carlo sampling, or we may adopt other approaches like Gaussian quadrature (see Chapter 2), variance reduction (see Chapter 8), or low-discrepancy sequences (see Chapter 9). The stochastic programming models that we consider here can be extended to account for risk aversion, as illustrated in Chapter 13.

  Another approach to decision making under uncertainty relies on dynamic programming (DP). We will illustrate the advantages and the disadvantages of DP with respect to SP models. The principle behind DP is embodied in the Bellman recursive equation and is extremely powerful; nevertheless, DP is plagued by the so-called curse of dimensionality. There are approaches to overcome this issue and to make DP a viable computational tool. On the one hand, we have numerical dynamic programming methods based on relatively straightforward tools from numerical analysis; on the other hand we have approximate dynamic programming (ADP) methods, which rely more heavily on learning by Monte Carlo simulation. ADP methods are also used in Chapter 11 to price options with early exercise features.

  Both SP and classical DP approaches rely on a formal model of the system, in the form of explicit mathematical expressions. However, we may have to cope with systems that are too complicated for analytical modeling. In such a case, we may rely on simulation-based optimization, where there is also a clear role for Monte Carlo methods. The idea is also appealing when we want to reduce the complexity of optimization under uncertainty by making decisions on the basis of parameterized decision rules. Learning by search and simulation is a possible strategy in such cases. Sometimes, we may also wish to take advantage of simulation to estimate the sensitivity of performance with respect to some parameters, in order to guide the search process. Sensitivity estimation is the topic of Chapter 12.

A further use of Monte Carlo methods is to check the out-of-sample robustness of a solution. Within an optimization model, we may only afford a limited representation of uncertainty; the in-sample value of the objective function may be a misleading indicator of quality. Thus, it may be useful to verify the performance of the selected solution in a much wider set of scenarios.

Stochastic optimization in all of its forms does not tell the whole story about the interaction between optimization and Monte Carlo methods. Stochastic search strategies are also used to solve _deterministic_ nonconvex optimization problems. There is an array of useful strategies, including some fancy ones mimicking biological systems, such as genetic algorithms or particle swarm optimization. Such methods are typically derivative-free, i.e., unlike classical nonlinear programming methods, they do not rely on information provided by the gradient or the Hessian matrix of the objective function. This allows to use them when the objective is not guaranteed to be differentiable or even continuous. Furthermore, by blending exploitation with exploration behavior, they may be able to overcome the local optimality issues that make nonconvex optimization difficult. The optimization of nonconvex functions may arise, e.g., in model calibration or parameter estimation by maximum likelihood (see Chapter 4), when the underlying model is complicated enough.

As the reader can imagine, there is an enormous range of models and methods involved, which cannot be covered in a single chapter. Nevertheless, we offer here a set of basic examples in order to get a taste for this wide and quite interesting application field. In Section 10.1 we outline a classification of optimization models, for the sake of the uninitiated. Then, we provide a few concrete examples in Section 10.2, with due emphasis on economic and financial applications. Given our limited aim and scope, we will _not_ go through traditional concepts like Lagrange multipliers in nonlinear programming or the simplex method for linear programming. When necessary, we will just use R functions from suitable packages implementing these methods. Emphasis is partly on modeling and partly on alternative methods, which are illustrated in Section 10.3 for global (nonconvex) optimization. These methods can also be applied to simulation-based optimization, which is the subject of Section 10.4. In Section 10.5 we illustrate model building in stochastic programming. We cannot cover specialized solution methods for this class of models, but we insist on scenario generation in Section 10.5.3, illustrating important concepts such as in-sample and out-of-sample stability. The last three sections are devoted to stochastic dynamic programming. The cornerstone principle of dynamic programming, the recursive functional equation due to Bellman, is introduced in Section 10.6, where we also compare this approach against stochastic programming with recourse. In Section 10.7 we describe "standard" numerical methods for dynamic programming, which are well-suited to problems with a rather moderate size, whereas in Section 10.8 we describe ADP strategies that are aimed at larger scale problems and feature interesting links with machine learning.

# 10.1 Classification of optimization problems

A fairly general statement of an optimization problem is

(10.1)

which highlights the following three building blocks:

  A vector of decision variables x, representing a solution of the problem in mathematical form.

  A feasible set _S_ , also called feasible region, to which x must belong; in this book we consider _S_ ⊆  _n_ , i.e., decisions are represented by an _n_ -dimensional tuple of real numbers.

  An objective function _f_ (·), mapping each feasible solution x   _S_ to a performance measure _f_ (x)    , which we are interested in optimizing.

This rather dry and abstract statement encompasses an incredible variety of decision problems. Note that there is no loss of generality in considering minimization problems, as a maximization problem can be easily converted into a minimization one:

Indeed, many software tools assume that the problem is either in minimization or in maximization form and, if necessary, leave the burden of changing the sign of the objective function to the user. In other cases, a flag is used to characterize the kind of problem.

In this book we only deal with finite-dimensional optimization problems, involving a finite number of decision variables. In continuous-time optimal control problems, where system dynamics is described by differential equations, decisions may be represented by functions _u_ ( _t_ ) where _t_   [ _t_ 0, _T_ ]; hence, such problems are infinite-dimensional. We do consider dynamic optimization problems in this chapter, but we assume that time has been discretized; this results in a finite-dimensional problem when the time horizon is finite. When the time horizon is infinite, in principle we do have an infinite-dimensional problem (though countably so); however, if the system is stationary, we may look for a stationary policy by dynamic programming. Dynamic programming relies on functional equations whose unknown are, in fact, infinite-dimensional objects if the underlying state space is continuous. As we shall see, we always boil down the overall problem to a sequence of finite-dimensional subproblems by suitable discretizations. Hence, what we really deal with are mathematical programming problems, but we will use this term and the more general "optimization problem" interchangeably.

When _S_ ≡  _n_ we speak of an _unconstrained_ optimization problem; otherwise we have a _constrained_ optimization problem. The feasible region _S_ , in concrete terms, is typically described by a set of of equality and inequality constraints, resulting in the mathematical program:

(10.2)

The condition **x**   _X_ may include additional restrictions, such as the integrality of some decision variables. As we shall see, the most common form of this restriction concerns binary decision variables, i.e., _x_ _j_   {0, 1}; this trick of the trade is useful to model logical decisions like "we do it" vs. "we do not." Simple lower and upper bounds, describing box constraints like _l_ _j_ ≤ _x_ _j_ ≤ _u_ _j_ , are usually considered apart from general inequalities for the sake of computational efficiency. So, we might also have _X_ = {x    _n_ | **1** ≤ **x** ≤ **u** }, where vectors **1** and **u** collect the lower and upper bounds on decision variables, respectively.

An optimization problem can be very simple or on the verge of intractability, depending on some essential features of its building blocks. Such features are often more important than the sheer size of the problem. We will consider concrete modeling examples later, shedding light on such features and showing the astonishing variety of problems that we may address, but an essential classification framework involves the following dimensions:

  Convex vs. nonconvex problems

  Linear vs. nonlinear problems

  Deterministic vs. stochastic problems

### Convex vs. nonconvex optimization problems

By far, the most important property of an optimization problem is convexity. Convexity is a property of sets, in particular of the feasible set _S_ , which can be extended to a property of functions, in our case, the objective function.

**DEFINITION 10.1 (Convex sets)** _A set S_ ⊆  _n_ _is a convex if_

Convexity can be grasped intuitively by observing that points of the form λ **x** \+ (1 − λ) **y** , where 0 ≤ λ ≤ 1, are simply the points on the straight-line segment joining **x** and **y**. So, a set _S_ is convex if the line segment joining any pair of points **x, y**   _S_ is contained in _S._ This is illustrated in Fig. 10.1: _S_ 1 is convex, but _S_ 2 is not. _S_ 3 is a discrete set and it is not convex; this fact has important consequences for discrete optimization problems. It is easy to see that the intersection of convex sets is a convex set. This does not necessarily apply to set union.

**FIGURE 10.1** **An illustration of set convexity.**

###   Example 10.1 Polyhedra are convex sets

Polyhedral sets are essential in describing the feasible set of a linear programming problem. A hyperplane **a** _i_  **x** = _b_ _i_ , where _b_ _i_     and **a** _i_ , **x**    _n_ , divides  _n_ into two _half-spaces_ expressed by the linear inequalities **a** _ _i__  **x** ≤ _b_ _i_ and **a** _i_  **x** ≥ _b_ _i_. Note that we always assume column vectors, so we use transposition to specify a row vector **a** _i_ . A _polyhedron P_ ⊆  _n_ is a set of points satisfying a finite collection of linear inequalities, i.e.,

where matrix **A**    _m,n_ collects row vectors **a** _i_  , _i_ = 1,..., _m_ , and column vector **b**    _m_ collects the right-hand sides _b_ _i_. Note that a linear equality can be rewritten as two inequalities with opposite senses. A polyhedron is therefore the intersection of a finite collection of half-spaces and is convex. As we will discuss later (see Eqs. (10.4) and (10.5)), the feasible set of a linear programming problem can be described, e.g., as _S_ = { **x**    _n_ | **Ax** = **b, x** ≥ 0} or _S_ = { **x**    _n_ | **Ax** ≥ **b** }, and it is a polyhedron.

###   Example 10.2 The union of convex sets need not be convex

The closed intervals [2, 5] and [6, 10] are clearly convex sets. However, their union [2, 5] ∪ [6, 10] is not. As a more practically motivated example, consider an activity whose level is measured by the decision variable _x_ _j_ ; in concrete, _x_ _j_ may represent the weight of an asset in a financial portfolio. A possible restriction is

meaning that we may not invest in that asset, i.e., _x_ _j_ = 0, but if we do, there are a lower bound _l_ _j_ and an upper bound _u_ _j_ on its weight. This restriction specifies a nonconvex set, and it should not be confused with the box constraint _l_ _j_ ≤ _x_ _j_ ≤ _u_ _j_ , which corresponds to a convex set. Later, we shall see how we can represent such a nonconvexity by the introduction of binary decision variables.

Set convexity can be extended to function convexity as follows.

**DEFINITION 10.2 (Convex functions)** _A function f_ :  _n_ →  , _defined over a convex set S_ ⊆  _n_ , _is a_ convex function _on S if, for any_ **y** _and_ **z** _in S, and for any_ λ   [0, 1], _we have_

(10.3)

The definition can be interpreted by looking at Fig. 10.2. If we join any two points on the function graph with a line segment, all of the segment lies above the function graph. In other words, a function is convex if its epigraph, i.e., the region above the function graph, is a convex set. If the condition (10.3) is satisfied with strict inequality for all **y** ≠ **z** , the function is _strictly_ convex. We illustrate examples of convex and nonconvex functions in Fig. 10.3. The first function is convex, whereas the second is not. The third function is a _polyhedral_ convex function, and it is kinky. This example shows that a convex function need not be differentiable everywhere. Note that in the second case we have a local minimum that is not a global one. Indeed, convexity is so relevant in minimization problems because it rules out local minima.

**FIGURE 10.2** **An illustration of function convexity.**

**FIGURE 10.3** **Convex and nonconvex functions.**

###   Example 10.3 Local minima in polynomial optimization

Consider the following unconstrained problem:

The objective function is a polynomial, which can be dealt with in R by the polynom package, as illustrated by the script in Fig. 10.4. The script produces the following output:

**FIGURE 10.4** **Local minima in polynomial optimization.**

    out1 <- optim(par=4,fn=f,method="BFGS")
    > out1$par; out1$value
    [1] 3.643656
    [1] −0.6934635
    out2 <- optim(par=1,fn=f,method="BFGS")
    > out2$par; out2$value
    [1] 1.488273
    [1] −1.875737

showing that we end up with different solutions, depending on the initial point. The script also produces the plot in Fig. 10.5: We observe that the polynomial features two local minimizers, one of which is also the global one, as well as one local maximizer. The function goes to infinity when _x_ is large in absolute value, so there is no global maximizer. With a high-degree polynomial, there are potentially many local maxima and minima; in fact, multivariable polynomial optimization is, in general, a difficult nonconvex problem.

**FIGURE 10.5** **Global and local optima of a polynomial.**

When solving a minimization problem, it is nice to have a convex objective function. When the objective has to be maximized, it is nice to deal with a concave function.

**DEFINITION 10.3 (Concave function)** _A function f is_ concave _if_ (− _f_ ) _is convex._

Thus, a concave function is just a convex function turned upside down. We have observed that a convex function features a convex epigraph; hence, function convexity relies on set convexity. A further link between convex sets and convex functions is that the set _S_ = { **x**    _n_ | _g_ ( **x** ) ≤ 0} is convex if _g_ is a convex function.

###   Example 10.4 Variations on a circle

Consider the function _g_ ( **x** ) = _x_ 12 \+ _x_ 22 − 1. The constraint _g_ ( **x** ) ≤ 0 describes a circle of radius 1, which is clearly a convex set. Indeed, _g_ ( **x** ) is a convex function. However, the constraint _g_ ( **x** ) ≥ 0 describes the _exterior_ of the circle, which is clearly nonconvex. Indeed, the latter constraint is equivalent to − _g_ ( **x** ) ≤ 0, where the function describing the set is concave. The equality constraint _g_ ( **x** ) = 0 describes the boundary of the circle, i.e., the circumference, which again is nonconvex.

In general, equality constraints like

do not describe convex sets. We may understand why by rewriting the equality as two inequalities:

There is no way in which _h_ (·) and − _h_ (·) can be both convex, unless the involved function is affine:

It is pretty intuitive that convexity of the objective function makes an unconstrained problem relatively easy, since local minima are ruled out. But what about convexity of the feasible set of a constrained problem? Actually, convexity must be ensured in both the objective function and the feasible set, as shown by the next example.

###   Example 10.5 Nonconvexities in the feasible set

Even if the objective function is convex, nonconvexities in the feasible set may make the problem difficult to solve. In Fig. 10.6 we observe two level curves of a convex objective function like _f_ ( _x_ 1, _x_ 2) = ( _x_ 1 − α)2 \+ ( _x_ 2 − β)2. The feasible set is a sort of bean and is nonconvex. We may notice that _B_ is the global minimum, as there the feasible set is tangent to the lowest level curve. However, we also have a local optimum at point _A_ : In its neighborhood there is no better feasible solution.

**FIGURE 10.6** **Local optima due to a nonconvex feasible set.**

**DEFINITION 10.4 (Convex optimization problem)** _The minimization problem_ min **x**  _S_ _f_ ( **x** ) _is convex if both S and f_ (·) _are convex. The maximization problem_ max **x**  _S_ _f_ ( **x** ) _is convex if S is convex and f is concave._

Note that we also speak of a convex problem when we are maximizing a concave objective. We speak of a concave problem when we _minimize_ a concave objective on a convex set. Concave problems are not as nice as convex problems, as the following example illustrates.

###   Example 10.6 A concave problem

Consider the following one-dimensional problem:

This is a concave problem, since the leading term in the quadratic polynomial defining the objective is negative, so that the second-order derivative is negative everywhere. In Fig. 10.7 we show the objective function and the feasible set. The stationarity point _x_ = 2 is of no use to us, since it is a maximizer. We see that local minimizers are located at the boundary of the feasible set. A local minimizer lies at the left boundary, _x_ = 1, and the global minimizer is located at the right boundary, _x_ = 4.

**FIGURE 10.7** **A concave problem may have local optima, but they lie on the boundary of the convex feasible set.**

It turns out that the property illustrated in the example, i.e., there is an optimal solution on the boundary of the (convex) feasible set, does apply to concave problems in general. This kind of structure may be exploited in global optimization procedures.

### Linear vs. nonlinear optimization problems

An optimization model in the concrete form (10.2) can be linear or nonlinear. We have a linear programming (LP) problem if all of the involved functions are linear affine, as in the following model:

(10.4)

where  , and  . Clearly, such a problem makes sense if _m_ < _n._ The above form is an LP in standard form, involving only equality constraints and non-negative decision variables. Another way to state the problem is the canonical form

(10.5)

Using tricks of the trade, it is possible to convert any LP in one of the two forms above. The important point is that LP is both a convex and concave programming problem. This implies that any local optimum is also a global one, and that we may just bother with points on the boundary of the feasible region. More precisely, we have an optimal solution corresponding to a vertex of the feasible region, which is polyhedral. This is exploited in the celebrated simplex algorithm, which is available in plenty of commercial packages. There are also alternative strategies based on interior point methods.

###   Example 10.7 The linprog **package**

Say that we want to solve the following LP:

We may resort to the linprog package, which provides us with the function solveLP to solve LPs by the simplex algorithm. The following script solves the problem:

    library(linprog)
    Amat <- rbind(c(15,35),c(25,15),c(1, 0),c(0,1))
    bvet <- c(2400,2400,100,50)
    cvet <- c(45,60)
    result <- solveLP(cvet,bvet,Amat,maximum=TRUE)
    cat('optimal obj', result$opt, '\n')
    cat('optimal x', result$solution, '\n')

Note that we have to include the upper bounds in the matrix of generic linear constraints; better solvers allow us to keep bounds apart. The resulting output is

    optimal obj 5538.462
    optimal x 73.84615 36.92308

If even one function involved in problem (10.2) is nonlinear, be it the objective or a function defining an equality or inequality constraint, we step into the domain of nonlinear programming. Contrary to what one may imagine, the real difficulty boundary is not marked by the linear/nonlinear dichotomy. The defining property is convexity.

For instance, we lose convexity in LP if we restrict a subset of variables to take only integer values: We get a mixed-integer LP model, MILP for short, which is nonconvex and requires much more complicated solution procedures. In principle, since there is no easy optimality condition to check for MILPs, enumeration of solution is the only viable approach; partial enumeration strategies based on branch and bound methods are widely available, even though they may be quite expensive in terms of computational effort. On the contrary, there are nonlinear programming problems that are rather easy to solve. One such example is convex quadratic programming (QP):

(10.6)

In QPs, we have an objective involving a quadratic form, whereas all of the constraints are linear. The problem is easy if the quadratic form is convex, which is the case if the matrix Q is positive semidefinite. It is worth noting that a quite relevant example of positive semidefinite matrix in financial applications is a covariance matrix; see Section 10.2.1.

###   Example 10.8 The quadprog **package**

Say that we want to solve the following QP:

with obvious solution **x** * = [2, 2] . We may resort to the quadprog package. The following script solves the problem:

        library(quadprog)
        Qmat <- rbind(c(2,0),c(0,2))
        bvet <- c(4)
        Amat <- matrix(c(1,1),nrow=2)
        result <- solve.QP(Qmat, c(0,0),Amat,bvet,meq=1)
        cat('optimal obj', result$value, '\n')
        cat('optimal x', result$solution, '\n')

The resulting output is

        optimal obj 8
        optimal x 2 2

We note a few peculiarities of solve.QP:

  We have to double the matrix **Q** , since the objective function is assumed in the form shown in Eq. (10.6), with a leading 1/2.

  We have to specify that the first meq=1 constraints are equalities, whereas all of the rest of them (none, here) are assumed to be inequalities.

  We have to transpose the matrix **A** , as for some reason solve.QP assumes constraints in the forms **A**  **x** ~ **b** , where ~ can be = and/or ≥.

As we show later, QPs have a long tradition in finance. Another class of relevant problems is quadratically constrained QP (QCQP), which involves convex quadratic forms in the constraints as well. There are other classes of nonlinear, yet convex problems for which quite efficient solution procedures are available. The more general nonlinear programming problems are typically harder but not intractable, provided that convexity applies. On the contrary, nonconvex problems require global optimization methods and are quite challenging. This is where Monte Carlo methods come into play.

### Deterministic vs. stochastic optimization problems

In most interesting and realistic problems we have to cope with some form of uncertainty in the data. This is certainly the case in financial problems, where return on investments is uncertain. We may have even problems with an uncertain time horizon, such as pension planning. A rather abstract form of a stochastic optimization problem is

(10.7)

which is an obvious generalization of the abstract problem (10.1). Here the objective function depends on decision variables **x** and random variables ξ. Taking the expectation eliminates ξ, yielding a function of **x**. Assuming that the random variables are continuous with joint density _g_ ξ( **0** ), we have

The random variables represent uncertainty that is realized _after_ we make the decision **x** , as illustrated in the following simple example.

###   Example 10.9 The newsvendor problem

The newsvendor problem is a prototypical problem in supply chain management and concerns production or purchasing decisions for items subject to perishability or obsolescence risk, like fashion goods. There is a limited time window for sales, and we must decide the order quantity _q_ before knowing the realization of a random demand _D_ during the time window. Items are bought (or produced) at unit cost _c_ and sold at the full retail price _p_ > _c_ within the sales time window; any unsold item after the time window must be marked down and sold at price _p_ _u_ < _c_. If the case _q_ < _D_ occurs, we have an opportunity cost corresponding to the lost profit margin _m_ ≡ _p_ − _c_ for each unit of unsatisfied demand. On the other hand, when _q_ > _D_ we incur a cost _c_ _u_ ≡ _c_ − _p_ _u_ for each unsold item. The newsvendor problem seems quite unrelated with financial problems. Nevertheless, to see a useful connection, imagine a retail bank that has to decide on the amount of cash required to meet withdrawal requests; ideally, we would like to set apart exactly the cash needed, and any surplus or shortfall is associated with a cost. Furthermore, the newsvendor problem is a simple decision problem with an analytical solution, which we use later to illustrate issues in scenario generation.

Let us use the notation _X_ + ≡ max{0, _X_ } and express profit as:

(10.8)

Our problem is the maximization of expected profit,

(10.9)

which can also be recast in terms of cost minimization:

(10.10)

To see the equivalence of models (10.9) and (10.10), observe that the expectation of Eq. (10.8) involves the the expected demand, which is a given constant and does not play any role. In the second form (10.10), we associate a cost _c_ _u_ with each unit of surplus and a cost _m_ with each unit of shortfall with respect to demand. In the case of a continuous demand distribution, it is easy to show that the optimal solution solves the equation

(10.11)

where _F_ _D_ ( _x_ ) ≡ _P_ { _D_ ≤ _x_ } is the cumulative distribution function (CDF) of demand. In other words, the profit margin _m_ and the cost of unsold items _c_ _u_ define a quantile of the demand distribution. If demand is normal, _D_ ~ N(μ, σ2), then _q_ * = μ + _z_ σ, where _z_ is the standard normal quantile corresponding to a probability level given by the ratio in Eq. (10.11). Note that, in general, the optimal solution is _not_ to order the expected value of demand. If uncertainty is significant, we should tilt our decision toward larger values if the critical ratio is larger than 0.5, and toward smaller values when it is smaller than 0.5. The important takeaway is that, when faced with an optimization problem involving uncertain data, we should _not_ solve a deterministic problem based on their expected values.

The formulation (10.7) actually encompasses cases in which we are not really able to formulate the problem as a mathematical program explicitly. It may be the case that the form of the objective function is not known, but we can only estimate its value by a simulation model. At first, this may sound weird, but there are indeed optimization problems in which we are not quite able to evaluate the objective function exactly. In fact, the objective function could be related to the performance of a complex dynamical system, such that there is no closed-form expression of the objective function. Under uncertainty, the decisions themselves may not be explicit, but given in the form of a decision rule to be applied on the basis of the current state and the previous realization of uncertain parameters. In such a setting, we have to resort to simulation-based optimization approaches, where Monte Carlo methods play a prominent role.

While the formulation of the model (10.7) is general, it obscures two different sides of the coin:

  Uncertainty in optimality

  Uncertainty in feasibility

Infeasibility is not an issue in the newsvendor model above, but it is in general. We will be more specific on how stochastic optimization can be addressed in Section 10.5.

# 10.2 Optimization model building

The ability of solving an optimization problem is of no use without the ability of building a sensible optimization model. Model building is a difficult art to learn, possibly a bit out of the scope of this book. Nevertheless, it is useful to illustrate a few relevant examples to get a glimpse of the resulting model classes:

  Financial portfolio optimization, possibly including qualitative constraints on portfolio composition that require the introduction of logical decision variables

  Asset pricing problems, in which emphasis is on the value of the objective function rather than the decision variables

  Parameter estimation and model calibration problems

## 10.2.1 MEAN–VARIANCE PORTFOLIO OPTIMIZATION

Mean–variance optimization is the prototypical _static_ portfolio optimization model. The decision concerns setting the weights _w_ _i_ , _i_ = 1,..., _n_ , of a set of _n_ assets, in such as way to strike a satisfactory trade-off between expected return and risk. Let

  μ = [μ1,..., μ _n_ ]  be a vector collecting the expected return of each asset.

  μ _T_ be a target expected return that we wish to achieve.

  ∑    _n,n_ be the covariance matrix collecting the covariances σ _ij_ between the return of assets _i, j_ = 1,..., _n_.

If we assume that risk can be measured by the standard deviation of portfolio return, one sensible model for the asset allocation decision is:

(10.12)

Here we minimize variance, rather than standard deviation, since this yields an equivalent, but highly tractable convex quadratic programming problem. The constraint (10.12) enforces the achievement of the target expected return; then we also require that portfolio weights add up to 1, and the non-negativity requirements forbid short-selling.

From a computational point of view, this is a trivial problem that can be solved quite efficiently. However, the model itself has a few weak points:

**1.** Using standard deviation as a risk measure might make sense for symmetric distributions, such as a multivariate normal; however, asset returns are not quite symmetric, especially if financial derivatives are involved. Alternative risk measures, such as conditional value-at-risk can be adopted; see Section 13.5.

**2.** The model is static and does not allow for dynamic rebalancing. We may formulate a full-fledged multiperiod model, as we do in Section 10.5.2; alternatively, we may simplify the problem by enforcing some structure on the investment policy, as we do in Section 10.2.4.

**3.** Even if we stay within the easier domain of static optimization models, we may wish to enforce additional constraints on the portfolio, such as an upper bound on the number of stock names in our portfolio, as we do in Section 10.2.2 below.

These variations on the theme lead to nonconvex and/or stochastic optimization models, which may be quite challenging to formulate and solve; Monte Carlo methods may play a pivotal role in tackling them, in one way or another.

## 10.2.2 MODELING WITH LOGICAL DECISION VARIABLES: OPTIMAL PORTFOLIO TRACKING

One of the features that may make an optimization problem nonconvex and much harder to solve is the presence of decision variables restricted to integer values. By far, the most common case involves logical (or binary) decision variables, taking values in the set {0, 1}. We first outline a few standard modeling tricks involving such variables, and then we describe a practically relevant portfolio optimization model requiring their introduction.

### Modeling fixed charges

Consider an economic activity whose level is measured by a continuous decision variable _x_ ≥ 0. The total cost TC( _x_ ) related to the activity may consist of two terms:

  A variable cost, which is represented by a term like _cx_ , where _c_ is the unit variable cost.

  A fixed charge _f_ , which is only paid if _x_ > 0, i.e., if the activity is carried out at a positive level.

In principle, we could introduce a step function such as

and express total cost as

Unfortunately, the step function is nonlinear and discontinuous at the origin, as it jumps from 0 to 1. To avoid the resulting technical difficulty, we may resort to an alternative representation by introducing the following binary decision variable:

Then, we link _x_ and δ by the so-called _big-M constraint:_

(10.13)

where _M_ is a suitably large constant. To see how Eq. (10.13) works, imagine that δ = 0; then, the constraint reads _x_ ≤ 0, whatever the value of _M_ is. This constraint, together with the standard non-negativity restriction _x_ ≥ 0, enforces _x_ = 0. If δ = 1, the constraint is _x_ ≤ _M_ , which is nonbinding if _M_ is large enough; in practice, _M_ should be a sensible upper bound on the level of activity _x_.

### Semicontinuous variables

A possible requirement on the level of an activity is that, if it is undertaken, its level should be in the interval [ _m_ _i_ , _M_ _i_ ]. Note that this is _not_ equivalent to requiring that _m_ _i_ ≤ _x_ _i_ ≤ _M_ _i_. Rather, we want something like

which involves a nonconvex set (recall that the union of convex sets need not be convex). For instance, we may use a constraint like this to avoid the inclusion of assets with negligible weights in a portfolio. Using the big- _M_ trick as above, we may write

These constraints define a _semicontinuous decision variable_.

### Portfolio tracking and compression

Active portfolio managers aim at achieving superior return by tacking advantage of skill or analytical capabilities. On the contrary, passive portfolio managers aim at replicating an index or tracking a benchmark portfolio as close as possible. There is some theoretical support for the passive view, as the efficient market hypothesis, in its various forms, essentially states that there is no way to beat the market. The capital asset pricing model, too, supports the view that the market portfolio is the optimal risky component in an overall portfolio. The matter is highly controversial, but one thing can be taken for granted: A passive portfolio must have a very low management cost. One way to achieve this is to restrict the number of asset names included in the portfolio, i.e., to enforce a cardinality constraint.

To formulate the problem, let us consider a universe of _n_ assets and a benchmark portfolio with given weights _w_ _i_ _b_ , _i_ ,..., _n_. We want to find the weights _w_ _i_ ≥ 0 of a tracking portfolio, in such a way to minimize a measure of distance between the target and the tracking portfolio. An additional restriction is that at most _C_ max assets can be included, i.e., at most _C_ max weights can be strictly positive (we assume that short-selling is forbidden). In order to express the cardinality constraint, we introduce a set of binary variables δ _i_ , one for each asset, modeling the inclusion of asset _i_ in the tracking portfolio:

The binary variables δ _i_ and the continuous variables _w_ _i_ are linked by the big- _M_ constraint

(10.14)

and the cardinality constraint reads as

Since a weight cannot be larger than 1, when we rule out short-selling, we can set _M_ = 1 in Eq. (10.14); the big- _M_ constraints can be tightened by setting _M_ =  _i_ , where  _i_ is some suitable upper bound on the weight of asset _i_.

Now the problem is how to measure the portfolio tracking error. A trivial distance metric can be defined by an _L_ 1 norm:

By proper model formulation, this metric yields an MILP model. However, this distance metric does not take any return model into account. To see the point, let us consider the following weights for two assets:

According to the _L_ 1 norm, the distance between the target and the tracking portfolio, considering only these two assets, is

However, what is the distance if the two assets are perfectly correlated? Of course, the actual distance would be zero, in such a case. More generally, the actual distance depends on the link between the two returns. An alternative distance metric is TEV (tracking error variance), defined as

where _R_ _p_ and _R_ _b_ are the return of the tracking portfolio and the benchmark, respectively. Then, TEV can be expressed as

where σ _ij_ is the covariance between the returns of assets _i_ and _j_. Straightforward minimization of TEV, subject to a cardinality constraint, yields the following model:

(10.15)

This model is a mixed-integer quadratic programming problem with a convex relaxation. Commercial solvers are able to tackle this model structure. Nevertheless, some stochastic search methods, such as the sampling based algorithms described in Section 10.3, have been proposed as alternatives for large-scale problem instances.

## 10.2.3 A SCENARIO-BASED MODEL FOR THE NEWSVENDOR PROBLEM

We have introduced the newsvendor problem in Example 10.9. The basic version of the problem can be solved analytically, and the same applies to many interesting variants. Nevertheless, it is instructive, as well useful for what follows, to consider a scenario-based model formulation. The idea is to approximate the probability distribution for demand by a set of _S_ scenarios, characterized by a probability π _S_ and a demand realization _D_ _s_ , _s_ = 1,..., _S_. The main decision variable is the ordered amount _q_ ≥ 0 and, in order to express the objective in cost minimization form, we introduce two non-negative deviations with respect to realized demand:

(10.16)

where _y_ + _s_ is a surplus and _y_ − _s_ is a shortfall. The, the model can be formulated as an LP model:

(10.17)

As we shall discover later, this can be considered as a very simple example of two-stage stochastic LP model. From a practical point of view, it is a basis for more interesting models allowing for interactions among multiple products, e.g., by demand substitution.

## 10.2.4 FIXED-MIX ASSET ALLOCATION

In Section 10.2.1 we have considered a static, single-period asset allocation model. Later, in Section 10.5.2, we will consider a full-fledged multiperiod model, allowing for dynamic portfolio adjustments. An intermediate strategy can be devised by specifying a fixed-mix, i.e., by keeping portfolio weights _w_ _i_ constants over multiple periods. Wealth will change along the way, and to keep the relative weights constant, we will sell high-price assets, and buy low-price assets. Thus, the strategy is intrinsically a contrarian one, rather than a trend-following one. Let us also assume that we only care about mean and variance, as we did before, and that our mean–risk objective is the expected value of terminal wealth penalized by its variance. Unlike the static case, expressing variance of the terminal wealth is not easily accomplished; hence, we resort to a set of _S_ return scenarios. Each scenario is a history of total asset returns _R_ _it_ _s_ , over a planning horizon consisting of time periods _t_ = 1,..., _T_. Each scenario is associated with a probability π _s_ , _s_ = 1..., _S_. The model we describe here is due to [32], to which we refer the reader for further information and computational experiments, and is basically an extension of the mean–variance framework.

Let _W_ 0 be the initial wealth. Then, wealth at the end of time period 1 in scenario _s_ will be

Note that wealth is scenario-dependent, but the asset allocation is not. In general, when we consider two consecutive time periods, we have

Unfolding the recursion, we see that wealth at the end of the planning horizon is

Within a mean–variance framework, we may build a quadratic utility function depending on the terminal wealth. Given a parameter λ related to risk aversion, the objective function is

To express the objective function, we recall that Var( _X_ ) = E[ _X_ 2] − E2[ _X_ ] and write the model as

This looks like a very complex problem; however, while the objective function is a bit messy, the constraints are quite simple. The real difficulty is that this is a nonconvex problem. To see why, just note that the objective turns out to be a polynomial in the decision variables; since polynomials may have many minima and maxima (see Example 10.3), we face a nonlinear nonconvex problem. Once again, stochastic search algorithms, based on Monte Carlo sampling, can be used to cope with these problems.

## 10.2.5 ASSET PRICING

Asset pricing is a central problem in financial engineering, as well as in financial economics and corporate finance. We know from Section 3.9.2 that the price of a European-style option can be expressed as the discounted expected payoff, under the risk-neutral measure. Assuming a constant risk-free rate _r_ , with continuous compounding, we have

(10.18)

where _S_ 0 is the underlying asset price now and Φ( _S_ _T_ ) is the option payoff, depending on the underlying asset price _S_ _T_ at maturity. In principle, pricing a European-style option by Monte Carlo methods is easily accomplished, provided that we can generate sample paths efficiently; if we do not consider hedging issues, we have just to sit down, cross our fingers, and wait until maturity. However, there are assets whose value depends on our choices along the way. Generally speaking, the value of a resource depends on how the resource itself is used. American-style options, unlike their European-style counterparts, can be exercised at any date prior to expiration. One easy conclusion is that an American-style option cannot cost less than the corresponding European one, as it offers more opportunities. However, early exercise opportunities make pricing much harder, as we should consider an optimal exercise strategy, which in turn calls for the solution of a stochastic dynamic optimization problem. At each exercise opportunity, if the option is in-the-money, we have to compare the _intrinsic value_ of the option, i.e., the immediate payoff that we may earn from exercising the option early, and the _continuation value_ , i.e., the value of postponing exercise and waiting for better opportunities.

Formally, the price of an American option can be written as

(10.19)

The difference with Eq. (10.18) is that now we have to solve an optimization problem with respect to a stopping time τ. The term "stopping time" has a very precise meaning in the theory of stochastic processes, but here we may simply interpret it as the time at which we exercise the option, according to some well-defined strategy.

###   Example 10.10 An American-style put option

A put option is in-the-money at time _t_ if _S_ ( _t_ ) < _K_. Immediate exercise in this case yields a payoff _K − S_ ( _t_ ). Clearly, early exercise will not occur if the option is not in-the-money, but even if _S_ ( _t_ ) < _K_ it may be better to keep the option alive and wait. This means that early exercise will occur only if the option is "enough" in-the-money. By how much, it will generally depend on time to expiration. When we are close to option expiration, there are less opportunities, and we are more inclined to exercise; but when we are far from expiration, we are more inclined to wait. Formally, we have to find a boundary separating the early exercise region, where the intrinsic value is larger than the continuation value, from the continuation region. Qualitatively, for an American put option we would expect an early exercise boundary like the one depicted in Fig. 10.8. This boundary specifies a stock price _S_ *( _t_ ) such that if _S_ ( _t_ ) < _S_ *( _t_ ), i.e., if the option is deeply enough in-the-money, then we exercise. If we are above the boundary, we are in the continuation region, and we keep the option alive. Finding the boundary is what makes the problem difficult; for a detailed treatment of the exercise boundary for American options see, e.g., [31, Chapter 4].

**FIGURE 10.8** **Qualitative sketch of the early exercise boundary for a vanilla American put. The option is exercised within the shaded area.**

From the last example, we understand that the time at which the option will be exercised is indeed a random variable, in this case the time at which we cross the exercise boundary, if we cross it. More complicated strategies are conceivable and must be applied to exotic and multidimensional options, and this is what the stopping time captures. The stopping time is related to any conceivable strategy, provided that it is nonanticipative; we may only use information gathered from time 0 up to time _t_. From a practical perspective, in order to price an option with early exercise features, we have to solve a possibly nontrivial optimization problem. For low-dimensional options, numerical methods based on suitable discretizations of the state space, like binomial or trinomial lattices, or finite-difference methods to solve partial differential equations work very well. For high-dimensional or path-dependent options, we have to resort to Monte Carlo methods. In the past, this was considered impossible, but by framing the problem within a stochastic dynamic programming framework, we may tackle it, as we shall see in Section 11.3.

## 10.2.6 PARAMETER ESTIMATION AND MODEL CALIBRATION

We have introduced maximum-likelihood estimation in Section 4.2.3. Parameter values are chosen in such a way as to maximize a likelihood function related to a selected probability distribution. In very simple cases, the solution is easily found analytically. In nontrivial cases, numerical optimization methods are needed for the solution of the resulting problems, possibly because of the complexity of the likelihood function or the addition of constraints on parameter values, on the basis of some additional information we might have. Furthermore, the resulting problem may be nonconvex and call for global optimization strategies.

A similar kind of problem is encountered when we have to calibrate a pricing model. For instance:

  Given a set of bond prices, we might want to infer the term structure of interest rates, i.e., the yield curve.

  Given a set of quoted option prices written on the same underlying asset, we might want to estimate a volatility surface.

###   Example 10.11 Fitting the yield curve

At time _t_ we observe a set of bond prices _P_ _k_ ( _t_ ), _k_ = 1,..., _n_. Let us assume for the moment that the cash flows associated with these bonds, i.e., the payment of coupons along the way and the face value at bond maturity, occur at a set of common time instants _T_ _i_ , _i_ = 1,..., _m_. Let _r_ ( _t, T_ ) denote the continuously compounded interest rate applying over the time interval [ _t, T_ ]. Based on observed bond prices, we want to estimate _r_ ( _t, T_ ) for a range of maturities. According to straightforward bond pricing formulas, the bond prices should be given by

where _c_ _k,i_ is the cash flow from bond _k_ at time _T_ _i_. These predicted prices are compared against the observed prices _P_ _k_ _o_ ( _t_ ). If _m_ = _n_ , the number of unknown parameters and prices are the same, and we may solve a system of nonlinear equations to find the interest rates for maturities _T_ _i_ , where we assume _ __k_ ( _t_ ) = _P_ _k_ _o_ ( _t_ ). A textbook approach, known as bootstrapping, assumes that exactly one bond will mature at each time instant _T_ _i_ ; then, the system of equations has a "triangular" structure and we can solve one equation at a time, finding one rate per equation. In practice, it may be difficult to find a suitable set of bonds lending itself to such a simplified solution strategy. Liquidity and other issues may affect bond prices, introducing noise and some irregularity in the resulting yield curve.

To smooth those effects, it is preferable to choose _m_ > _n_ and minimize an error distance between observed and predicted prices, with respect to rates:

This is a nonlinear least-squares problem that can be solved by numerical optimization methods. However, it is typically desirable not to consider a restricted set of cash flow times _T_ _i_ , in order to widen the set of bonds we may use for calibration. Furthermore, we would like to fit a term structure with some reasonable degree of smoothness, in such a way that we can estimate rates for maturities different from the aforementioned cash flow times _T_ _i_. One way to accomplish this is by spline interpolation, but due to possible pricing anomalies exact interpolation may yield unreasonable results, and least-squares approximation may be preferred. More generally, we may consider a set of basis functions φ _j_ ( _T_ ), _j_ = 1,..., _J_ , whose linear combination

gives the yield curve as a function of maturity _T_. Here, the decision variables of the optimization problem are the coefficients α _j_ , and we might also consider estimating the discount curve directly, which is then recast as the yield curve. The use of basis functions to boil down an infinite dimensional problem to a finite dimensional one is very common, and we will take advantage of the idea later, when dealing with numerical dynamic programming.

The calibration of the yield curve on the basis of quoted prices is an example of an _inverse problem_. Model calibration is also a cornerstone of option pricing in incomplete markets. As we have seen in Section 3.9.3, market incompleteness arises whenever we step out of the safe BSM domain based on GBM process with very simple drift and volatility terms. If volatility is stochastic or the underlying variable is nontradable, we may need a more sophisticated approach. In Eq. (3.126) we have pointed out the existence of an unknown function _m_ *(·, ·) serving as a drift in an Itô differential equation, under a pricing measure. We report the equation here for the sake of convenience, recasting it in terms of a generic state variable _X_ _t_ :

Once again, note that for pricing purposes the drift term is related to a risk-neutral measure, and it cannot be estimated on the basis of a time series of _X_ _t_. By a similar token, we may not want to fit the volatility function on the basis of historical values of volatility; rather, we want to extract the current market view, which is implicit in asset prices. Hence, we are faced with an inverse problem. Depending on the kind of dependence on time, the functions _m_ *(·, ·) and _s_ (·, ·) may depend on a finite or infinite set of parameters. The second case occurs when these terms explicitly depend on time, in order to improve their matching performance against quoted prices. Whatever the case, by a suitable discretization mechanism, possibly based on basis functions, interpolation, and whatnot, we end up with a pricing model depending on a vector of parameters α. The pricing model predicts a price _ __k_ (α) for a set of securities with quoted (observed) price _P_ _k_ _o_ , and it calibrated by solving the optimization problem

One might wonder why we should calibrate a model by fitting a set of unobservable parameters against quoted prices of exchange-traded instruments. Clearly, the aim is not to recover quoted prices per se, but to fit a model in order to price over-the-counter assets and to assess option price sensitivity with respect to a set of risk factors.

Depending on the complexity of the pricing model, we may end up with relatively simple optimization models, amenable to classical nonlinear programming procedures (most notably, the Levenberg–Marquardt algorithm), or an extremely difficult global optimization problem. In the latter case, the Monte Carlo based global optimization methods that we describe in the next section might be the only viable solution method.

# 10.3 Monte Carlo methods for global optimization

When dealing with a convex optimization problem featuring a well-defined objective function, we may take advantage of useful information provided by its gradient and Hessian matrix. Unfortunately, it may be the case that the objective function is badly behaved, i.e., nonconvex, possibly discontinuous, or even not known in analytical form. Then, we may resort to algorithms that rely only on function evaluations, possibly obtained by stochastic simulation, and a search mechanism to explore the solution space randomly. Monte Carlo methods may be used at both levels:

**1.** In this section we consider stochastic search methods for global optimization. The optimization problem, per se, is a purely deterministic one, but we take advantage of random exploration to overcome local optimality issues.

**2.** Later, in Section 10.4, we consider simulation-based optimization, where Monte Carlo simulation is needed to estimate the objective function itself, and a deterministic search strategy is used to explore the solution space.

As the reader can imagine, there is no clear-cut line separating the two ideas, as random sampling may be employed at both levels.

For any specific decision problem it is possible to devise an ad hoc heuristic method providing us with a good, possibly near-optimal, solution. Here we only consider fairly general strategies, which may be customized to the specific problem at hand. Indeed, a wide class of such methods are known under the name of _metaheuristics_ , which emphasizes their generality and flexibility. Indeed, they are able to cope with both continuous nonconvex problems as well as discrete optimization problems, like optimal portfolio tracking, where the nonconvexity arises from the feasible set, which includes binary decision variables. We first introduce local search and other metaheuristic principles in Section 10.3.1. Then, in Section 10.3.2 we describe simulated annealing, a simple stochastic search method inspired by an interesting physical analogy. Other methods, such as genetic algorithms (Section 10.3.3) and particle swarm optimization (Section 10.3.4) rely on analogies with biological systems and, unlike simulated annealing, work on a population of solutions, rather than a single one.

## 10.3.1 LOCAL SEARCH AND OTHER METAHEURISTICS

Local search is a wide class of simple exploration strategies based on the idea of generating a sequence of solutions exploring the neighborhood of the current one. Consider a generic optimization problem

defined over a feasible set _S_. The basic idea is to improve a given feasible solution by the application of a set of local perturbations. We associate a feasible solution _x_ with a neighborhood _N_ ( _x_ ), which is defined by applying a set of simple perturbations to _x_. Different perturbations yield different _neighborhood structures_. The simplest local search algorithm is _local improvement_ : Given a current (incumbent) solution **x** 0, an alternative (candidate) solution **x** ' is searched for in the neighborhood of the incumbent:

If the candidate solution improves the objective function, i.e., _f_ ( **x** ') < _f_ ( **x** 0), then the incumbent solution is replaced by the candidate and the process is repeated. The search is terminated when we cannot find any improving solution, and the incumbent solution is returned. Clearly, there is no guarantee that an optimal solution is found, as the process may get trapped in a bad locally optimal solution. Furthermore the solution is locally optimal with respect to the selected neighborhood structure, which must be simple enough to allow for an efficient search, but not so simplistic that the exploration process is ineffective. If the candidate solution is the optimal solution in the neighborhood, we speak of a _best-improving_ local search; if we stop exploring the neighborhood as soon as an improving solution is found, we speak of a _first-improving_ local search.

###   Example 10.12 Neighborhood structures for local search

Neighborhood structures may be devised for both discrete and continuous optimization problems. If all the components of **x**    _n_ are restricted to binary values, a simple neighborhood structure is obtained by complementing each variable in turn, i.e., for each _i_ = 1,..., _n_ we set

The neighborhood consists of _n_ points and it is easy to explore, but possibly very poor. We may consider also complementing pairs of variables, as well as swapping values of variables having different values.

The case of continuous variables adds a degree of difficulty, as it is not clear by how much we should perturb each component. We may define steps δ _i_ for each component and generate 2 _n_ alternative solutions by applying the perturbation in both directions:

where **e** _i_ is a unit vector with 1 in position _i_ and 0 in the other positions. This is the simplest pattern for searching the neighborhood, but some mechanism should be devised to dynamically adjust the step lengths.

Alternatively, we may generate a random candidate at a time, by sampling randomly in the neighborhood of the incumbent solution using a suitable probability distribution. For instance, we may adopt a multivariate normal distribution with expected value **x** 0 and a diagonal covariance matrix ∑. This approach lends itself to first-improving strategy.

Actually, devising a clever and effective neighborhood is not trivial at all. A further complication is that due attention must be paid to constraints. In some cases we might just discard solutions violating them; however, this is not easily accomplished when dealing with equality constraints, as staying inside the feasible region might be difficult. An alternative strategy is to relax constraints and augment the objective function by a corresponding penalty function. For instance, the constrained problem

may be transformed into the unconstrained problem

for suitably large penalty coefficients ω _i_ , possibly expressing the relative importance of constraints. The penalty coefficients might also be dynamically adjusted, allowing the search process to occasionally leave the feasible region and getting back into it at another point. This kind of approach, called strategic oscillation, may be used, along with other trick of the trade, to balance exploitation (the need to stick to a good solution and explore its neighborhood) and exploration (the need to explore other subregions of the feasible set).

Local improvement is not suitable to global optimization. However, one may relax the requirement that the new solution in the search path is an improving one. By accepting nonimproving candidates, we may hope to escape from local optima. Two such strategies are:

  Simulated annealing, whereby the neighborhood is randomly sampled and a nonimproving solution is accepted with a suitably chosen probability.

  Tabu search, whereby we explore the neighborhood and always accept the best solution found, even if it is nonimproving. Since this may result in repeating a cycle of solutions, tabu attributes are used to control the search process and forbid revisiting solutions. Tabu search, in its basic form, is a deterministic search approach and we will not deal with it.

Both simulated annealing and tabu search work with a single incumbent solution. In order to find a better trade-off between exploration and exploitation, one may work with a _population_ of solutions. Examples of population-based approaches are genetic algorithms and particle swarm optimization.

## 10.3.2 SIMULATED ANNEALING

Simulated annealing owes its name to an analogy between cost minimization in optimization and energy minimization in physical systems. The local improvement strategy behaves much like physical systems do according to classical mechanics. It is impossible for a system to have a certain energy level at a certain time instant then and to increase it without external input: If you place a ball in a hole, it will stay there. This is not true in statistical mechanics; according to these physical models, at a temperature above absolute zero, thermal noise makes an increase in the energy of a system possible. This applies to changes at a microscopic level, and an increase in energy is more likely to occur at high temperatures. The probability _P_ of this upward jump depends on the increase in energy Δ _E_ and the temperature _T_ , according to the Boltzmann distribution

where _K_ is the Boltzmann constant. Annealing is a metallurgical process by which a melted material is slowly cooled in order to obtain a good (low-energy) solid-state configuration. If the temperature is decreased too fast, the system gets trapped in a local energy minimum, and a glass is produced. But if the process is slow enough, random kinetic fluctuations due to thermal noise allow the system to escape from local minima, reaching a point very close to the global optimum.

In simulated annealing we consider the change in the objective function when moving from the incumbent solution to a candidate solution:

The candidate solution is accepted with probability

An improving candidate yields Δ _f_ < 0 and is always accepted. A nonimproving candidate is randomly accepted, according to a probability depending on the amount of cost increase Δ _f_ and the control parameter _T_ , as shown in Fig. 10.9. At very high temperatures, most candidate solutions are accepted, whereas at intermediate temperatures candidate solutions that increase cost are accepted with lower probability if the deterioration is significant. For _T_ → 0 the acceptance probability collapses into a step function, and the method behaves like local improvement. In Section 14.4 we will interpret simulated annealing within the framework of Markov chain Monte Carlo methods, as a way to sample from a density that is concentrated on the set of globally optimal solutions. However, to reach that set without getting trapped in local optima, the temperature should be decreased according to a _cooling schedule_. The simplest cooling schedule is

**FIGURE 10.9** **Acceptance probabilities as a function of cost increase for different temperatures in simulated annealing.**

In practice, it is advisable to keep the temperature constant for a certain number of steps, in order to reach a thermodynamic equilibrium before changing the control parameter. More sophisticated adaptive cooling strategies have been proposed. Furthermore, we have to establish a termination criterion. The simplest idea is to stop the algorithm after a given number of iterations; alternatively, we may stop after a maximum number of iterations without accepting a candidate.

We show an extremely naive implementation of simulated annealing in Fig. 10.10, for illustrative purpose only. We may test it on the Rastrigin function

**FIGURE 10.10** **A naive implementation of simulated annealing.**

which has a global optimum for **x = 0** :

    > Rastrigin<-function(x)sum(x^2-10*cos(2*pi*x))+10*length(x)
    > dim <- 20
    > lb <- rep(−100,dim)
    > ub <- rep (100,dim)
    > set.seed(55555)
    > x0 <- runif(dim,min=lb,max=ub)
    > out <- optim(par=x0,fn=Rastrigin,lower=lb,upper=ub,
    +    method="L-BFGS-B")
    > out[c("value","counts")]
    $value
    [1] 1361.003
    $counts
    function gradient
        43    43
    >
    > set.seed(55555)
    > out <- NaiveSA(fn=Rastrigin,par=x0,temp0=100,
    +            beta=0.9999,maxit=200000,sigma=1)
    > out[c("value","counts")]
    $value
    [1] 125.437
    $counts
    [1] 2e+05

We use _n_ = 20 and start search from a randomly selected point within a box. For compatibility reasons with other R functions that we want to use for comparison purposes, we constrain all decision variables in a box with lower bound −100 and upper bound 100. A classical nonlinear programming method, available as an option in the standard R function optim gets quickly stuck in a bad solution, whereas the naive annealing does better. Actually, we may take advantage of a better implementation of simulated annealing offered by optim:

    > set.seed(55555)
    > out <- optim(par=x0,fn=Rastrigin,method="SANN",
    +            control=list(temp=100,tmax=10,maxit=200000))
    > out[c("value","counts")]
    $value
    [1] 81.33197
    $counts
    function gradient
        200000        NA

Here, tmax is the maximum number of iterations without decreasing the temperature. Still better results are obtained by function GenSA offered in the package having the same name:

    > library(GenSA)
    > set.seed(55555)
    > out <- GenSA(fn = Rastrigin,lower=lb,upper=ub,par=x0)
    > out [c("value","par","counts")]
    $value
    [1] 0
    $par
        1.149916e-11 8.491049e-12 −1.670109e-11 −3.363457e-11
       −2.144013e-11 −4.907911e-13 −3.319979e-11 −3.287746e-11
        6.624013e-12 −2.850014e-11 3.173777e-11 −1.580739e-11
       −2.003915e-13 −2.192858e-09 2.288534e-11 3.180884e-11
        1.039823e-11 −7.188424e-12 3.151424e-11 1.477490e-11
    $counts
    [1] 257528

This function implements a generalized simulated annealing algorithm; see [44] for more information.

## 10.3.3 GENETIC ALGORITHMS

Unlike simulated annealing, genetic algorithms work with a set of solutions rather than a single point. The idea is based on the survival-of-the-fittest mechanism of biological evolution, whereby a population of individuals evolves by a combination of the following elements:

  Selection according to a fitness criterion, which in an optimization problem is related to the objective function.

  Mutation, whereby some features of an individual are randomly changed.

  Crossover, whereby two individuals are selected for reproduction and generate offsprings with features resulting from a mix of their parents' features.

There is a considerable freedom in the way the above idea is translated into a concrete algorithm. A critical issue is how to balance exploration and exploitation. On the one hand, biasing the selection of the new population toward high quality individuals may lead to premature termination; on the other hand, unless a certain degree of elitism is used to keep good individuals into the population, the algorithm might wander around and skip the opportunity of exploring promising subregions of the solution space.

A key ingredient is the encoding of a feasible solution as a string of features, corresponding to the genes of a chromosome. For continuous optimization in  _n_ the choice of representing each solution by the corresponding vector x is fairly natural. Let us consider the simplest form of crossover: Given two individuals x and y in the current pool, a "breakpoint" position _k_   {1, 2,..., _n_ } is randomly selected and two offsprings are generated as follows:

Variations on the theme are possible; for instance, a double crossover may be exploited, in which two breakpoints are selected for the crossover. The two new solutions are clearly feasible for an unconstrained optimization problem in  _n_ ; when constraints are added, this guarantee is lost in general. As we have already mentioned, one can enforce hard constraints by eliminating noncomplying individuals. A usually better alternative is a relaxation by a suitable penalty function.

There are a few packages implementing genetic algorithms in R. Figure 10.11 shows how to use one of them, GA, to minimize the Rastrigin function. Note that we have to set the "real-valued" option to specify that we are dealing with a continuous optimization, and that there are other settings aimed at discrete optimization. Furthermore, genetic algorithms typically maximize a fitness function; hence, we have to change the sign of the Rastrigin function. We obtain the following (edited) output:

**FIGURE 10.11** **Using the GA package to minimize the Rastrigin function.**

We notice that some options related to crossover and mutation probabilities, as well as the degree of elitism, have been set to default values. We may also provide the ga function with customized functions to carry out mutation, crossover, etc. Genetic algorithms offer plenty of customization options, which is a mixed blessing, as it may be awkward to find a robust and satisfactory setting. It is also possible to collect the best solutions found along the way and to refine them by a classical nonlinear programming method, finding a local minimizer nearby.

## 10.3.4 PARTICLE SWARM OPTIMIZATION

Particle swarm optimization (PSO) methods are an alternative class of stochastic search algorithms, based on a population of _m_ particles exploring the space of solutions, which we assume is a subset of  _n_. The position of each particle _j_ in the swarm is a vector

which changes in discrete time ( _t_ = 1, 2, 3,...) according to three factors:

**1.** _Inertia._ Each particle is also associated with a velocity vector **v** _j_ ( _t_ ), which tends to be maintained.

**2.** _Cognitive factor._ Each particle tends to move toward its personal best, i.e., the best point **p** _j_ * that it has visited so far.

**3.** _Social factor._ Each particle tends to move toward the global best of the swarm, i.e., the best point g* that has been visited so far by the whole set of particles.

The idea is to mimic the interactions of members of a swarm looking for food, and a classical statement of the algorithm is the following:

(10.20)

(10.21)

Equation (10.20) governs the evolution of each component _i_ = 1,..., _n_ of the velocity of each particle _j_ = 1,..., _m_. The new velocity depends on:

  The previous velocity, i.e., the inertia factor.

  The difference between the personal best and the current position, i.e., the cognitive factor, scaled by a coefficient _c_ 1 and multiplied by a random variable _r_ 1 _j_ ( _t_ ); a typical choice is a uniform distribution U(0, 1).

  The difference between the global best and the current position, i.e., the social factor, scaled by a coefficient _c_ 2 and multiplied by a random variable _r_ 2 _j_ ( _t_ ); in this case, too, a typical choice is a uniform distribution U(0, 1).

Equation (10.21) simply changes each component of the current position according to the new velocity. At each iteration, the personal and global bests are updated when necessary, and other adjustments are used in order to keep velocity within a given range, as well as the positions, if bounds on the variables are specified.

PSO can be regarded as another Monte Carlo search approach, and several variants have been proposed (see the end of chapter references):

  In small world methods, the global best position for neighboring particles is used, rather than the whole swarm.

  In quantum PSO different rules are used to evolve particle positions. A version of the algorithm generates a new particle position as

where _U_ is a uniformly distributed random variable, β is a coefficient to be chosen, and **p** _j_ ( _t_ ) is a random combination of the personal and the global best. The ± is resolved by the flip of a fair coin, i.e., + and − have both 50% probability.

  Some algorithms use Lévy flights, i.e., the shocks to current positions are generated by using heavy-tailed distributions, which is contrasted against the normal distribution underlying geometric Brownian motion and standard random walks.

  In the firefly algorithm, the quality of each solution corresponds to the light intensity of a firefly attracting other particles.

Some of these algorithms have a physical motivation, whereas some have a biological motivation; they differ in the number of coefficients to be set and the corresponding complexity of fine tuning, though some self-adaptive methods are available. As the reader can imagine, it is easy to get lost because of the sheer number of proposals and combinations. The key is to experiment with some of these variants and find the best combination for the specific problem at hand. A welcome tool is the pso R package, which implements some variants of the approach. The code in Fig. 10.12 shows how to use the psoptim function to minimize the Rastrigin function. The function has a similar interface as the standard optim function, plus several control parameters controlling the maximum number of iterations (maxit), the tracing of execution (trace and REPORT), as well as the possibility of carrying out a local optimization for each particle using a standard nonlinear optimizer (if hybrid is set). The code produces the following output:

**FIGURE 10.12** **Using the pso package to minimize the Rastrigin function.**

    > out <- psoptim(par=x0,fn=Rastrigin,lower=lb,upper=ub,
    +            control=list(maxit=10000,trace=1,REPORT=1000,
    +            hybrid=F))
    S=18, K=3, p=0.1576, w0=0.7213, w1=0.7213, c.p=1.193,
    c.g=1.193, v.max=NA, d=894.4, vectorize=FALSE, hybrid=off
    It 1000: fitness=27.86
    It 2000: fitness=27.86
    It 3000: fitness=27.86
    It 4000: fitness=27.86
    It 5000: fitness=27.86
    It 6000: fitness=27.86
    It 7000: fitness=27.86
    It 8000: fitness=27.86
    It 9000: fitness=27.86
    It 10000: fitness=27.86
    Maximal number of iterations reached
    > out <- psoptim(par=x0,fn=Rastrigin,lower=lb,upper=ub,
    +            control=list(maxit=1000,trace=1,REPORT=100,
    +                    hybrid=T))
    S=18, K=3, p=0.1576, w0=0.7213, w1=0.7213, c.p=1.193,
    c.g=1.193, v.max=NA, d=894.4, vectorize=FALSE, hybrid=on
    It 100: fitness=35.82
    It 200: fitness=28.85
    It 300: fitness=24.87
    It 400: fitness=21.89
    It 500: fitness=18.9
    It 600: fitness=14.92
    It 700: fitness=14.92
    It 800: fitness=14.92
    It 900: fitness=8.955
    It 1000: fitness=6.965
    Maximal number of iterations reached

Indeed, we notice that the hybrid algorithm is more effective, at the cost of a possibly larger computational effort. Some experimentation with control parameters may be needed to fine tune the algorithm performance.

# 10.4 Direct search and simulation-based optimization methods

In the previous section we have considered Monte Carlo-based stochastic search approaches to optimize a deterministic objective function. The difficulty was in the nonconvexity of the objective function, but not in its evaluation. When dealing with optimization under uncertainty, as in the case

the function evaluation itself may be difficult. Thus, we may be forced to adopt Monte Carlo sampling to obtain noisy estimates of the objective function. In well-structured problems, we may build a formal optimization model, which can be tackled by stochastic programming with recourse or stochastic dynamic programming, to be discussed later. However, sometimes the system is so complicated that we may only estimate the objective function by some stochastic simulator acting as a black box: We feed the black box with x and we obtain an estimate of the expected performance. As a consequence, some nice properties such as continuity and differentiability may not be always guaranteed. Population-based approaches, like genetic algorithms and particle swarm optimization may help, since they do not place any restriction on the objective function; furthermore, since they work on a population, they tend to be less vulnerable to estimation errors. We may also keep a pool of the most promising solutions found, which could be simulated to a greater degree of accuracy to yield the optimal solution.

Here we discuss two approaches which may be used for simulation-based optimization:

  _Simplex search_ , not to be confused with the simplex algorithm for linear programming, is a sort of population-based algorithm for local optimization of possibly discontinuous functions.

  _Metamodeling_ aims at building a suitable approximation of the true objective function by regression strategies. This higher-level model may be tackled by traditional nonlinear programming techniques.

We should note that these methods are not aimed at global optimization, unlike those discussed in the previous section. Nevertheless they may be more effective and efficient in specific settings.

## 10.4.1 SIMPLEX SEARCH

The simplex search algorithm, also known as Nelder–Mead method, is a deterministic search strategy relying on functions evaluations only. Rather than working with a single point, it uses a simplex of _n_ +1 points in  _n_. A simplex in  _n_ is the convex hull of a set of _n_ +1 affinely independent points **x** 1,..., **x** _n_ +1. The convex hull of a set of points is just the set of points that may be obtained by taking convex combinations (linear combinations whose weights are non-negative and add up to 1) of the elements of the set. For instance, in two dimensions, a simplex is simply a triangle, whereas in three dimensions it is a tetrahedron.

The rationale behind the method is illustrated in Fig. 10.13 for a minimization problem in  2. In this case, the simplex search generates a sequence of sets consisting of three points, in which the worst one is discarded and replaced by a new one. For instance, let us assume that x3 is associated with the worst objective value; then, it seems reasonable to move away from x3 by reflecting it through the center of the face formed by the other two points, as shown in the figure. A new simplex is obtained and the process is repeated. The generation of the new point is easily accomplished algebraically. If x _n_ +1 is the worst point, We compute the centroid of the other _n_ points as

**FIGURE 10.13** **Reflection of the worst value point in the Nelder–Mead simplex search procedure.**

and we try a new point of the form

Clearly, the key issue is finding the right reflection coefficient α > 0. If x _r_ turns out to be even worse than x _n_ +1, we may argue that the step was too long, and the simplex should be contracted. If x _r_ turns out to be the new best point, we have found a good direction and the simplex should be expanded. Different strategies have been devised in order to improve the convergence of the method. Although the simplex search procedure has its merit, it does not overcome the possible difficulties due to the nonconvexity of the objective function Or the discrete character of some decision parameters. In fact, it was originally developed for experimental, response surface optimization.

The Nelder–Mead algorithm is offered as an option in the basic optim function. Furthermore, there are some R packages implementing variants of this and other derivative-free optimization methods, such as neldermead and dfoptim. The package optimsimplex offers a framework to implement specific variants of simplex search.

## 10.4.2 METAMODELING

We have discussed several derivative-free algorithms. Avoiding reliance on information provided by the gradient and the Hessian matrix of the objective function is certainly a good idea when we are dealing with a badly behaved function. However, in other cases, we may miss an opportunity to better drive the search process. Metamodeling aims to build an approximation of the objective function

on the basis of simulation experiments. Hence, we use a simulation model as a black box, whose response surface is modeled by a suitably chosen approximating function. This kind of higher-level model explains the name of the overall approach.

If we want to find the gradient ∇ _h_ (x _k_ ) at some point x _k_ , we may build a local linear approximator like

Such a model can be easily found by linear regression. Then, we just set

Apparently, the gradient does not depend on x _k_. However, the linear metamodel does depend on that point, as the simulation experiments that we use in linear regression have to be carried out in its neighborhood. They should not be too close, otherwise the noise in the evaluation of _f_ (·, ·) will be overwhelming. However, they should not be placed too far, as the linear approximation is likely to be valid only on a neighborhood of x _k_. Statistical methods for design of experiments can also be put to good use to limit the number of simulation runs needed to feed the linear regression model. Also variance reduction strategies, like common random numbers may help.

The estimated gradient may be used within standard nonlinear programming algorithms to generate a sequence of points like

for a step length α _k_ > 0. This is the steepest descent (or gradient) method, which suffers from poor convergence. One way to improve performance is to build a quadratic metamodel

If the square matrix Γ is positive semidefinite, the above quadratic form is convex and it is easily minimized by solving a system of linear equations, yielding a new approximation x _k_ +1 of the optimal solution. A new quadratic approximation is built, centered on the new point, and the procedure is repeated until some convergence condition is met. This kind of approach is related to sequential quadratic programming methods for traditional nonlinear programming. It may be advisable to use a linear metamodel at the early stages of the process, since this may result in a faster movement toward the minimizer; then, when convergence of the gradient method becomes problematic, the procedure may switch to the quadratic metamodel.

A noteworthy feature of metamodeling is that it can cope with constraints on the decision variables, since we solve a sequence of nonlinear programming models, whereas alternative approaches have to resort to penalty functions. An obvious disadvantage is that it is likely to be quite expensive in computational terms, if many simulation experiments are needed to filter random noise and find a good metamodel. Alternative methods, such as perturbation analysis, have been proposed to estimate sensitivities with a single simulation run. We will consider a few simple application examples to financial derivatives in Chapter 12.

# 10.5 Stochastic programming models

Earlier in the chapter we have introduced decision models under uncertainty. In this section we describe one possible modeling framework, stochastic programming with recourse. For the sake of simplicity, we will just deal with linear programming (LP) models under uncertainty, but the approach can be extended to nonlinear optimization models. We begin with two-stage models in Section 10.5.1. Then, we generalize the approach to multistage problems in Section 10.5.2. Finally, we stress important issues related to scenario generation in Section 10.5.3. An alternative approach to dynamic decision making under uncertainty, stochastic dynamic programming, will be the subject of the last sections of the chapter.

## 10.5.1 TWO-STAGE STOCHASTIC LINEAR PROGRAMMING WITH RECOURSE

To get a clear picture on how uncertainty can be introduced into an optimization model, let us consider an uncertain LP problem:

where all problem data depend on random variables ξ with support  . Stated as such, the model is not even posed in a sensible way, since the objective function is a random variable, i.e., a function of the underlying events, and the minimization is not well defined. When we select a decision, we only select the probability distribution of cost or profit, and in order to assign a proper meaning to the problem, we must specify how probability distributions are ranked. The most natural approach is to take the expected value of the objective, if we ignore risk aversion. However, even if we settle for some sensible optimality criterion, there are serious issues with _feasibility._ We should wonder whether we can require feasibility for any possible contingency, i.e.,

This is clearly not possible in general, as we get an inconsistent set of equations. The case of inequality constraints is a bit easier. Nevertheless, requiring

yields a _fat_ solution that may be overly expensive, assuming that such a feasible solution exists. More often than not, we must accept a controlled violation of constraints. One possibility is to settle for a solution with a probabilistic guarantee:

for a suitably small α. This idea leads to chance-constrained models, which have a sensible interpretation in terms of solution reliability. Chance-constrained models are indeed useful in some cases, but they suffer from definite limitations:

  Per se, they do not model the flow of information and the sequence of decision stages; in real life, we make a decision here and now, but then we revise and adjust it along the way, when uncertainty is progressively resolved.

  They can lead to quite risky solutions, as nothing is said about the effect of a constraint violation, even though this has low probability α.

  Even if we choose a very small α, can we trust our ability to estimate very small probabilities?

  In general, chance-constrained models do not result in convex optimization problems. The essential reason for this unpleasing state of the matter is that the union of convex sets need not be convex.

An alternative modeling framework, to which one may resort to overcome these issues, is stochastic programming with recourse. Let us consider the stochastic constraints

As we have pointed out, it is impossible to satisfy these constraints with an assignment of x that is valid for any realization of the random variable ξ. However, we may think of "adjusting" the solution, _after_ observing uncertainty, by some action represented by decision variables **y** (ξ):

These variables are called recourse variables, and they are random in the sense that they are adapted to the random variable ξ, as they are made after uncertainty is resolved. They are, therefore, second-stage decisions corresponding to contingency plans, whereas the **x** are first-stage, here-and-now, decisions. The matrix **W** is called recourse matrix and represent the "technology" that we use to adjust first-stage decisions. Its structure has a remarkable impact on the difficulty of the problem, and it is worth noting that we are assuming deterministic recourse, as the matrix **W** is not random. Problems with stochastic recourse are generally more difficult and are subject to some pathologies.

###   Example 10.13 Simple recourse in the newsvendor model

In Section 10.2.3 we have formulated the newsvendor model within a scenario-based framework. The first-stage decision _q_ , the amount we produce, should ideally match demand _D_ _s_ in every scenario, which is clearly impossible. In the model of Eq. (10.17) we just penalize deviations from the target:

Thus, we immediately see that in this model the structure of the recourse matrix is

where **I** is the identity matrix of order _S._ This case is referred to as _simple_ recourse, as the second-stage variables are just used for bookkeeping and cost evaluation, and are not really linked to decisions. However, we could apply more clever decisions, such as asking a third-party to produce or sell us additional items, possibly using express shipping, or trying to persuade the customer to accept a substitute item. When we include these recourse actions in the model, and we account for multiple items, we really introduce second-stage decisions.

Clearly, these adjustments are not for free and incur some cost. We should minimize the total expected cost of both our initial and recourse decisions, which yields the following optimization model:

(10.22)

(10.23)

(10.24)

(10.25)

This is a two-stage stochastic LP model with recourse and can be interpreted as follows:

  The first-stage decisions x must satisfy the deterministic constraints (10.23) and are associated with the deterministic first-stage cost **c**  **x**.

  At the second stage, a random event occurs, associated with random data depending on ξ. Given this information, a set of second-stage (recourse) actions **y** (ξ) ≥ **0** are taken.

  The second-stage decisions **y** (ξ) are related to first-stage decisions by constraints (10.24).

  The second-stage decisions result in a cost **q** (ξ)  **y** (ξ).

  In (10.22) we minimize the sum of the first-stage cost and the expected value of second-stage cost.

We may also introduce the recourse function _Q_ ( **x** ) and rewrite the model as the deterministic equivalent

where

and

This formulation shows that a stochastic linear program is, in general, a nonlinear programming problem. Nonlinear problems are usually less pleasing to solve than linear ones, but this is not the really bad news. In fact, the recourse function _Q_ ( **x** ) looks like a "hopeless" function to deal with:

  It is an expectation, with respect to the joint distribution of ξ; hence, it is a multidimensional integral, if random variables are continuous. This integral may require Monte Carlo methods for its approximation.

  As if a multidimensional integral were not bad enough, it involves a function that we do not really know, as it is implicitly defined by an optimization problem.

Luckily, in many cases of practical interest, we can prove interesting properties of the recourse function, most notably convexity, which implies continuity (on an open domain). In some cases, _Q_ ( **x** ) is differentiable; in other cases it is polyhedral. This does not imply that it is easy to evaluate the recourse function, but we may resort to statistical sampling (scenario generation) and take advantage of both convexity and problem structure.

We may represent uncertainty by a discrete probability distribution, where _S_ is the set of scenarios and π _s_ is the probability of scenario _s_   _S._ We obtain the scenario tree (fan) in Fig. 10.14, where ξ _s_ is the realization of random variables corresponding to scenario _s._ One obvious way to sample a scenario tree is by Monte Carlo, in which case the scenarios have the same probability π _s_ = 1/ | _S_ |, but other scenario generation strategies are available. This yields the following LP:

**FIGURE 10.14** **A scenario fan.**

Thus we are left with the task of solving a plain LP, even though a possibly large-scale one. A crucial point is the ability to generate a good and robust solution on the basis of a limited number of representative scenarios.

## 10.5.2 A MULTISTAGE MODEL FOR PORTFOLIO MANAGEMENT

Multistage stochastic programming formulations arise naturally as a generalization of two-stage models. At each stage, we gather new information and we make decisions accordingly, taking into account immediate costs and expected future recourse cost. The resulting decision process may be summarized as follows:

  At the beginning of the first time period (at time _t_ = 0) we select the decision vector **x** 0; this decision has a deterministic immediate cost **c** 0  **x** 0 and must satisfy the constraint

  At the beginning of the second time period we observe random data ( **A** 10, **A** 11, **c** 1, **b** 1); then, on the basis of this information, we make decision **x** 1; this second decision has an immediate cost **c** 1  **x** 1 and must satisfy the constraint

Note that these data are not known at time _t_ = 0, but only at time _t_ = 1; the new decision depends on the realization of these random variables and is also affected by the previous decision.

  We repeat the same scheme as above for time periods up to _H_ − 1, where _H_ is our planning horizon.

  Finally, at the beginning of the last time period _H_ , we observe random data ( **A** _H,H_ −1, **A** _H H_ , **c** _H_ , **b** _H_ ); then, on the basis of this information we make decision **x** _H_ , which has an immediate cost **c** _H_  **x** _H_ and must satisfy the constraint

From the perspective of time period _t_ = 0, the decisions **x** 1,..., **x** _H_ are random variables, as they will be adapted to the realization of the underlying stochastic process. However, the only information that we may use in making each decision consists on the observed history so far. The resulting structure of the dynamic decision process can be appreciated by the following recursive formulation of the multistage problem:

(10.26)

In this formulation, we observe that a decision **x** _t_ depends directly only on the previous decision **x** _t_ −1 In general, decisions may depend on all of the past history, leading to a slightly more complicated model. However, we may often introduce additional state variables, in such a way that the above formulation applies. What the above formulation hides is the dependence structure of the random variables ξ _t_ , _t_ = 1,..., _H_ , which are the underlying risk factors of the random matrices and vectors **A, b** , and **c** :

  In the easiest case, they are mutually independent.

  In the Markovian case, ξ _t_ depends only on ξ _t_ −1.

  In the most general case, a possibly complicated path dependency should be accounted.

From the viewpoint of stochastic programming, all of the above cases may be tackled. On the contrary, the stochastic dynamic approach that we consider later relies on a Markovian structure and is more limited. We shall also see that dynamic programming takes advantage of a recursive formulation, which is implicit in the above multistage model. On the other hand, a definite advantage of the dynamic programming framework is that decisions are given in feedback form, as a function of the current system state. Indeed, it should be noted that, in practice, the relevant output of a stochastic programming model is the set of immediate decisions **x** 0. The remaining decision variables could be regarded as contingency plans, which should be implemented as time goes by; however, it is more likely that the model will be solved again and again according to a rolling-horizon logic.

While this formulation points out the dynamic optimization nature of multistage problems, we usually resort to deterministic equivalents based on discretized scenario trees, like the one depicted in Fig. 10.15. Each node _n_ _k_ in the tree corresponds to an event, where we should make a decision. We have an initial node _n_ 0 corresponding to time _t_ = 0. Then, for each event node, we branch descendant nodes. In the figure, each node has two successors, but the branching factors are arbitrary. The terminal nodes (the leaves) have no successor and their number gives the number of scenarios in the tree. A scenario is a path from the root node to a leaf node. For instance, scenario 2 consists of the node sequence ( _n_ 0, _n_ 1, _n_ 3, _n_ 8). Each branch is associated with a conditional probability of occurrence P( _n_ _k_ | _n_ _i_ ), where _n_ _i_ = _a_ ( _n_ _k_ ) is the immediate predecessor of node _n_ _k_. If scenarios are sampled by plain Monte Carlo, this probability is related to the branching factor. For instance, we may assume that all conditional probabilities in Fig. 10.15 are 0.5. The probability of a scenario is the product of the conditional probabilities of the branches we meet on the path. Hence, in this example we have 8 scenarios with probability 1/8.

**FIGURE 10.15** **Scenario tree for a simple asset–liability management problem.**

Decision variables are associated with each node of the tree, as we show in the next section. It is important to realize that the tree structure enforces nonanticipativity of decisions. For instance, at node _n_ 1 we do not know if we are on scenario 1, 2, 3, or 4. The decisions **x** ( _n_ 1) at node _n_ 1 will be the same for each of those 4 scenarios, which cannot be distinguished at node _n_ 1. Alternatively, we could associate decisions **x** ( _s, t_ ) with scenarios and time steps. Therefore, rather than second-stage decisions **x** ( _n_ 1) and **x** ( _n_ 1), we would include decisions **x** ( _s_ , 1), for _s_ = 1, 2,..., 8. However, we should include nonanticipativity constraints explicitely:

In the next section we illustrate a simple financial model where decisions are associated with nodes, rather than scenarios, resulting in a more compact model formulation. The choice between the two approaches, however, depends on the solution algorithm that we choose for solving a possibly large-scale optimization model. In fact, we realize that if the branching structure is not kept under control, the number of nodes will grow exponentially.

### 10.5.2.1 A multistage financial planning model

To illustrate how multistage stochastic programming can be used to tackle financial problems, let us consider a simple asset–liability management problem.

  We have a stream of stochastic liabilities that should be funded by a portfolio of assets. We represent uncertainty by a scenario tree, and _L_ _n_ is the liability we have to meet in node  , where   is the set of nodes in the tree.

  We denote the root node of the tree by _n_ 0; each node   has a unique predecessor denoted by _a_ ( _n_ ).

  We may invest in assets indexed by _i_    . Of course, asset prices are random, and let _P_ _i_ _n_ is the price for asset _i_ at node _n._

  We are given a set of initial holdings   for each asset _i_   at the root node.

  The initial portfolio can be rebalanced at node _n_ 0, as well as at other nodes of the tree.

  For the sake of simplicity, we do not consider the possibility of receiving new cash along the way, as it would be the case, e.g., for a pension fund receiving new contributions. The only way to raise cash is by selling assets.

  When we buy or sell assets, we incur proportional (linear) transaction costs; the transaction cost is a percentage _c_ of the traded value, for both buying and selling any asset. It is easy to introduce proportional transaction costs depending on the liquidity of each single asset, as well as on the sign of the trade.

  We want to maximize the expected utility of the terminal wealth at the leaves of the tree. Let _S_ be the set of terminal nodes in the tree, where we assume to liquidate the portfolio and pay the last liability. Actually, this need not be really the case, as the model is solved in a rolling-horizon framework.

As we have already pointed out, a scenario in the tree is a path from the root node _n_ 0 to a terminal node _s_   _S_ ; let π _s_ be the corresponding probability. We introduce the following decision variables:

  _z_ _i_ _n_ , the amount of asset _i_ purchased at node _n_

  _y_ _i_ _n_ , the amount of asset _i_ sold at node _n_

  _x_ _i_ _n_ , the amount of asset _i_ held at node _n_ , after rebalancing

  _W_ _s_ , the wealth at terminal node _s_   _S_

Variables _z_ _i_ _n_ , _y_ _i_ _n_ , and _x_ _i_ _n_ are only defined at nodes  , as we assume that no rebalancing occurs at terminal nodes, where the portfolio is liquidated, we pay the liability _L_ _s_ , and we measure the terminal wealth _W_ _s_. Let _u_ ( _w_ ) is the utility for wealth _w_ ; this function is used to express utility of terminal wealth.

On the basis of this notation, we may write the following model:

(10.27)

(10.28)

(10.29)

(10.30)

(10.31)

(10.32)

The objective (10.27) is the expected utility of terminal wealth. Equation (10.28) expresses the initial asset balance, taking the current holdings into account; the asset balance at intermediate trading dates, i.e., at all nodes with exception of root and leaves, is taken into account by Eq. (10.29). Equation (10.30) ensures that enough cash is generated by selling assets in order to meet the liabilities; we may also reinvest the proceeds of what we sell in new asset holdings. Note how proportional transaction costs are expressed for selling and purchasing. Equation (10.31) is used to evaluate terminal wealth at leaf nodes, after portfolio liquidation and payment of the last liability. In practice, we would repeatedly solve the model on a rolling-horizon basis, so the exact expression of the objective function is a bit debatable. The role of terminal utility is just to ensure that we are left in a good position at the end of the planning horizon, in order to avoid nasty end-of-horizon effects.

## 10.5.3 SCENARIO GENERATION AND STABILITY IN STOCHASTIC PROGRAMMING

The key ingredient in any stochastic programming model is a scenario tree, as the quality of the solution depends critically on how well uncertainty is captured. In principle, given a model of the underlying stochastic process, straightforward Monte Carlo simulation could be used to generate a scenario tree, on the basis of the sampling and path generation methods of Chapters 5 and 6. Unfortunately, a suitably rich tree may be needed to capture uncertainty and, given the exponential growth of nodes, this is not practical in the multistage case. An additional difficulty in financial problems is that the tree should be arbitrage-free, which places additional requirements on the branching structure.

One possible way out is a radical change in the solution paradigm. Rather than generate the tree and then solve a large-scale optimization problem, we may interleave optimization and sampling and stop when a convergence criterion is met; this kind of approach relies on sophisticated decomposition strategies that are beyond the scope of this book, and it requires the development of specific software. If we want to stick to commercial solvers, we may resort to the following ideas for clever scenario generation:

  Gaussian quadrature

  Variance reduction methods or low-discrepancy sequences

  Optimized scenario generation

In the next sections we outline these ideas. Whatever scenario generation strategy we select, it is important to check its effect on solution stability, as we discuss in Section 10.5.3.4. We will use the simple newsvendor model to illustrate these concepts in Section 10.5.3.5.

We also remark a couple of possibly useful guidelines:

**1.** If the model aims to yield robust first-stage solutions, it may be advisable to use a rich branching factor at the first-stage, and a limited branching factor at later stages.

**2.** The number of stages can also be limited by using non-uniform time steps. For instance, the first two time steps could correspond to one month, and the later ones to one year.

**3.** A clever way to limit the number of stages is to include in the objective function a term measuring the quality of the terminal state. By doing so, we may avoid myopic decisions and reduce the size of the tree.

**4.** Sometimes, we may reduce the effective dimensionality of the problem by applying data reduction techniques such as principal component analysis; see Section 3.8.1. However, we must be able to sample the few principal components accurately and generate corresponding scenarios for all the random variables involved in the model.

### 10.5.3.1 Gaussian quadrature

We have introduced Gaussian quadrature in Section 2.2 as a tool for numerical integration. When the integral represents the expected value of a function _g_ ( **x** , ξ) depending on continuous random variables ξ with joint density _f_ ξ(·), a quadrature formula represents a discretization of the underlying continuous distribution:

(10.33)

where the weights π _s_ correspond to scenario probabilities, and the integration nodes ξ _s_ are the realizations of the random variables in the discrete approximation. Note that, unlike Monte Carlo sampling, different probabilities are associated with different scenarios. Since Gaussian quadrature formulas ensure exact integration of polynomials up to a given order, this ensures moment matching. As we have remarked, this approach can be applied up to a limited dimensionality, however. Nevertheless, it may work very well for two-stage models.

### 10.5.3.2 Variance reduction and low-discrepancy sequences

Low-discrepancy sequences are another tool for numerical integration that can be used to generate scenarios; see [29]. There is also some room for variance reduction strategies:

  Common random numbers are used in the stochastic decomposition approach [19].

  For an application of importance sampling see, e.g., [22].

  Several variance reduction strategies are used in [18] to improve the accuracy of objective function evaluation in stochastic programming.

  Antithetic sampling and randomized quasi-Monte Carlo are used in [28] to hedge contingent claims.

See also [40, Chapter 4] to appreciate the role of the aforementioned techniques for statistical inference in stochastic programming.

### 10.5.3.3 Optimized scenario generation

Arguably, the most sophisticated approaches to scenario generation are those based on the solution of an optimization problem. We have already appreciated the role of moment matching in Section 2.5. The idea can be applied to scenario generation as well, since we aim at finding a good discretization of a continuous distribution.

To illustrate the approach, consider a variable distributed as a multivariate normal, **X** ~ **N** (μ, ∑). The expected values and covariances of the discrete approximation should match those of the continuous distribution. Furthermore, since we are dealing with a normal distribution, we know that skewness and kurtosis for each component should be 0 and 3, respectively.

Let us denote by _x_ _i_ _s_ the realization of _X_ _i_ , _i_ = 1,..., _n_ , in scenario _s_ , _s_ = 1,..., _S_. Natural requirements are:

Approximate moment matching is obtained by minimizing the following squared error:

(10.34)

The objective function includes four weights _w_ _k_ which may be used to fine tune performance. It should be mentioned that the resulting scenario optimization problem need not be convex. However, if we manage to find any solution with a low value of the "error" objective function, this is arguably a satisfactory solution, even though it is not necessarily the globally optimal one. The idea can be generalized to "property matching," since one can match other features that are not necessarily related to moments; see, e.g., [20].

Moment (or property) matching has been criticized, since it is possible to build counterexamples, i.e., pairs of quite different distributions that share the first few moments. An alternative idea is to rely on metrics fully capturing the distance between probability distributions. Thus, given a scenario tree topology, we should find the assignment of values and probabilities minimizing some distance with respect to the "true" distribution. Alternatively, given a large scenario tree, we should try to reduce it to a more manageable size by optimal scenario reduction. From a formal point of view, there are different ways to define a distance between two probability measures   and  . One possibility has its roots in the old Monge transportation problem, which consists of finding the optimal way of transporting mass (e.g., soil, when we are building a road). The problem has a natural probabilistic interpretation, which was pointed out by Kantorovich, when we interpret mass in a probabilistic sense (see [38], for more details). Thus, we may define a transportation functional:

Here _c_ (·, ·) is a suitably chosen cost function; the problem calls for finding the minimum of the integral over all joint measures η, defined on the Cartesian product  , where   is the support of random variable ξ, whose marginals coincide with   and  , respectively (π1 and π2 represent projection operators).

As the reader can imagine, we are threading on difficult ground, requiring some knowledge of functional analysis. Nevertheless, the idea is not too difficult to grasp if we deal with discrete distributions, where mass is concentrated on points (atoms). The Kantorovich distance, in the discrete case, boils down to

where

Therefore, the evaluation of distance requires the solution of a transportation problem, which is a well-known instance of a linear programming problem. Given this distance, it is possible to build heuristics in which scenarios are successively eliminated, redistributing probability mass in an optimal way, after having prescribed the scenario cardinality or the quality of the approximation. See, e.g., [17] for more details.

We should mention that a counterargument to proponents of the optimal approximation approach is that we are not really interested in the distance between the ideal distribution and the scenario tree. What really matters is the quality of the solution. For instance, if we are considering a mean–variance portfolio optimization problem along the lines of Section 10.2.1, it can be argued that matching the first two moments is all we need. See, e.g., [26, Chapter 4] for a discussion of these issues.

### 10.5.3.4 In- and out-of-sample stability

Whatever scenario reduction strategy we employ, it is important to check the stability of the resulting solution. There are quite sophisticated studies on the stability of stochastic optimization, relying on formal distance concepts. Here we just consider a down-to-earth approach along the lines of [26]. The exact problem is

which involves the expectation of the objective under the true measure  . The actual problem we solve is based on an approximate scenario tree τ, possibly resulting from random sampling:

where with the notation   we point out that we are just estimating the true expected value of the objective, given the generated scenario tree. Each scenario tree induces an optimal solution. Thus, if we sample trees  _i_ and  _j_ , we obtain solutions _x_ _i_ * and _x_ _j_ *, respectively. If the tree generation mechanism is reliable, we should see some stability in what we get. We may consider stability in the solution itself, but it is easier, and perhaps more relevant, to check stability in the value of the objective function. After all, if we want to minimize cost or maximize profit, this is what matters most.

There are two concepts of stability. _In-sample stability_ means that if we sample two scenario trees, the values of the solutions that we obtain are not too different:

(10.35)

This definition does not apply directly if the scenario tree is generated deterministically; in that case, we might compare trees with slightly different branching structures to see if the tree structure that we are using is rich enough to ensure solution robustness. This concept of stability is called in-sample, as we evaluate a solution using the same tree we have used to find it. But since the tree is only a limited representation of uncertainty, we should wonder what happens when we apply the solution in the real world, where a different scenario may unfold. This leads to _out-of-sample stability_ , where we compare the objective value from the optimization model against the actual expected performance of the solution. If the trees are reliable, we should not notice a significant difference in performance:

(10.36)

This kind of check is not too hard in two stage models, as we should just plug the first-stage solution into several second-stage problems, sampling a large number of solutions. Typically, solving a large number of second-stage problems is not too demanding, especially if they are just linear programs; as a rule, it takes more time to solve one large stochastic LP than a large number of deterministic LPs. Unfortunately, this is not so easy in a multistage setting. To see why, consider the second-stage decisions. They are associated with a realization of random variables, which will be different from the out-of-sample realization, and there is no obvious way to adapt the solution. Some ideas can be devised of course, but a realistic evaluation should rely on a rolling-horizon simulation whereby we solve a sequence of multistage stochastic programming problems and apply only the first-stage solution. This is technically feasible but quite expensive computationally. A possible (cheaper) alternative is to check solutions on different trees to see if

In the next section we apply these ideas on a very simple problem, for the purpose of illustration.

### 10.5.3.5 Scenario generation and the newsvendor problem

We have introduced the newsvendor model in Example 10.9, as a basic optimization model under uncertainty. Given its structure, we are able to find the optimal solution in explicit form. If we assume that demand is normally distributed, _D_ ~ N(μ, σ2), then we know that the optimal decision is to make/buy a quantity

of items, where _z_ β is the quantile of the standard normal for the probability level

defined by the profit margin _m_ (underage cost) and by the cost of unsold items _c_ _u_ (overage cost). We have also seen that maximization of expected profit is equivalent to minimization of an expected deviation cost

In Section 10.2.3 we have formulated the problem as a scenario-based LP, and it is interesting to check the accuracy of scenario generation mechanisms on this prototypical example, as well as investigating stability. To this end, we should also express the objective function analytically, which is not too difficult to accomplish for normally distributed demand,

Let us denote the normal PDF of demand by _f_ _D_ ( _x_ ) and express the expected deviation cost in the case of normal demand:

Since we have assumed a normal distribution, the integration domain starts from −∞, which looks unreasonable for a non-negative demand. The assumption is plausible only if the probability of a negative demand is practically negligible. The last integral represents expected lost sales as a function of _q_ and is related to the _loss function._ We digress a little to illustrate the essential properties of the loss function in the normal case.

**Technical note: The loss function.** We define the standard loss function for a standard normal variable as

where φ( _x_ ) is the standard normal density. For a standard normal, the loss function can be expressed in terms of the PDF φ( _x_ ) and the corresponding CDF

The first integral can be evaluated by substitution of variable:

Changing the integration limits accordingly we find

Putting all of it together, we end up with

(10.37)

Given the standard loss function, we may express lost sales in the case of a generic normal random variable:

where we have used the substitution of variable _t_ = ( _x_ − μ)/σ.

Using the loss function of a generic normal, we may express the deviation cost for a normal demand explicitely:

The R code of Fig. 10.16 takes advantage of this knowledge and yields the optimal decision and the related cost, given purchase cost, selling price, markdown price, as well as expected value and standard deviation of demand:

**FIGURE 10.16** **Solving the newsvendor problem optimally in the normal case.**

    > pur <- 10
    > sell <- 12
    > mdown <- 5
    > mu <- 100
    > sigma <- 30
    > newsOptimal(pur,sell,mdown,mu,sigma)
    optimal q 83.02154
    optimal cost 71.38016

In a practical setting, we would probably be forced to round the solution when demand is restricted to integer values, in which case the normal distribution would be an approximation. By the way, since μ > 3σ we may neglect the embarrassing possibility of negative demand.

Given this analytical benchmark, we may use the scenario-based LP model of Section 10.2.3 to check the accuracy of the solution and the estimated expected cost. If we use straightforward Monte Carlo for scenario generation, then the scenarios are equiprobable: π _s_ = 1/ _S_ , _s_ = 1,..., _S_. We may use the function solveLP of package linprog to solve the model, but this requires building the technology matrix explicitly, which is in this case quite simple as the recourse matrix is just **W** = [ **I** , **I** ]. The resulting code is shown in Fig. 10.17. Demand scenarios and probabilities are an input to the procedure, so that we may check the performance of Monte Carlo scenario generation against other strategies. The figure also includes code to check in- and out-of-sample stability. Note that in scenario generation we clip demand to non-negative values, as we may not rule out generation of a negative demand, especially with a large sample. In order check in-sample stability, let us repeat the generation of 50 random scenarios five times:

**FIGURE 10.17** **Solving the newsvendor problem by stochastic LP and checking in- and out-of-sample stability.**

    obj 66.801 q 82.7
    obj 58.2748 q 79.6
    obj 77.0129 q 82.3
    obj 68.2175 q 80.9
    obj 59.6849 q 85.3

We observe some variability, especially in the objective function. As this casts some doubts on the quality of the solution, we may repeat the procedure with 1,000 scenarios, with a corresponding increase in computational effort, which yields

    obj 71.4185 q 81.3
    obj 70.1544 q 82.4
    obj 70.801 q 82.5

    obj 71.3892 q 84.7
    obj 72.8487 q 84.3

Now we see that the solution is more stable, and fairly close to the true optimum. As a further check, we may assess out-of-sample stability, by generating a large number of scenarios (one million) and estimating the cost of each solution generated by the stochastic LP model. A surprising fact is that the solution obtained by using 50 scenarios proves to be a rather good one:

    true cost 71.4412
    true cost 71.8959
    true cost 71.4542
    true cost 71.6127
    true cost 71.6429

The same happens with the solutions obtained with 1,000 scenarios:

    true cost 71.5528
    true cost 71.4514
    true cost 71.4477
    true cost 71.5522
    true cost 71.5014

This finding can be explained by observing that in some cases the true objective function is rather flat around the optimal solution, which makes the model rather robust. However, this need not be the case in other settings. For instance, in financial applications the model might take full advantage of the faulty information included in badly generated scenarios, yielding a very unstable solution. Furthermore, here we may afford a large number of scenarios because this is a very simple model, but a parsimonious approach must be pursued in general. Thus, it is also interesting to see what happens if we generate scenarios using Gaussian quadrature:

    > library(statmod)
    > quad <- gauss.quad.prob(10,dist="normal",mu=mu,sigma=sigma)
    > out <- newsLP(m,cu,pmax(quad$nodes,0),quad$weights)
    > cat('obj ',round(out$cost,4),' q ',round(out$q,1),'\n')
    obj 65.5008 q 85.5
    > trueCost <- outScost(out$q,m,cu,outsDemand)
    > cat('true cost ',round(trueCost,4),'\n')
    true cost 71.6757

With only ten scenarios, we get a solution that proves to be good out-of-sample, even though the in-sample evaluation of the objective function does not look too satisfactory. A less pleasing feature of Gaussian quadrature for this example is that, in order to capture the tails of the distribution, it generates negative as well as rather extreme demand values:

    > quad$nodes
    -45.783885 −7.454705 25.470225 56.020327 85.451929
    114.548071 143.979673 174.529775 207.454705 245.783885
    > quad$weights
    4.310653e-06 7.580709e-04 1. 911158e-02 1.354837e-01
    3.446423e-01 3.446423e-01 1.354837e-01 1.911158e-02
    7.580709e-04 4.310653e-06

Even though negative values have small probability, this shows a potential issue, which is actually related to the fact that we are using a normal distribution to model an intrinsically non-negative variable. A similar issue could arise when modeling rates of return of stock shares, which cannot be less than −100%.

In Gaussian quadrature we determine both nodes and weights together. We may also pursue a simple approach in which probabilities are fixed to a uniform value, and we look for nodes. One such approach can be devised by integrating random variate generation by the inverse transform method with a sort of "deterministic stratification." The generation of a standard normal random variable by the inverse transform method, as we have seen in Chapter 5, requires the generation of a uniform random variable _U_ ~ U(0, 1) and the inversion of the CDF of the standard normal, _Z_ = Φ−1( _U_ ). We may partition the unit interval uniformly, define a representative point of each subinterval, and generate the corresponding standard normal. For instance, if we use five intervals, their end points are

Each subinterval has probability 0.2, and we might consider their midpoints as their representative points. If so, the nodes would be

We note that, in the stated form, this is at best a simple heuristic; in order to apply stratification we should find the conditional expectation of the standard normal on each interval. This may actually be done, but it is left as an exercise for the reader, and we just check the performance of this simple approach:

    > numVals <- 50
    > aux <- (0:numVals)/numVals
    > U <- (aux[−1]+aux[−(numVals+1)])/2
    > demand <- pmax(0, mu+sigma*qnorm(U))
    > probs <- rep(1/numVals,numVals)
    > out <- newsLP(m,cu,demand,probs)
    > cat('obj',round(out$cost,4),' q ',round(out$q,1),'\n')
    obj 70.8987 q 83.4
    > trueCost <- outScost(out$q, m,cu,outsDemand)
    > cat('true cost ',round(trueCost,4),'\n')
    true cost 71.4444

Needless to say, the performance is arguably satisfactory because of the simple and well-behaved problem we are tackling. Still, the idea might be worth considering, and it can be applied to small-dimensional cases (where we build hyper-rectangular cells, rather than subintervals), possibly with some preliminary Monte Carlo runs to find the representative point of each cell. It is interesting to note that in this case we do not generate "extreme" node values:

    > range(demand)
    [1] 30.20956 169.79044

Whether this is a strength or a weakness depends on the application. As a final example, let us resort to low-discrepancy sequences and try scrambled Sobol sequences:

    > library(randtoolbox)
    > numVals <- 50
    > eps <- sobol(numVals,dim=1,normal=TRUE,scrambling=2)
    > demand <- pmax(0, mu+sigma*eps)
    > probs <- rep(1/numVals, numVals)
    > out <- newsLP(m,cu,demand,probs)
    > cat('obj',round(out$cost,4),' q', round(out$q,1),'\n')
    obj 73.4044 q 82
    > trueCost <- outScost(out$q,m,cu,outsDemand)
    > cat('true cost ',round(trueCost,4),'\n')
    true cost 71.4798

Once again, the result does not look bad, but the reader is warned against drawing any conclusion on the basis of this extremely limited experiment. Anyway, we observe that both Monte Carlo and quasi-Monte Carlo methods can be applied to relatively high-dimensional problems, whereas both Gaussian quadrature and stratification are more limited. Unfortunately, the limitation of R as an optimization tool does not allow to cope with more interesting examples. However, we should mention that state-of-the-art commercial solvers, such as IBM CPLEX and Gurobi do offer an interface with R.

# 10.6 Stochastic dynamic programming

Dynamic programming (DP) is, arguably, the single most powerful optimization principle that we have at our disposal. In fact, it can be used to tackle a wide variety of problems:

  Continuous as well discrete problems

  Discrete- as well as continuous-time problems

  Deterministic as well as stochastic problems

  Finite- as well as infinite-horizon problems

Actually, for some problem families, dynamic programming is the only viable approach. Since there is no free lunch, power and generality must come at some price. On the one hand, dynamic programming is a principle, rather than a well defined optimization algorithm. Thus, unlike the simplex method for linear programming, there is no on-the-shelves software package, and a considerable degree of customization and implementation effort may be needed to make it work. On the other hand, dynamic programming is associated with the so-called curse of dimensionality. As we shall see, the curse actually takes different forms, but the essence is that, as a rule, conventional dynamic programming can only be applied to rather small-scale problems. These limitations can be overcome, at least in part, by resorting to numerical approximations where Monte Carlo methods play a fundamental role. Here we are interested in discrete-time decision models under uncertainty; hence, there is a considerable connection with stochastic programming with recourse.

To introduce the dynamic programming concept in the simplest way, we show how it can be used to solve a deterministic shortest path problem in Section 10.6.1. Then, in Section 10.6.2, we formalize the principle in the form of the recursive functional equation originally introduced by Bellman; this form allows us to cope with stochastic optimization problems. We outline the adaption of the approach to infinite-horizon problems in Section 10.6.3. Finally, in Section 10.6.4, we briefly compare stochastic programming with recourse and stochastic dynamic programming; as expected, the two approaches have somewhat complementary strengths and weaknesses, as well as many intersections.

## 10.6.1 THE SHORTEST PATH PROBLEM

The easiest way to introduce dynamic programming is by considering one of its most natural applications, i.e., finding the shortest path in a network. Graph and network optimization are not customary topics in finance or economics, but a quick look at Fig. 10.18 is enough to understand what we are talking about. A network consists of a set of nodes (numbered from 0 to 7 in our toy example) and a set of arcs joining pairs of nodes. Arcs are labeled by a number which can be interpreted as the arc length (or cost). Our purpose is finding a path in the network, starting from node 0 and leading to node 7, such that the path has total minimal length. For instance, by summing the arc lengths that are followed on the path (0, 1, 4, 7), we see that its total length is 18, whereas path (0, 1, 3, 5, 7) has length 16. At each node, we must choose the next node to visit. We may immediately appreciate that this problem bears some resemblance to dynamic decision making; given some state we are in, we should decide what to do in order to optimize an outcome that depends on the whole path. A greedy decision need not be the optimal one; for instance, the closest node to the starting point 0 in our network is node 2, but there is no guarantee that this arc is on an optimal path.

**FIGURE 10.18** **A shortest path problem.**

Of course, we could simply enumerate all of the possible paths to spot the optimal one; here we have just a finite set of alternatives and there is no uncertainty involved, so the idea is conceptually feasible. However, this approach becomes quickly infeasible in practice, as the network size increases. Furthermore, we need a better idea in view of an extension to a stochastic case: If arc costs are random, running a Monte Carlo simulation for each possible path to estimate its expected length is out of the question. So we must come up with some clever way to avoid exhaustive enumeration. Dynamic programming is one possible approach to accomplish this aim. It is worth noting that more efficient algorithms are available for the shortest path problem, but the idea we illustrate here can be extended to problems featuring infinite state spaces (count-able or not) and uncertain data. Let   = {0, 1, 2,..., _N_ } be the node set and   be the arc set; let the start and final nodes be 0 and _N_ , respectively. For simplicity, we assume that the network is acyclic and that the arc lengths _c_ _ij_ , _i_ , _j_    , are non-negative: If we had the possibility of getting trapped in a loop of negative length arcs, the optimal cost would be −∞, and we do not want to consider such pathological cases.

The starting point is to find a _characterization_ of the optimal solution, which can be translated into a constructive algorithm. Let _V_ _i_ be the length of the shortest path from node   to node _N_ (denoted by _i_   _N_ ). Assume that, for a specific _i_    , node _j_ lies on the optimal path _i_   _N_. Then the following property holds:

In other words, the optimal solution for a problem is obtained by assembling optimal solutions for smaller subproblems, which suggests a useful decomposition strategy. To understand why, consider the decomposition of _i_   _N_ into the subpaths _i_ → _j_ and _j_ → _N_. The length of _i_   _N_ is the sum of the lengths of the two subpaths:

(10.38)

Note that the second subpath is not affected by _how_ we go from _i_ to _j_. This is strongly related to the concept of state in Markovian dynamic systems: How we get to state _j_ has no influence on the future. Now, assume that the subpath _j_ → _N_ is not the optimal path from _j_ to _N_. Then we could improve the second term of Eq. (10.38) by assembling a path consisting of _i_ → _j_ followed by _j_   _N_. The length of this new path would be

which is a contradiction, as we have assumed that _V_ _i_ is the optimal path length.

This observation leads to the following recursive equation for the shortest path from a generic node _i_ to the terminal node _N_ :

(10.39)

In other words, to find the optimal path from node _i_ to node _N_ , we should consider the immediate cost _c_ _ij_ of going from _i_ to all of its immediate successors _j_ , plus the optimal cost of going from _j_ to the terminal node. Note that we do not only consider the immediate cost, as in a greedy decision rule. We also add the future cost of the optimal sequence of decisions starting from each state _j_ that we can visit next; this is what makes the approach nonmyopic. The function _V_ _i_ is called _cost-to-go_ or _value function_ and is denned recursively by Eq. (10.39). The value function, for each point in the state space, tells us what the future optimal cost would be, if we reach that state and proceed with an optimal policy. This kind of recursive equation, whose exact form depends on the problem at hand, is the heart of dynamic programming and is an example of a _functional equation._ In the shortest path problem, we have a finite set of states, and the value function is a vector; in a continuous-state model, the value function is an infinite-dimensional object.

Solving the problem requires finding the value function _V_ 0 for the initial node, and to do that we should go backward from the terminal node. We can associate a terminal condition _V_ _N_ = 0 with our functional equation, as the cost for going from _N_ to _N_ is zero. Then we unfold the recursion by considering the immediate predecessors _i_ of the terminal node _N_ ; for each of them, finding the optimal path length is trivial, as this is just _c_ _iN_. Then we proceed backward, labeling each node with the corresponding value function. In this unstructured network, we may label a node only when all of its successors have been labeled; we can always find a correct node ordering in acyclic networks.

###   Example 10.14 A shortest path

Let us find the shortest path for the network depicted in Fig. 10.18. We have the terminal condition _V_ 7 = 0 for the terminal node, and we look for its immediate predecessors 4 and 6 (we cannot label node 5 yet, because node 6 is one of its successors). We have

Now we may label node 5:

Then we consider node 3 and its immediate successors 4, 5, and 6:

By the same token we have

Apart from getting the optimal length, which is 16, we may find the optimal path by looking for the nodes optimizing each single decision, starting from node 0:

## 10.6.2 THE FUNCTIONAL EQUATION OF DYNAMIC PROGRAMMING

The shortest path problem is one of the prototypical examples of dynamic programming, yet it lacks the structure of most problems of interest in finance and economics, which are dynamic problems where uncertainty unfolds over time. We have introduced the three essential families of dynamic models in Chapter 3, Dynamic programming can be applied to continuous-time and discrete-event models, too, but here we only consider discrete-time models, which are relatively simple but do hide some potential traps. In fact, when discretizing time it is necessary to be very clear about the meaning of time instants and time intervals:

  We consider _time instants_ indexed by _t_ = 0, 1, 2,.... At these time instants we observe the system state and make a decision.

  By a _time interval t_ we mean the time interval between time instants _t_ − 1 and _t_. After the decision at time instant _t_ − 1, the system evolves and a new state is reached at time _t_. During the time interval, some random "disturbance" will be realized, influencing the transition to the new state.

These definitions are illustrated in Fig. 10.19. Note that, with this timing convention, we emphasize the fact that noise is realized _during_ time interval _t, after_ making the decision at time instant _t_ − 1.

**FIGURE 10.19** **Illustrating time conventions.**

Let us introduce a bit of notation:

  The vector of state variables at time instant _t_ is denoted by **s** _t_ ; with the time convention above **s** _t_ is the value of the state variables at the end of time period _t_ , as a result of what happened between time instants _t_ − 1 and _t_ , i.e., during time interval _t_.

  The vector of decision variables is denoted by **x** _t_ ; these decisions are based on the knowledge of **s** _t_. Here we assume that states are perfectly observable, but the framework can be applied to inexact observations as well.

  The state **s** _t_ +1 at time instant _t_ \+ 1 depends on **s** _t_ and **x** _t_ , but also on possible random perturbations that take place during time interval _t_ \+ 1, i.e., between time instants _t_ , when we made the last decision, and _t_ \+ 1, when we have to make another one. We represent these perturbations by  _t_ +1; the subscript _t_ \+ 1 emphasizes the fact this is _new information._

Then, the system dynamics is represented by a state equation like

(10.40)

where _S_ _t_ +1 is the state transition function over time period _t_ \+ 1; the subscript may be omitted if the dynamics do not change over time, i.e., if the system is stationary. The latter is the common assumption when dealing with infinite-horizon problems.

By making the decision **x** _t_ in state **s** _t_ we incur an immediate cost _C_ _t_ ( **s** _t_ , **x** _t_ ). We would like to minimize the expected cost incurred over a given planning horizon,

(10.41)

where γ < 1 is a suitable discount factor. Clearly, when dealing with profits or utility functions, we rewrite the problem in terms of a maximization. However, in the statement above we are not stating precisely what kind of decision process we are dealing with. In a deterministic setting, we would minimize with respect to decision variables **x** _t_ , and we would come up with a plan specifying all of the future decisions. However, we have already seen in multistage stochastic programming with recourse that decisions should be adapted to the occurred contingencies. As a consequence, apart from the first-stage ones, decisions are in fact random variables. In stochastic dynamic programming we make this even clearer, as decisions are obtained by the application of a feedback policy. We define a policy, denoted by π, whereby an action is selected on the basis of the current state:

(10.42)

Hence, a more precise statement of the problem is

(10.43)

where Π is the set of feasible policies. By feasible we mean that each decision **x** _t_ is constrained to be in _X_ _t_ ( **s** _t_ ), the set of feasible decisions at time _t_ if we are in state **s** _t_.

Stated as such, the problem does not look quite tractable. However, the shortest path example suggests a suitable decomposition strategy. When in state **s** _t_ at time _t_ , we should select the decision **x** _t_   _X_ _t_ ( **s** _t_ ) that minimizes the sum of the immediate cost, and the expected value of cost-to-go that we will incur from time period _t_ \+ 1 onward:

(10.44)

More generally, we talk of a value function, as the term is better suited to both minimization and maximization problems. The conditioning with respect to **s** _t_ stresses the fact that we are taking an expectation conditional on the current state; it might even be the case that uncertainty depends on our decisions, and if so we should also condition with respect to **x** _t_. We will not analyze the conditions under which Eq. (10.44) can be used to solve problem (10.43). The intuition gained from the shortest path problem suggests that the stochastic process  _t_ must be Markovian. With more general path dependency, the decomposition argument behind dynamic programming fails. Sometimes, a model reformulation based on additional state variables can be used to transform a non-Markovian model into a Markovian one, at the price of an increase in complexity. The careful reader will immediately notice a similarity between this modeling framework and stochastic programming with recourse, where the recourse cost plays the role of the value function. The recursive structure of a multistage problem, which is pretty evident in Eq. (10.26), is also visible here. However, multistage stochastic programming with recourse does not require the Markov property. On the other hand, we should note that standard stochastic programming models with recourse assume exogenous uncertainty.

Equation (10.44) is the functional equation of dynamic programming, also called the optimality equation, and it requires to find the value function for each time instant. Typically, the functional equation is solved backward in time, taking advantage of the fact that the problem in the last time period is a myopic choice. At the last decision time instant, _t_ = _T_ , we should solve the problem

(10.45)

for every possible state **s** _T_. The value function _V_ _T_ +1 ( **s** _T_ +1) is given by a boundary condition. For problem (10.41), the boundary condition is

since the objective function includes only a trajectory cost. Sometimes, a cost (or reward) Φ _T_ +1( **s** _T_ +1) is associated with the terminal state, and the problem is

(10.46)

For instance, in life-cycle consumption-saving models for pension economics, the objective function could include both a utility function depending on consumption and some utility from bequest. In this case, we impose the boundary condition

Whatever boundary condition we select, problem (10.45) is a _static_ optimization model. By solving it for every state **s** _T_ , we build the value function _V_ _T_ ( **s** _T_ ) at time _t_ = _T_. Then, unfolding the recursion backward in time, we find the value functions _V_ _T_ −1( **s** _T_ −1), _V_ _T_ −2( **s** _T_ −2), and so on, down to _V_ 1( **s** 1). Finally, given the initial state **s** 0, we find the next optimal decision by solving the static problem

(10.47)

Unlike stochastic programming with recourse, we have a clear way to make also the next decisions, as the full set of value functions allows us to solve each problem over time, finding the decision **x** _t_ as a function of the current state **s** _t_. Thus, we find the optimal policy in an implicit feedback form, and the dynamic problem is decomposed into a sequence of static problems.

The skeptical reader will wonder what is the price to pay for this extremely powerful approach. The price is actually pretty evident in Eq. (10.44). We should solve an optimization problem for each possible value of the state variable **s** _t_. If we are dealing with a continuous state space, this is impossible, unless we may take advantage of some specific problems structure (we shall see an example in Section 10.7.1). But even if the state space is discrete, when the dimensionality is large, i.e., when **s** _t_ is a vector with several components, the sheer computational burden is prohibitive. This is called the _curse of dimensionality._ Actually, a close look at Eq. (10.44) shows that we face a three-fold curse of dimensionality:

**1.** The dimension of the state space, which is related to the state variable **s** _t_.

**2.** The need to evaluate, or approximate, the expected value of the value function in the next state, which is related to the disturbance  _t_ +1 and may call for expensive numerical integration.

**3.** The need to solve a possibly large-scale optimization problem with respect to a high-dimensional decision vector **x** _t_.

The shortest path problem is so easy because it is deterministic, it features a discrete state space, and the of possible decisions is discrete as well. In general the application of DP is not trivial, and calls for the integration of several numerical tricks of the trade, as we show in Section 10.7. Even so, DP in its standard form can be tackled only for moderately sized problems. However, recent trends in approximate dynamic programming, which we outline in Section 10.8, integrate statistical learning with optimization, paving the way for a much wider array of applications. Typically, these approaches rely on Monte Carlo simulation.

## 10.6.3 INFINITE-HORIZON STOCHASTIC OPTIMIZATION

The recursive form of Eq. (10.44) needs some adjustment when coping with an infinite-horizon problem like

(10.48)

where we assume that immediate costs are bounded and γ < 1, so that the series converges to a finite value. In this case, we typically drop the subscript _t_ from the immediate cost, as well as from the state-transition equation

i.e., we assume a stationary model. An infinite-horizon model may be of interest per se, or it can be a trick to avoid the need to specify a terminal state value function. In this case, the functional equation boils down to

(10.49)

where _X_ ( **s** ) is the set of feasible decisions when we are in state **s**. The good news is that we need only one value function, rather than a value function for each time instant. The bad news is that now we have a value function defined as the fixed point of a possibly complicated operator, as we have _V_ ( **s** ) on both sides of the equation. In general, an iterative method is needed to solve Eq. (10.49).

## 10.6.4 STOCHASTIC PROGRAMMING WITH RECOURSE VS. DYNAMIC PROGRAMMING

Before we delve into numerical solution methods for stochastic dynamic programming, it is useful to pause and compare this approach with stochastic programming with recourse, since there is a clear connection between the two.

As a general consideration, stochastic programming models are mostly favored within the operations research community, whereas dynamic programming is favored by economists. One reason is that stochastic programming is intrinsically operational: What we care about is the first-stage decision, which is what we actually implement. The next stages are included to find a nonmyopic and robust solution, but they are not really contingency plans to be implemented. When uncertainty is progressively resolved, we typically recalibrate and resolve the model, within a rolling-horizon framework. On the contrary, the use of a model in economics is generally quite different. Consider a life-cycle, strategic asset allocation model. Usually, a very limited number of assets is included in the model, possibly only two, risk-free and risky. Indeed, the aim there is not really to manage a portfolio of a pension fund. The aim is to investigate the behavior of individual workers in order to see what drives their decisions (precautionary savings, habit formation, income path over time, etc.). To simulate consumption-saving and asset allocation decisions over several years, dynamic programming is definitely more convenient, as once we have found the sequence of value functions, we may simulate the sequence of optimal decisions, which calls for the solution of a sequence of static optimization models, without the need of solving a dynamic model repeatedly. The result is a pattern in terms of consumption–saving decisions, as well as an asset allocation between the risky and risk-free assets. The pattern may be checked against empirical evidence in order to assess the validity of the model and the assumptions behind it.

There is also a clear connection between recourse functions in stochastic programming and value functions in dynamic programming. However, a recourse function is instrumental in finding a first-stage decision. Decisions at latter stages are made after observing the new state, which need not be included in the scenario tree; however, there is no obvious way to adjust a recourse decision when an unplanned scenario is realized. The value function is essential in finding decisions in feedback form; hence, they may be used for any scenario. This is a definite advantage of dynamic programming, which comes at a price. First, we have to rely on a Markovian structure, whereas stochastic programming may cope with more general path dependence. Second, stochastic programming may cope with more complicated, less stylized decision models.

Another feature of dynamic programs is that, as a rule, they deal with multi-stage models where the same kind of decision must be made over and over. For instance, in a consumption–saving problem, at each time step we have to decide how much to consume and how much to save. However, there are decision problems in which the decisions ate different stages are completely different. For instance, consider the design of a logistic network. At one stage, we cope with the fixed costs of installing warehouses and other facilities; hence, the decision variables define the network structure, which must be designed under demand uncertainty. At later stages, we have to cope with variable transportation costs; hence, the second-stage decision are related to the optimal routing of items, in order to satisfy a known demand, with constraints imposed by the selected network structure. Clearly, this latter kind of model is less common in finance or economics, but it contributes to explain with stochastic programming is more common in operational decision problems.

Another noteworthy point is that, so far, we have assumed that the important output of the optimization model is a set of decisions, possibly in feedback form. However, there are problems in which it is the value of the objective function that is relevant. One important class of such problems is the pricing of American-style options. As we have seen in Section 10.2.5, pricing options with early exercise features requires the solution of a stochastic optimization problem. Since our ability to represent uncertainty by scenario trees is limited, the validity of a price found by stochastic programs is questionable. Armed with a sequence of value functions, possibly approximated ones, we may simulate the application of an early exercise policy with a large number of scenarios. Due to approximations in the solution of the dynamic programming problems, we will get a lower bound on the option price, but at least we may check its reliability, as we shall see in Chapter 11.

Finally, we observe that DP is well-suited to cope with infinite horizon problems, whereas stochastic programming with recourse is not. However, the two approaches may be integrated, since the value function for an infinite-horizon program may be used to avoid end-of-horizon effects in a finite-horizon stochastic program. Indeed, there are important connections between the approaches, especially when it comes to devising sophisticated decomposition algorithms to cope with large-scale problems. Thus, both approaches should be included in our bag of tools to tackle a wide class of relevant problems.

# 10.7 Numerical dynamic programming

The decomposition principle behind dynamic programming is extremely powerful, but it requires a set of value functions, which are infinite-dimensional objects when the state space is continuous. Solving a sequence of functional equations involves a mix of numerical methods for integration, optimization, function approximation, and equation solving. In this section we illustrate the building blocks of the mix, by applying dynamic programming to three examples. In Section 10.7.1 we deal with a deterministic budget allocation problem with a continuous-state space. The problem structure allows for an analytical solution, which we use as a benchmark for a numerical method based on the discretization of the state space and the approximation of value functions by cubic splines. Then, in Section 10.7.2 we tackle a simple infinite-horizon, stochastic problem. The example is very simple, as it features a discrete state space and a discrete set of decisions, nevertheless it illustrates the concept of value iteration, which is a key ingredient in approximate dynamic programming. Finally, we tackle a simple consumption–saving problem with income uncertainty in Section 10.7.3. Even though quite simplified with respect to real-life problems, this example is challenging, as it involves mixed states, both continuous and discrete, continuous disturbances, as well as continuous decisions.

## 10.7.1 APPROXIMATING THE VALUE FUNCTION: A DETERMINISTIC EXAMPLE

Let us consider a simple resource allocation problem. We have a resource budget _B_ that can be allocated to a set of _n_ activities; let _x_ _i_ ≥ 0 be the amount of resource allocated to activity _i._ If the contribution to profit from activity _i_ is represented by _C_ _i_ ( _x_ _i_ ), our problem is

A typical choice for the contribution function is a concave function like _C_ ( _x_ ) =  , to model decreasing marginal returns. Note that we assume that a continuous allocation of resources is feasible. If _C_ (·) is increasing, then we may replace the inequality by an equality constraint, as the budget will be fully used for sure. Clearly, there is no need for dynamic programming to solve the above problem, which may be easily tackled as an ordinary nonlinear program. By the way, if the contribution functions are the same for each activity and is concave, it is easy to see the in the optimal solution the budget is equally allocated to each activity. Nevertheless, using dynamic programming could be valuable in the case of different contribution functions and discrete decisions, i.e., when resources are not infinitely divisible; in such a case, the above problem would be a potentially nasty nonlinear integer programming problem. Furthermore, it is instructive to see how a problem where time does not play any role can be recast as a sequential decision process. Since the structure of the problem is simple, we may be able to use dynamic programming and solve the problem analytically by finding the value functions explicitly. Then, we may compare the analytical solution with a numerical solution based on the discretization of the state space and the approximation of the value functions.

To recast the problem as a dynamic program, let us imagine a sequence of decisions. We start with an initial budget _b_ 1 = _B_. Then we allocate an amount of resource _x_ 1 ≤ _b_ 1 to the first activity, which yields an immediate reward _C_ 1( _x_ 1) and leaves us with a residual budget _b_ 2 = _b_ 1 − _x_ 1. After deciding _x_ 2, we have a residual budget _b_ 3 = _b_ 2 − _x_ 2, and so on. It is easy to see that the residual budget _b_ _k_ , before making decision _x_ _k_ , is the natural state variable, with state transition function _b_ _k_ +1 = _b_ _k_ − _x_ _k_. Furthermore, we define the value function _V_ _k_ ( _b_ _k_ ), the optimal profit that we obtain by optimally allocating the residual budget _b_ _k_ to activities _k_ , _k_ \+ 1,..., _n_. The value function satisfies the optimality equation

(10.50)

It is a fairly simple exercise26 to solve the equation analytically by backward induction. Let us assume that  . At the last stage, we have

since at the last stage the obvious decision is to allocate whatever is left to the last activity. Plugging this value function into Eq. (10.50) for _k_ = _n_ − 1, we find

whose optimal solution is _x_ _n_ −1* = _b_ _n_ −1/2. Plugging the optimal decision into this objective function, we find the value function

If we repeat the argument going backward with respect to index _k_ = _n_ − 2, _n_ − 3,..., 1, and we also apply the principle of mathematical induction, we obtain

and

Note that when we consider subscript _k_ , it means that we are about to make a choice for activity _k_ ; thus, we have already allocated a share of budget to activities from 1 to _k_ − 1, and there are still _n_ − ( _k_ − 1) to go. The last condition, in fact, tells us that the residual budget should be shared in fair slices.

Of course, since the profit contributions are identical and concave, the conclusion is far from surprising, and we have arguably used a pretty huge sledge-hammer to crack a tiny nut. The message is that dynamic programming can be also applied when we have different profit contributions, discrete units of budget, and uncertainty as well. Furthermore, we are now in the position to experiment with a numerical method, in order to get acquainted with and to gain confidence in the solution strategies that are needed in more complicated cases. The big issue here is that each value function is an infinite dimensional object, since we are dealing with a continuous state variable. In the discrete-state case, we may just tabulate the value function; in the continuous case, we must somehow discretize it. Therefore, we may define a grid of budget values, and solve a finite number of problems of the form (10.50) for points on the grid. Each optimization is solved by a numerical method. The numerical method to find _V_ _k_ ( _b_ _k_ ) will often need values _V_ _k_ +1( _b_ _k_ +1) outside the grid. Values for state values outside the grid can be found by an interpolation strategy; a common interpolation strategy relies on cubic splines. A cubic spline is a piecewise polynomial function passing through a specified set of points: Each polynomial piece is of order three, and some essential continuity and differentiability properties are guaranteed. Given a set of points, the R function splinefun returns a function that may be used to interpolate outside the grid. An example if its use is given in the script in Fig. 10.20, which produces the spline depicted in Fig. 10.21.

**FIGURE 10.20** **A script to illustrate cubic splines.**

**FIGURE 10.21** **A cubic spline.**

Figure 10.22 shows a script to solve the budget allocation problem using state space discretization and interpolation by cubic splines. The function receives the following inputs:

**FIGURE 10.22** **Using state space discretization and splines to approximate the value functions of the budget allocation problem.**

  The available budget.

  The list funcList containing the profit function for each activity; we need not use the same function for each activity.

  The number of points numPoints on the grid; the grid spacing is uniform and the same discretization is used for each value function.

Then, the function allocates a matrix of function values valFunMatrix, which will store the values of each value function on the grid. These values are used to specify one cubic spline for each decision stage; the list of cubic splines is contained in the list splinesList. The core of the procedure consists of two nested for loops, one over the decision stages and one over the points on the grid. For each point on the grid, we build the objective function as

    objFun<-function(z) funcList[[t]](z)+splinesList[[t+1]](b-z)

Note that this objective function consists of the sum of the profit function for the current activity, if carried out at level z, and the next value function evaluated for the remaining budget b-z. The optimization is carried out numerically by the function optimize. This function can maximize over the bounded interval [0, _b_ ], but it returns an error if _b_ = 0; the code takes care of this case. After finding the whole sequence of value functions, we have a second part of the procedure, where we step forward and apply the optimal decisions stage by stage. The output consists of a list including the optimal solution and the overall profit, as well as the value functions on the grid. We may try and check the procedure by the following R script:

    > budget <- 20
    > numPoints <- 50
    > f <- function(x) sqrt(x)
    > funcList <- list(f,f,f)
    > out <- Budgeting(budget,funcList,numPoints)
    > out$par
    [1] 6.666658 6.666673 6.666669
    > out$value
    [1] 7.745967

In this trivial case, the optimal solution and the corresponding profit are, trivially:

As we can see, we do find a sensible approximation of the optimal solution. Approximating the value function by splines is only one possibility. We may also use approaches based on collocation methods or the projection onto a space of elementary functions, such as polynomials, by linear regression. In a stochastic problem, function approximation must be coupled with a strategy to approximate the expectation, as we shall see later.

## 10.7.2 VALUE ITERATION FOR INFINITE-HORIZON PROBLEMS

When we deal with a finite-horizon problem, we may build a sequence of value functions iteratively, starting from a boundary condition. However, if we tackle an infinite-horizon problem, we face the genuine functional equation

(10.51)

We notice that this equation defines the value function _V_ ( **s** ) as a fixed point of an operator that involves the solution of an optimization problem. The basic approach to solve the above equation is value iteration. The approach is conceptually very simple to apply to problems with a finite state space and a discrete set of actions. Such problems are referred to as Markov decision processes (MDP), and we consider a toy example in the next section. Then we outline an extension to continuous-state problems and some ideas to speed up value iteration.

### 10.7.2.1 Value iteration for Markov decision processes

If the state space is finite, we may think of the value function as a vector **V**    _n_ , where _n_ is the size of the state space. If the set  _i_ of possible decisions in each state _i_ = 1,..., _n_ is finite as well, we may associate each decision _a_    _i_ with a transition probability matrix _P_ _ij_ _a_ , which gives the probability of moving from state _i_ to state _j_ if we choose action _a_. In this context, decisions are typically referred to as _actions_. The transition probability matrix defines a Markov chain, and it is easy to compute the expectation in the functional equation. Solving the optimization problem is easy as well (unless we are dealing with a huge state space), as it may be tackled by explicit enumeration.

The value iteration algorithm for such a problem can be stated as follows, where _V_ _i_ _n_ , collected into vector **V** _n_ , is the value function approximation for state _i_   _S_ at iteration _n_ :

**Step 0.** Initialization:

  Set _V_ _i_ 0 = 0 for all states _i_   _S_.

  Choose a tolerance parameter  .

  Set the iteration counter _n_ = 1.

**Step 1.** For each state _i_ compute

(10.52)

**Step 2.** If the termination condition

is met, stop and return the policy that solves problem (10.52), as well as the approximation **V** _n_ ; otherwise, increment the iteration counter _n_ = _n_ \+ 1 and repeat step 1.

The value iteration algorithm is best illustrated by an example. Figure 10.23 depicts a toy problem that serves well as an illustration.27 The state space consists of four states:

**FIGURE 10.23** **A simple infinite-horizon MDP.**

  When we are at state 1, we stay there with probability 0.7, and we will move to state 2 with probability 0.3. In state 1, there is no decision to make.

  In state 2, we have a choice between two actions. We may "flush" the state and move back to state 1, earning an immediate reward of 10; if we do not flush, the next state is random: At the next time instant we may still be there, or move on to state 3, with probabilities 0.8 and 0.2, respectively.

  In state 3, a similar consideration applies, but if we flush, we get a larger immediate reward of 15; the probability of moving on to a better state is 0.1, which is lower for state 3 than for state 2.

  If we are patient enough and we reach state 4, then we may move back to square 1 and earn an immediate reward of 20.

Clearly, there is a trade-off between flushing the state immediately and waiting for better opportunities, which is not too different from early exercise decisions in American-style derivatives, where we have to compare the immediate payoff from early exercise against the continuation value. Since this is an infinite-horizon problem, we also need to define a discount factor, say, γ = 0.8.

Ths script of Fig. 10.24 implements value iteration for this toy example. In this case, we modify value iteration by maximizing the value with respect to the available actions, rather than minimizing it. The following snapshot shows the optimal solution that we find by its application:

**FIGURE 10.24** **Numerical value iteration for a simple MDP.**

    > probs = c(.7,.8,.9,0)
    > payoff = c(0,10,20,30)
    > discount=.8
    > out <- PW311_ValueIteration(probs, payoff, discount)
    > out $stateValues
    [1] 9.677415 17.741931 27.741931 37.741931
    $policy
    [1] 0 1 1 0

When the action is 0, we wait and hope to move to a better state; if the action is 1, we grab the reward and move back to square 1. Note that, for the sake of simplicity, we do not use full state transition probabilities as the input; we only use the probability of remaining in each state if we wait and see. We observe that we should always flush the state, when we have to make a decision. A good check on the values of states 2, 3, and 4 is that they differ by 10, which is exactly the increment in the payoff if we start in a better state and apply the optimal policy. With the above data, the best policy is greedy, in the sense that we should always go for the immediate payoff, without waiting for better opportunities. This depends on two factors: the low probability of moving to a better state and the heavy discount factor. Let us play with these numbers to check our intuition:

    > probs = c (.6,.6,.6,0)
    > payoff = c(0,10,20,30)
    > discount=.95
    > out <- PW311_ValueIteration(probs, payoff, discount)
    > out
    $stateValues
    [1] 60.51969 68.48281 77.49371 87.49371
    $policy
    [1] 0 0 1 0
    > probs = c (.6,.6,.6,0)
    > payoff = c(0,10,20,30)
    > discount=.99
    > out <- PW311_ValueIteration (probs, payoff, discount)
    > out
    $stateValues
    [1] 342.1122 350.7518 359.6096 368.6910
    $policy
    [1] 0 0 0 0

It is important to notice that, in this very simple setting, we may easily compute the expectation, since it involves a finite sum with very few terms. In other settings, this may be very hard to do, possibly because of the sheer size of the state space, or because the dynamics are complicated and it is very difficult to find the transition probability matrix. Later, we will consider approximate dynamic programming approaches that rely on Monte Carlo simulation and do not require an explicit transition probability matrix. Monte Carlo simulation is also quite useful to simulate a policy which is implicit in the value function. The code in Fig. 10.25 receives the problem data, as well as the initial state and the value function, and it estimates the value of the optimal policy over a given number of steps. Due to discounting, there is little point to simulate a large number of steps. For instance 0.840 = 0.0001329228. Thus, numRepl complete replications should be run, which may also be used to come up with a confidence interval.

**FIGURE 10.25** **Simulating a policy for a simple MDP.**

### 10.7.2.2 Value iteration for continuous-state problems

When dealing with a continuous-state problem, we have to adopt a suitable discretization strategy for the state space, but also for the value function. Cubic splines are one possible choice, but a more general idea is to approximate the value function by a linear combination of simple building blocks:

The functions φ _j_ (·) are called basis functions, and they allow us to boil down the infinite-dimensional functional equation (10.51) to the finite-dimensional problem of finding the coefficients _c_ _j_ , _j_ = 1,..., _n_. This kind of approach is quite common in numerical methods for functional equations, including partial differential equations and integral equations, and it is called the _collocation method._ In this context, the points in the discretized state space are called collocation nodes. Given collocation nodes **s** 1,..., **s** _n_ , where now the subscript refers to the discretization and not time, the Bellman equation for each state **s** _i_ reads:28

(10.53)

The reader may notice that if the number of collocation nodes and the number of basis functions are indeed the same, Eq. (10.53) is a set of _n_ nonlinear equations in the _n_ unknown coefficients _c_ _j_. We may rewrite it as

where Φ is the collocation matrix with elements Φ _ij_ = φ _j_ ( **s** _i_ ), and the collocation function ν( **c** ) is a vector function with components

To solve the nonlinear equations, we may use the Newton's method. When dealing with a single equation _f_ ( _x_ ) = 0, where _x_    , the method consists of the following iterative scheme:

When dealing with a system of linear equations, which can be represented as **F** ( **x** ) = 0, where **F** :  _n_ →  _n_ , the division by the first-order derivative is replaced by the premultiplication by the inverse of the Jacobian matrix:29

We recall that the entries of the Jacobian matrix are the first-order derivatives of each component of the vector function **F** (·) with respect to each element of the vector variable **x**. In our case, we must take a derivative of a function defined by an optimization operator. To this end, we need a result in sensitivity analysis of optimization problems, known as the envelope theorem.30 Consider an optimization problem depending on a vector α    _m_ of parameters:

The solution of the problem yields an optimal solution **x** *(α) depending on the parameters, as well as an optimal cost

We may be interested in investigating the sensitivity of _g_ (·) with respect to the parameters. The envelope theorem, under some suitable conditions, yields a surprisingly simple result:

By putting all of this together, we come up the iteration scheme

where the Jacobian matrix Θ( **c** ) may be computed by resorting to the envelope theorem,

and **x** _i_ * is the optimal decision for the optimization problem that we solve to find the value of the collocation function ν _i_ ( **c** ) corresponding to state **s** _i_.

We are left with one issue: How can we approximate the expectation, assuming the shock   is a continuous random variable? We may use the scenario generation approaches that we have considered in Section 10.5.3. For instance, we may use a Gaussian quadrature formula with weights (probabilities) π _k_ and nodes (discretized shock values)  _k_ , respectively, _k_ = 1,..., _K_. Therefore, in order to evaluate the collocation function, for each collocation node **s** _i_ we have to solve the following optimization problem:

with Jacobian

We will not delve into this approach any further, but the message is that, by choosing a suitable set of basis functions, we may approximate the value function. The idea can also be applied when the dimensionality of the random shocks precludes the application of quadrature formulas and we have to resort to Monte Carlo sampling.

### 10.7.2.3 Speeding up value iteration

Value iteration is a fairly straightforward approach, whose convergence is not too difficult to prove in theory. However, it may suffer from slow convergence in practice. There are a few ways to speed convergence up, and we refer to the end of chapter references. Here we just want to point out one intrinsic weakness of value iteration. The value function is instrumental in determining the optimal policy in feedback form. It may happen that we do find the optimal policy relatively early, but the value function takes many more steps to converge, as it may take several iterations to assess the value of the optimal policy. Thus, we may try to improve convergence by a better assessment of the value of a policy. This leads to policy iteration approaches. Due to space limitations, we cannot illustrate policy iteration, but we may mention that value and policy iteration can be integrated within approximate dynamic programming strategies.

## 10.7.3 A NUMERICAL APPROACH TO CONSUMPTION–SAVING

In Section 1.4.3 we have used a simple life-cycle consumption–saving problem to motivate stochastic DP. Here we consider a little variation on the theme in order to illustrate the application of numerical methods to a simple but interesting problem. The additional feature that we introduce is a stylized model of income uncertainty based on a discrete-time Markov chain. Let us summarize our modeling assumptions:

  Time is discretized by time instants indexed by _t_ = 0, 1,..., _T_ , where the time horizon _T_ is assumed deterministic. In more realistic models, quite common in pension economics, the time horizon is a random variable whose distribution is given by mortality tables.

  At time _t_ = 0, the decision maker (or agent) is endowed with an initial wealth _W_ 0 and has to decide the consumed amount _C_ 0. It is not possible to borrow money, so 0 ≤ _C_ 0 ≤ _W_ 0. What is not consumed is the saved amount, _S_ 0 = _W_ 0 − _C_ 0.

  A second decision is the allocation of _S_ 0 between a risky and a risk-free asset; let α0   0, 1] be the fraction allocated to the risky asset, _R_ 1 the rate of return of the risky asset, and _r_ _f_ the rate of return of the risk-free asset. Note that we use _R_ 1 rather than _R_ 0 to emphasize that this piece of information is not known when the decision is made; in other words, return _R_ 1 is realized over the time period _t_ = 1, defined by the pair of time instants (0, 1) (see [Fig. 10.19). For the sake of simplicity, both the risk-free rate and the distribution of risky return are assumed constant over time. There is no intertemporal dependence between risky returns, which are assumed independent.

  At time instant _t_ = 1, the available wealth _W_ 1 is the sum of capital and labor (noncapital) income. The capital depends on the realized portfolio return:

The labor income is _L_ 1, which may take one of three values depending on the state of employment. Again, our notational choice emphasizes that this piece of information is not known when the decision is made at _t_ = 0.

  The state of employment at time _t_ , denoted by λ _t_ , may take three values in the set   = {α, β, η}, and _L_ α > _L_ β > _L_ η. We may interpret η as "unemployed," α as "fully-employed," and β as an intermediate situation. The dynamics of this state is modeled by a matrix of time-independent transition probabilities with elements

(10.54)

The initial employment state λ0 is given, and the corresponding income is already included in the initial wealth _W_ 0.

  An immediate utility _u_ ( _C_ _t_ ) is obtained from consumption.

  All of the above holds, with the natural adjustment of time subscripts, for time periods up to time _t_ = _T_ − 1. At time _t_ = _T_ , the terminal wealth _W_ _T_ is fully consumed; the model could be extended by accounting for some utility from bequest. This implies that terminal wealth is completely consumed, _C_ _T_ = _W_ _T_.

  The overall objective is to maximize total expected utility

where γ   (0, 1) is a subjective discount factor.

In order to tackle the above problem within a dynamic programming framework, we have to specify the state variables. In our case, the natural choice is the pair

The peculiarity of this model is the mixed nature of the state, which is continuous with respect to the first component and discrete with respect to the second one. Dynamic programming requires to find the set of value functions

with the boundary condition

The general form of the functional equation is

(10.55)

where the notation E _t_ [·] points out that expectation is conditional on the current state. The dynamic equations for state transitions are

(10.56)

(10.57)

where _M_ π in Eq. (10.56) represents the stochastic evolution of the employment state according to the matrix II collecting the transition probabilities of Eq. (10.54), and the evolution of wealth in Eq. (10.57) depends on the random rate of return of the risky asset and the labor income _L_ _t_ +1 which is a function of employment state λ _t_ +1. Note that we are also assuming that the employment state evolution and the return from the risky asset are independent. If we assume that both financial returns and employment depend on underlying macroeconomic factors, we should account for their correlation.

Now, in order to solve the functional equation (10.55) numerically, we have to specify the following three pieces of the puzzle:

**1**. The discretization of the state space

**2**. The approximation of the value function at each time instant

**3**. The discretization of the conditional expectation

Then, we will state the corresponding optimization model to be solved at each point in the grid and prepare R code to solve the DP recursion and simulate the optimal policy.

### State space discretization

We have to discretize only the wealth component of the state variable. The simplest idea, as we have seen in Section 10.7.1, is to use a uniform grid that does not depend on time. This may be accomplished by setting up a grid of wealth values _W_ ( _k_ ), _k_ = 0, 1,..., _K_ , where _W_ (0) = 0 and _W_ ( _K_ ) =   is some sensible upper bound on wealth. One might wonder if such a bound exists, if the probability distribution of return of the risky asset has infinite support; depending on how the value function is approximated, we may need a way to _extrapolate_ outside the grid, whereas in the budget allocation problem we only have to interpolate inside the grid. Another issue is if we should really use the same bound for all time periods, as wealth might increase; adaptive grid may be used to improve accuracy and/or efficiency. However, in our case, the agent consumes wealth along the way and unlike pension models, we have no accumulation-decumulation phase. Hence, for the sake of simplicity, we refrain from using adaptive grids and just use a time-invariant grid of states:

where we suppress dependence on time to ease notation.

### Approximation of the value function

To approximate the value function, we use cubic splines, as we did in Section 10.7.1. The value function depends on two state variables, but the second one is discrete and can only take three values. Hence, we associate three cubic splines

with each time bucket, depending only on wealth. The nodes for these splines are defined by the discretized grid **s** _k_ , _i_. As we have already pointed out, we may need to extrapolate the splines for large values of wealth. Whatever choice we take, there is a degree of arbitrariness, but some standard choices are available in R, in the form of options for use with the function splinefun:

  If the type option is set to natural, a natural cubic spline is built, and extrapolation is linear, using the slope of nearest point on the grid.

  If the choice is monoH.FC, interpolation based on Hermite polynomials is carried out, ensuring the monotonicity in the data, if present.

  The hyman is similar to the previous one, but cubic polynomials are selected.

### Discretization of the conditional expectation

We have to discretize only the dynamics described by Eq. (10.57), which in turn requires generating suitable return scenarios for the risky asset. Let us assume that the price of the risky asset is modeled by geometric Brownian motion. If we assume that time periods correspond to one year, this means that consecutive prices of the asset are related by

where   ~ N(0, 1). This in turn implies that the annual rate of return is

All we need, then, is to discretize the standard normal distribution, e.g., by Gaussian quadrature, or by any other scenario generation strategy that we have discussed in Section 10.5.3.5. Whatever approach we select, we will define scenarios characterized by realizations  _l_ , with corresponding rate of return _R_ _l_ , and probabilities _q_ _l_ , _l_ = 1,..., _L_. Note that the subscript _l_ has nothing to do with time, as we assume that the rates of return of the risky asset for different time periods are independent and identically distributed. Note that μ is related to the continuously compounded return on the risky asset, but the risk-free rate _r_ _f_ must be expressed with annual compounding for consistency in Eq. (10.57).

### The optimization model

After all of the above choices, we are finally ready to state the discretized optimization model corresponding to the single-step problem (10.55). At time _t_ , we have one such problem for each point in the state grid **s** _k,i_ , where wealth available is _W_ _t_ = _W_ ( _k_ ) and the current employment state is λ _t_ =  . We have discretized the return of the risky asset with _L_ scenarios, associated with probability _q_ _l_ and rate of return _R_ _l_ , and there are three possible employment states; hence, after making decisions _C_ _t_ and α _t_ , the next state in the optimization model is one of _L_ × 3 future scenarios, indexed by ( _lj_ ) and characterized by available wealth _W_ _t_ +1( _lj_ ). Since the two risk factors are assumed independent, each probability is given by _q_ _l_ x π _ij_. Hence, the optimization model _P_ ( **s** _k,i_ ), conditional on the state at time _t_ , reads as follows:

(10.58)

(10.59)

where the objective function requires knowledge of the spline functions _v_ _t_ +1, _j_ (·) for the next time period. At the terminal time instant _T_ the available wealth is fully consumed. Therefore, the boundary conditions are

We solve the functional equation backward to find the approximate value functions for each time period _t_ = _T_ − 1, _T_ − 2,..., 1. To find the initial decision, we solve the above problem at time instant _t_ = 0, with initial wealth _W_ 0 and current employment state λ0; in the solution we use the splines _v_ 1, _j_.

### Implementation in R

The code implementing numerical DP for the consumption-saving problem is shown in Fig. 10.26. The function DPSaving receives the following input arguments:

**FIGURE 10.26** **Numerical DP for the consumption–saving problem.**

  The initial states w0 and emp10.

  The utility function utilFun and the discount factor gamma.

  The number of time periods numSteps.

  The specification of the wealth grid, consisting of numPoints points ranging from Wmin to Wmax.

  The vector income containing the income for each state, and the corresponding transition matrix transMatrix.

  The return scenarios retScenarios and the corresponding probabilities probs for the risky asset, as well as the risk free rate rf.

  The splineOption parameter specifying the type of spline, which is natural by default, but could be monotonicity preserving.

The output of the function is the list valFunsList. This is a nested list containing, for each time period, a list of three spline functions, one for each employment state. To generate the value functions, the code does the following:

  It prepares the vector wealthValues containing the wealth values on the grid.

  It allocates the matrix valFunMatrix, which stores the value functions at each wealth value on the grid, for each employment state; these values are used to build the spline functions.

  valFunsList [[numSteps]] is the list of value functions for the last time instant, and it consists of a three-element list referring to the utility function (at the last stage, whatever the employment state, the agent consumes the available wealth).

  Then, the main loop goes backward in time preparing the objective function at each time step, according to the optimization moodel (10.58); the objective function objFun depends on the two decision variables and is the sum of immediate utility and the discounted expected value of the next stage value function.

  The model is solved by nlminb, which minimizes a nonlinear function subject to bounds; note that we change the sign of objFun, as it has to be maximized; the initial solution for the optimization solver is arbitrarily chosen (we consume 50% the available wealth and 50% of saving is allocated to the risky asset).

  valFunMatrix, on exit from the loop for each time stage, stores the values of each state; this is used to build the three splines, one per employment state, by the selected method (natural spline by default); the result is assigned to valFunsList[[t]][[i]].

The code in Fig. 10.27 is used to run the numerical DP. The function MakeScenariosGQ generates scenarios of risky asset returns using Gaussian quadrature. We use 10 quadrature nodes, and the state grid consists of 30 points. It is important to plot some value functions to check that they make sense, which also depends on our selection of grid boundaries and the type of spline. Note that the natural spline can be used to extrapolate outside the grid, with a slope corresponding to the closest boundary point on the grid. Figure 10.28 shows some value functions in the case of a logarithmic utility function. The functions look concave and increasing, which is what we expect. We may also consider a power utility function

**FIGURE 10.27** **Running the function** DPSaving **and plotting selected value functions.**

**FIGURE 10.28** **Value functions for logarithmic utility.**

The degree of risk aversion of the power utility function depends on the coefficient ρ. When ρ = 1, the power utility boils down to the logarithmic one. A moderate degree of risk aversion is associated to ρ = 5.31 In Fig. 10.29 we show the impact of the choice of spline function, for a power utility with ρ = 5. The value functions refer to _t_ = 9, 10 and employment state β, and have been built using both the natural and the monotonicity preserving splines, corresponding to spline option monoH.F. We notice that the power utility at the end of the time horizon _t_ = 10 displays larger risk aversion than the logarithmic utility. When we step back to _t_ = 9, the natural spline yields an unreasonable function featuring some spurious oscillations. The monotonicity preserving spline, on the contrary, avoids this behavior. This shows that some care must be taken when choosing an interpolation strategy, especially with long time horizons, since errors tend to cumulate when we proceed backward in time.

**FIGURE 10.29** **Value functions for power utility.**

### Simulation of the resulting policy

As we have remarked before, a notable advantage of stochastic DP is that we obtain a set of value functions that can be used to simulate a sequence of decisions out of sample, e.g., in scenarios that are different from those used to learn a policy. We might even check robustness against uncertainty in the probability distribution itself. In this section we use Monte Carlo runs to compare the performance of the DP policy against a simple rule based on fixed decisions. According to the fixed rule, at each time instant from _t_ = 0 to _t_ = _T_ − 1 a fixed percentage of the available wealth is consumed, and a fixed percentage of savings is allocated to the risky asset; at the end of the planning horizon, the whole wealth is consumed. The first thing we need is a piece of code to generate scenarios of risky asset returns and employment states. This is accomplished by the code in Fig. 10.30. The returns are generated by sampling a geometric Brownian motion, and the stochastic process for the employment state is a discrete-time Markov chain.32 The output sample paths are collected in a list consisting of two matrices with numRepl rows and numSteps columns. The code in Fig. 10.31 receives these scenarios and returns a list including the discounted utility, as well as the corresponding sample paths for α _t_ , _C_ _t_ , and _W_ _t_. Clearly, α _t_ is fixed here, but we prefer to return the same output as the code in Fig. 10.32, which solves an optimization model at each step.

**FIGURE 10.30** **Sample path generation for the life-cycle problem.**

**FIGURE 10.31** **Simulating a fixed decision policy.**

**FIGURE 10.32** **Simulating the optimal DP policy.**

Figure 10.33 reports a script to compare the optimal DP and the fixed policy, using the same out-of-sample scenarios (100 scenarios are generated to this purpose). These out-of-sample scenarios should not be confused with the scenarios that are used to solve the DP recursion, as well as each single-stage decision problem. We may compare the average utility achieved, for the logarithmic utility function as follows:

**FIGURE 10.33** **Comparing the optimal DP and a fixed policy.**

    > mean(utilValsFixed)
    [1] 31.28172
    > mean(utilValsDP)
    [1] 32.3301
    > t.test(utilValsDP,utilValsFixed)
    Welch Two Sample t-test
    t = 2.9255, df = 196.126, p-value = 0.003845
    alternative hypothesis: true difference in means
    is not equal to 0

The difference does not look huge, even though a paired _t_ -test suggests that it is statistically significant. Actually, comparing utility functions may be misleading, as they are meant as ordinal, rather than cardinal utility. If we repeat the experiment for the power utility function, with ρ = 5, we obtain an apparently different picture:

    > mean(utilValsFixed)
    [1] −6.53674e-05

    > mean(utilValsDP)
    [1] −2.547044e-05
    > t.test (utilValsDP,utilValsFixed)
    Welch Two Sample t-test
    t = 3.4024, df = 124.686, p-value = 0.0008983
    alternative hypothesis: true difference in means
    is not equal to 0

Now the difference looks more relevant, but once again the comparison should just be taken as an illustration. Furthermore, we could use the simulation-based optimization strategies of Section 10.4 to find the best settings for the fixed policy. We urge the reader to further investigate these policies. For instance, by checking the paths of α _t_ , which is stored in the output matrix out$alphaPaths by the simulation function, we find that with the logarithmic utility we should always invest 100% in the risky asset, whereas the allocation is more conservative for the power utility with ρ = 5.

# 10.8 Approximate dynamic programming

The numerical methods for stochastic dynamic programming that we have discussed in the previous section are certainly a useful tool for tackling some dynamic optimization problems under uncertainty, but they are not a radical antidote against the curses of dimensionality. Over the years, an array of techniques, collectively labeled as approximate dynamic programming (ADP) have been proposed. They rely more exensively on Monte Carlo methods to learn an approximation of the value function _V_ _t_ ( **s** _t_ ) at time instant _t_ for state **s** _t_ , by an iterative process. Let   be such an approximation at iteration _n._ The iteration superscript _n_ should not be confused with the time subscript _t_ , which is relevant for finite-horizon problems; for infinite-horizon problems, the notation may be replaced by  . The idea is to use Monte Carlo methods to generate observations of the value functions at different states, by applying approximate policies for a sequence of sample paths. After having observed _n_ sample paths, the current approximation is  .

ADP is related to reinforcement learning and borders with artificial intelligence and machine learning. It is also worth mentioning that some model-free forms of ADP have been devised for applications in which there is no model of the system, and one can only learn by trial-and-error experimentation with the real system. Needless to say, we cannot offer a full account of ADP methods,33 but we can point out some of their most relevant features:

  In Section 10.8.1 we outline a basic version of ADP, stressing the role of Monte Carlo sampling to learn a sequence of approximated value functions.

  In Section 10.8.2 we introduce a powerful device in ADP: the post-decision state variable.

  Finally, in Section 10.8.3, we revisit the toy MDP of Section 10.7.2.1 to illustrate the application of _Q_ -learning, which can be interpreted as an ADP approach based on post-decision state variables.

  We will see another, more significant, application of ADP in Section 11.3.3, where we show how the idea can be used to price American-style options by approximating value functions with linear regression and a set of basis functions.

## 10.8.1 A BASIC VERSION OF ADP

Figure 10.34 outlines a very basic version of ADP. This procedure is just the statement of a principle, rather than a working algorithm. Nevertheless, we may observe one common ingredient of many ADP methods: the absence of the usual backward recursion. Indeed, the algorithm generates a sequence of states and decisions, based on the approximation of the value function for iteration _n_ − 1, by going _forward_ in time. The solution of the optimization problem (10.60) provides us with an observation of the value of state **s** _t_ _n_ , which is used to improve the approximation of the value function. How this is accomplished exactly depends on the kind of state space. If we are dealing with a small and finite state space, like the Markov decision process of Fig. 10.23, the value function is just a vector listing the value for each state. Then, we might update the value function for the visited state **s** _t_ _n_ as follows:

**FIGURE 10.34** **A simple ADP approach.**

(10.61)

where the smoothing coefficient α _n_   (0, 1) mixes the new and the old information. When looking at Eq. (10.61), it is important to realize that we are _not_ estimating the expected value of a random variable by sampling a sequence of i.i.d. observations: The decisions we make depend on the current approximation of the value function and, since the approximations themselves change over the iterations, we are chasing a moving target. An effective ADP algorithm, among other things, must strike a difficult balance between conflicting requirements:

  _Speed and convergence._ If the smoothing coefficient α _n_ is close to zero, much weight is given to the previous aproximation of the value function, possibly resulting in painfully slow learning. However, if it is too large, erratic behavior may arise. It is possible to adapt the coefficient dynamically, starting with a relatively large value in order to quickly improve the approximation at early stages, and then reducing it in order ensure convergence at later stages. We also mention that convergence may be improved if we start with a sensible initialization of the value function, possibly found by simulating the application of some good heuristic policy.

  _Exploitation and exploration._ When solving the problem (10.60) at each iteration, we rely on the current approximation of the value function. It may happen that a good state looks like a very bad one because of a poor value function approximation. The problem is that we may make decisions in such a way to steer away from that state, preventing the possibility of learning that it is actually a good one. Hence, we should allow for a significant amount of exploration in order to learn where good states are located. Unfortunately, this conflicts with the need for exploitation, i.e., using the knowledge we have about good states in order to produce good policies.

The above observations suggest that there are indeed quite some challenges in devising a good ADP algorithm. However, from the pseudocode of Fig. 10.34 it may be difficult to see any definite advantage with respect to more classical numerical approaches in dealing with the three curses of dimensionality:

  One curse of dimensionality concerns the difficulty in solving each optimization model, when the decision variables **x** _t_ are continuous and high-dimensional. From this point of view, we may rely on sophisticated optimization technology, but there is really no difference between ADP and classical numerical DP.

  The traditional curse of dimensionality concerns the dimensionality of the state space, and it is clearly impossible to update the approximation of value functions using Eq. (10.61) for a continuous state space. Discretizing the state space using a regular grid is only feasible for low-dimensional spaces, but we may approximate value functions like we have done with the collocation method of Section 10.7.2.2, relying on a set of basis functions φ _j_ (·), _j_ = 1,..., _m_ :

(10.62)

Here we assume that the set of basis functions is the same for all time instants, whereas the coefficients _c_ _jt_ can depend on time. Clearly, finding a suitable set of basis function capturing the features of good states is non trivial. Nevertheless, we may estimate the coefficients by linear regression, based on the observations provided by Eq. (10.60). Since, as we have mentioned, we are chasing a moving target, it may be advisable to use weighted least squares, in order to assign more weights to recent observations. Furthermore, rather than using least squares at each step, recursive procedures can be used to estimate the coefficients iteratively, rather than classical batch estimation where all the observations are used.34

  A last curse of dimensionality, however, is still present in the ADP procedure of Fig. 10.34, as we need the expectation of the of the value function for the next state at the next time step. For high-dimensional random variables  _t_ , this is an overwhelming task. A powerful trick to get rid of the expectation is to resort to a different way of writing the recursive Bellman equation, based on post-decision state variables, which we discuss in the next section.

## 10.8.2 POST-DECISION STATE VARIABLES IN ADP

The classical way of writing the Bellman equation involves a thorny expectation. When dealing with a stochastic optimization problem, there is no way to avoid having to deal with an expectation somewhere. Nevertheless, we may try to move it to a more convenient place, by rewriting the functional equation around post-decision state variables.35 Before introducing the concept, it is useful to illustrate the rationale behind the approach in a simple context.

###   Example 10.15 Forecasting by linear regression models

Consider a multiple linear regression model

where _m_ factors are used as regressors (explanatory variables, if you prefer). Now, imagine that we want to use this model to forecast _Y_ _t_. If the regressors are stochastic, there is no way to use a model like this, unless we are able to forecast the factors themselves. An alternative model could be

Now we are relating the future value of _Y_ _t_ to _past_ observations of the factors, which may be more useful for forecasting purposes.

The idea of the above example can also be applied to ADP, by breaking down the transition dynamics

in two steps. The sequence of events is:

**1.** We observe the state **s** _t_.

**2.** We make the decision **x** _t_.

**3.** The random variable  _t_ +1 is realized, leading to a new state **s** _t_ +1.

This results in the following history of the stochastic decision process:

We may introduce a post-decision state variable **s** _t_ _x_ , which represents the system state after the decision **x** _t_ , but _before_ the realization of the disturbance  _t_ +1. The state evolution is modified as

Let us illustrate the idea by a simple example.

###   Example 10.16 Post-decision state in the life-cycle problem

In Section 10.7.3 we have defined the state as the pair **s** _t_ = ( _W_ _t_ , λ _t_ ), consisting of available wealth and the current employment state. This is a _pre-decision_ state variable, as we still have to make the consumption and allocation decision. After making the decision, we may introduce the post-decision state variable **s** _t_ _x_ = ( _Q_ _t_ , _B_ _t_ ,λ _t_ ), where _Q_ _t_ and _B_ _t_ are the amounts invested in the risky and risk-free asset, respectively, and λ _t_ has the same interpretation as before (in fact, the employments state is not affected by our decision).

In the above example, we introduce one more component in the state variable. In general, if we have _n_ assets and there are no transaction costs, the essential state variable for an asset allocation problem is just available wealth. If we introduce post-decision variables, we need _n_ variables, which increases complexity a lot. However, if we include transaction costs into the model, there is no increase in complexity, as we must account for each asset individually, anyway. Whether the post-decision state is more or less complex than the pre-decision state depends on the problem structure.36 However, the significant advantage of the former is apparent when we rewrite the optimality equations in terms of _V_ _t_ _x_ ( **s** _t_ _x_ ), the value of being in the post-decision state **s** _t_ _x_ , rather than in terms of _V_ _t_ ( _s_ _t_ ). We have:

(10.63)

(10.64)

(10.65)

Equation (10.63) relates the value function for the post-decision state variable at time _t_ − 1 with the value function for the pre-decision state variable at time _t_ ; the expectation is with respect to  _t_ , the disturbance term used in the transition from **s** _t_ −1 _x_ to **s** _t_. The most relevant point is the form of Eq. (10.64). This is the optimality equation written using the post-decision value function: This equation does not involve any expectation and requires the solution of a _deterministic_ optimization problem. Equation (10.65) is just Eq. (10.63) moved forward by one time period. Clearly, we obtain the standard Bellman equation by plugging (10.65) into (10.64):

But if we substitute (10.64) into (10.63) we find a different form of the optimality equation:

(10.66)

Now, the expectation is outside the minimization. Obviously, we did not eliminate the expectation altogether, but the outside expectation may be dealt with by statistical learning based on Monte Carlo methods. With this alternative DP form, we get rid of one curse of dimensionality, which may have a significant impact on some problems.

If we adopt post-decision state variables, the basic ADP algorithm of Fig. 10.34 is modified as shown in Fig. 10.35. A few comments are in order:

**FIGURE 10.35** **ADP based on the post-decision state variable.**

    is the approximation at iteration _n_ of the post-decision value function at time instant _t._

  We denote by _S_ _t_ _x_ (·,·) the transition function from the current pre-decision to the next post-decision state. This transition function is deterministic.

  An important point is how we use the observation   to update the post-decision value function. When tackling the optimization problem (10.67), we are at pre-decision state **s** _t_ _n_ , and the optimal decision leads us to the post-decision state **s** _t_ _x,n_. However, we do _not_ want to update the value of **s** _t_ _n_ , as in order to use this knowledge in the next iterations we should use an expectation, and the related curse of dimensionality would creep back into our algorithmic strategy. We update the value of the post-decision state **s** _t_ −1 _x,n_ that was visited before reaching the current pre-decision state **s** _t_ _n_. In other words, we use the optimal value of the objective function in problem (10.67) to estimate the value of the post-decision state that lead us to the current post-decision state **s** _t_ _n_ , via the realization of the random variable  _t_ _n_. Hence, we take   as an observation of the random variable whose expectation occurs in the optimality equation (10.66).

Needless to say, the procedure of Fig. 10.35 is just an outline, and many critical details must be filled to come up with in a working algorithm. We refer to the extensive treatment in [37] for several examples. The idea is illustrated in the next section for the toy Markov decision process of Section 10.7.2.1.

## 10.8.3 _Q_ -LEARNING FOR A SIMPLE MDP

We refer again to the MDP depicted in Fig. 10.23. In this very small problem instance computing an expectation is quite easy and poses no challenge at all; however, we may use this example to investigate how we can adapt the idea of post-decision states. For instance, imagine that we are in state 2. If we decide to take the immediate payoff, we will be at state 1 at the next step; otherwise, a flip of a (biased) coin will decide if we will stay at state 2 or move on to state 3. Let _s_   {1, 2, 3, 4} be the current pre-decision state and _a_   {0, 1} be the selected action (we assume that 1 corresponds to taking the immediate payoff and getting back to state 1). The post-decision state, for this problem structure, is the pair ( _s, a_ ).37 In general, the action _a_ selects a transition probability matrix, which is used explicitly in value iteration. We can estimate the value of taking action _a_ in state _s_ , denoted by _Q_ ( _s, a_ ), by a strategy known as _Q_ -learning, which may be applied to discrete-state, discrete-action problems with an infinite horizon. The values _Q_ ( _s, a_ ) are called _Q_ -factors, and we learn them iteratively in this approach. Let   be the estimate of the factor at iteration _n_ − 1. Then, if we are in the pre-decision state _s_ _n_ at iteration _n_ , we select the next action as

(10.68)

Note that we do not need any expectation to solve this optimization problem. After selecting the optimal action, we evaluate the immediate cost (or reward) and observe the next pre-decision state _s_ _n_ +1. Note that we associate the next state with the next iteration _n_ \+ 1, rather than the next time instant, as we are dealing with an infinite-horizon problem solved iteratively. When using Monte Carlo simulation, the next state may be generated by a suitable sampling strategy, and we do not need to know the transition matrix explicitly.38 Now we may update the _Q_ -factor on the basis of the new observation39

(10.69)

using the smoothing equation

where α _n_ -1   (0, 1) is a smoothing coefficient. To see the connection with value functions, consider the estimate of the value of state _s_ _n_ ,

and rewrite Eq. (10.69) as

(10.70)

Indeed, the essential advantage of the approach is that no expectation is involved. The disadvantage is that a lot of _Q_ -factors are needed if the state and the action spaces are large.

The R code of Fig. 10.36 shows how _Q_ -learning may be applied to the simple MDP of Section 10.7.2.1. The code is rather straightforward, and the only detail worth noting is that action indexes in R are incremented by 1 when accessing vectors and matrices. In fact, we associate action _a_ = 1 with the decision of collecting the immediate payoff, and action _a_ = 0 with the decision of waiting. However, vector/matrix indexing starts from 1 in R, so those actions correspond to indexes 2 and 1, respectively. We also recall that we have to make a decision only in the interior states 2 and 3; in the boundary state 1 we cannot earn any immediate reward, and in the boundary state 4 we just collect the reward and move back to state 1.

**FIGURE 10.36** **Tackling the MDP of Section 10.7.2.1 by _Q_ -learning.**

Let us compare the result of _Q_ -learning with the result of value iteration. The _Q_ -factors have to be initialized. For action _a_ = 0, we initialize the factors to 50; for action _a_ = 1, the factors are initialized with the payoffs (zero for state 1). We select the smoothing coefficient α = 0.1 and run 10,000 iterations, obtaining the following output:

    > probs <- c(.7,.8,.9,0); payoff <- c (0,10,20,30)
    > discount <- 0.8
    > PW311_ValueIteration(probs, payoff, discount)
    $stateValues
    [1] 9.677415 17.741931 27.741931 37.741931
    $policy
    [1] 0 1 1 1

    > numSteps <- 10000; startState <- 1; alpha <- 0.1
    > startQFactors <- matrix(c(rep(50,4),payoff),ncol=2)
    > set.seed(55555)
    > PW311_QLearning(probs,payoff, discount,numSteps,
    startState,startQFactors,alpha)
    $QFactors
    [,1] [,2]
    [1,] 9.877252 0.00000
    [2,] 9.851703 18.63811
    [3,] 19.704227 25.67057
    [4,] 50.000000 40.12486
    $policy
    [1] 0 1 1 1

If we compare the factors corresponding to the optimal decisions with the state values obtained by value iteration, we find them in fairly good agreement. What really, matters, however, is that the state value estimates are good enough to induce the optimal policy. This may look reassuring, but what if we change the initialization a bit? In the following run, we initialize the factors for action _a_ = 0 to 500:

    > startQFactors <- matrix(c(rep(500,4),payoff),ncol=2)
    > set.seed(55555)
    > PW311_QLearning(probs,payoff,discount,numSteps,
        startState,startQFactors,alpha)
    $QFactors
            [,1]    [,2]
    [1,] 50.24164 0.00000
    [2,] 92.49797 10.00000
    [3,] 194.02190 20.00000
    [4,] 500.00000 70.24866
    $policy
    [1] 0 0 0 1

With this initialization, we assign a vary large value to the option of waiting, and after 10,000 iterations we fail to find the optimal policy. We may observe that the value of the action _a_ = 0 in states 2 and 3 is lower than its initial value, and we may argue that after several steps we will end up with the right choices. This is also a case for adopting a larger value of α, at least in the initial steps, in order to accelerate learning.40 However, the next run points out a more disturbing facet of value function initialization:

    > startQFactors <- matrix(c(rep(1,4),rep(0,4)),ncol=2)
    > set.seed(55555)
    > PW311_QLearning(probs,payoff,discount,numSteps,
                startState,startQFactors,alpha)
    $QFactors
        >    [,1]    [,2]
    [1,] 3.256359 0.00000
    [2,] 6.009678 0.00000
    [3,] 12.634094 0.00000
    [4,] 1.000000 32.60307
    $policy
    [1] 0 0 0 1

This is a case of bad initialization, too. The initial value of action _a_ = 0 in states 1, 2, and 3 seems to converge from below to the correct value. However, the corresponding value of the action _a_ = 1 is stuck to zero, and it will remain there. With the above version of _Q_ -learning, we will never choose action _a_ = 1 in those states, and we will never learn its true value. This is a general concern related to the balance between exploitation and exploration. On the one hand, we want to exploit a good policy to optimize the objective function; on the other hand, we want to explore actions and states in order to learn a possibly better policy.

One way to strike a better balance between these conflicting objectives is to select a random action every now and then, without solving the optimization problem (10.68). The code in Fig. 10.37 applies this idea, which is known as epsilon-greedy policy. The function is essentially the same as the _Q_ -learning version of Fig. 10.36, but it takes one additional parameter, eps, which is the probability of choosing the next action randomly; when we are in interior states 2 and 3, a coin is flipped, a random action is taken with probability  , and the currently best decision is taken with probability 1 −  . As we may see from the snapshot below, now we do find the optimal policy, even with the bad initialization:

**FIGURE 10.37** **Adapting the _Q_ -learning code of Fig. 10.36 to improve exploration (abridged code).**

    > startQFactors <- matrix (c (rep(1,4),rep(0,4)),ncol=2)
    > eps <- 0.1
    > set.seed(55555)
    > PW311_QepsLearning(probs,payoff,discount, numSteps,
            startState,startQFactors,alpha,eps)
    $QFactors
                [,1]    [,2]
    [1,] 10.423041 0.00000
    [2,] 15.748525 18.87826
    [3,] 4.153173 23.19135
    [4,] 1.000000 17.75869
    $policy
    [1] 0 1 1 1

We may also observe that the state values are not exact, but this is enough to find the optimal policy. Clearly, balancing exploration and exploitation is not trivial with large state spaces, and a successful strategy requires a lot of problem-specific adaptation. Another useful observation is that with discrete-state, discrete-action systems we do not necessarily need a very precise assessment of value functions in order to find the optimal policy. Indeed, alternative policy iteration strategies use a policy to assess state values and aim at improving the policy itself.

In continuous-state problems, the accuracy of the value function may be more or less relevant, depending on the case. For instance, in the newsvendor problem of Section 10.2.3, we may not need a very accurate representation of uncertainty, which has an impact on the approximation of the objective function, as even a solution reasonably close to the optimal one performs rather well out of sample. However, a poor representation of uncertainty may prove fatal in other applications; for instance, badly generated scenarios may contain arbitrage opportunities in portfolio optimization, which leads to unreasonable solutions. Finally, we note that there are problems where the main output of the optimization is the value of the objective function, rather than the decision variables or the policy. One example is pricing options with early exercise features. The regression-based ADP approach that we discuss later in Section 11.3.3 bears some similarity with _Q_ -learning and the example we have discussed here, as we must compare the value of waiting, the continuation value of the option, against the value of immediate exercise, the immediate payoff. These are not _Q_ -factors associated with discrete states, and after learning the policy it is important to simulate its application out-of-sample, on a large set of scenarios, in order to improve the estimate. Furthermore, bias issues should be carefully considered. As we shall see, a suboptimal policy yields a _lower_ bound on the option value, which need to be complemented by an upper bound.

# For further reading

  The most important skill in optimization is model building, rather than model solving. Readers interested in sharpening their model building skills will find plenty of enlightening examples in [46]. For more specific examples of optimization modeling in finance see, e.g., [13].

  An elementary treatment of optimization algorithms can be found in [8], where some model building tricks of the trade are also illustrated. If you look for deeper technical treatments of optimization methods, [1] and [47] are excellent references for nonlinear programming and integer programming, respectively.

  Model calibration problems are often solved as nonlinear programs; see, e.g., [12] or [23, Chapter 15]. An example of application of integer programming to financial optimization can be found in [5].

  Several Monte Carlo methods for optimization problems are discussed in [30], including some approaches that we did not mention, like the cross-entropy method.

  There is a huge literature on metaheuristics and related approaches. We did not cover tabu search; see, e.g., [16] for a financial application. Genetic algorithms are covered in [33]. For the original references on particle swarm optimization, see [14] and [36]. Several variants of swarm optimization have been proposed; see, e.g., [48] for the firefly variant and the use of Levy flights, or [42] for the quantum variant.

  A general reference on stochastic optimization is [41]. Simulation-based optimization is dealt with in [15]; see [27] for an extensive treatment of metamodeling.

  Modeling with stochastic programming is illustrated in [25]. An extensive treatment of theoretical and algorithmic issues can be found in [6] or [26].

  The key reference for approximate dynamic programming is [37]. Omer useful references are [2] and [3]. The former illustrates dynamic programming in general, and the latter is more focused on ADP. See also [4] for the use of neural networks as tools for the approximation of value functions. _Q_ -learning was introduced in the Ph.D. thesis by Watkins; see [45].

  Several applications of dynamic programming to economics can be found in [34], which also describes numerical DP methods implemented in a MATLAB toolbox. Several numerical DP tricks are described in [11].

  R offers several optimization packages; see <http://cran.r-project.org/web/views/Optimization.html> for a listing. However, if you need to solve really tough optimization problems, you may need to resort to state-of-the-art solvers like CPLEX (see <http://www-01.ibm.com/software/info/ilog/> or www.ilog.com) and Gurobi (see <http://www.gurobi.com/>). Both CPLEX and Gurobi offer an interface with R.

# References

1 M.S. Bazaraa, H.D. Sherali, and C.M. Shetty. _Nonlinear Programming. Theory and Algorithms_ (2nd ed.). Wiley, Chichester, West Sussex, England, 1993.

2 D.P. Bertsekas. _Dynamic Programming and Optimal Control Vol. 1_ (3rd ed.). Athena Scientific, Belmont, MA, 2005.

3 D.P. Bertsekas. _Dynamic Programming and Optimal Control Vol. 2_ (4th ed.). Athena Scientific, Belmont, MA, 2012.

4 D.P. Bertsekas and J.N. Tsitsiklis. _Neuro-Dynamic Programming._ Athena Scientific, Belmont, MA, 1996.

5 D. Bertsimas, C. Darnell, and R. Stoucy. Portfolio construction through mixed-integer programming at Grantham, Mayo, Van Otterloo and Company. _Interfaces_ , 29:49–66, 1999.

6 J.R. Birge and F. Louveaux. _Introduction to Stochastic Programming_. Springer, New York, 1997.

7 P. Brandimarte. _Numerical Methods in Finance and Economics: A MATLAB-Based Introduction_ (2nd ed.). Wiley, Hoboken, NJ, 2006.

8 P. Brandimarte. _Quantitative Methods: An Introduction for Business Management_. Wiley, Hoboken, NJ, 2011.

9 P. Brandimarte and A. Villa. _Advanced Models for Manufacturing Systems Management_. CRC Press, Boca Raton, FL, 1995.

10 J.Y. Campbell and L.M. Viceira. _Strategic Asset Allocation_. Oxford University Press, Oxford, 2002.

11 C. Carroll. Solving microeconomic dynamic stochastic optimization problems. Lecture notes downloadable from <http://www.econ.jhu.edu/people/ccarroll/index.html>.

12 T.F. Coleman, Y. Li, and A. Verma. Reconstructing the unknown volatility function. _Journal of Computational Finance_ , 2:77–102, 1999.

13 G. Cornuejols and R. Tütüncü. _Optimization Methods in Finance_. Cambridge University Press, New York, 2007.

14 R.C. Eberhart and J. Kennedy. Particle swarm optimization. In _Proceedings of the IEEE International Conference on Neural Networks_ , p. 1942 U1948, 1995.

15 M.C. Fu. Optimization by simulation: A review. _Annals of Operations Research_ , 53:199–247, 1994.

16 F. Glover, J.M. Mulvey, and K. Hoyland. Solving dynamic stochastic control problems in finance using tabu search with variable scaling. In I.H. Osman and J.P. Kelly, editors, _Meta-Heuristics: Theory and Applications_ , pp. 429–448. Kluwer Academic, Dordrecht, The Netherlands, 1996.

17 H. Heitsch and W. Roemisch. Scenario reduction algorithms in stochastic programming. _Computational Optimization and Applications_ , 24:187–206, 2003.

18 J.L. Higle. Variance reduction and objective function evaluation in stochastic linear programs. _INFORMS Journal of Computing_ , 10:236–247, 1998.

19 J.L. Higle and S. Sen. _Stochastic Decomposition_. Kluwer Academic Publishers, Dordrecht, 1996.

20 K. Hoyland and S.W. Wallace. Generating scenario trees for multistage decision problems. _Management Science_ , 47:296–307, 2001.

21 D.G. Humphrey and J.R. Wilson. A revised simplex search procedure for stochastic simulation response surface optimization. _INFORMS Journal on Computing_ , 12:272–283, 2000.

22 G. Infanger. _Planning under Uncertainty: Solving Large-Scale Stochastic_ _Linear Programs_. Boyd and Fraser, Danvers, MA, 1994.

23 J. James and N. Webber. _Interest Rate Modelling_. Wiley, Chichester, 2000.

24 K.L. Judd. _Numerical Methods in Economics_. MIT Press, Cambridge, MA, 1998.

25 P. Kall and S.W. Wallace. _Stochastic Programming_. Wiley, Chichester, 1994.

26 A.J. King and S.W. Wallace. _Modeling with Stochastic Programming_. Springer, Berlin, 2012.

27 J.P.C. Kleijnen. _Design and Analysis of Simulation Experiments_. Springer, Berlin, 2008.

28 M. Koivu. Variance reduction in sample approximations of stochastic programs. _Mathematical Programming_ , 103:463–485, 2005.

29 M. Koivu and T. Pennanen. Epi-convergent discretizations of stochastic programs via integration quadratures. _Numerische Mathematik_ , 100:141–163, 2005.

30 D.P. Kroese, T. Taimre, and Z.I. Botev. _Handbook of Monte Carlo Methods_. Wiley, Hoboken, NJ, 2011.

31 Y.K. Kwok. _Mathematical Models of Financial Derivatives_. Springer, Berlin, 1998.

32 C.D. Maranas, LP. Androulakis, C.A. Floudas, A.J. Berger, and J.M. Mulvey. Solving long-term financial planning problems via global optimization. _Journal of Economic Dynamics and Control_ , 21:1405–1425, 1997.

33 Z. Michalewicz. _Genetic Algorithms_ \+ _Data Structures_ = _Evolution Programs_. Springer, Berlin, 1996.

34 M.J. Miranda and P.L. Fackler. _Applied Computational Economics and Finance_. MIT Press, Cambridge, MA, 2002.

35 L.M.V.G. Pinto M.V.F. Pereira. Multi-stage stochastic optimization applied to energy planning. _Mathematical Programming_ , 52:359–375, 1991.

36 R. Poli, J. Kennedy, and T. Blackwell. Particle swarm optimization: An overview. _Swarm Intelligence_ , 1:33–57, 2007.

37 W.B. Powell. _Approximate Dynamic Programming: Solving the Curses of Dimensionality_ (2nd ed.). Wiley, Hoboken, NJ, 2011.

38 S.T. Rachev. _Probability Metrics and the Stability of Stochastic Models_. Wiley, New York, 1991.

39 A. Ruszczyński and A. Shapiro. Stochastic programming models. In A. Ruszczyński and A. Shapiro, editors, _Stochastic Programming_. Elsevier, Amsterdam, 2003.

40 A. Shapiro, D. Dentcheva, and A. Ruszczyński. _Lectures on Stochastic Programming: Modeling and Theory_. SIAM, Philadelphia, PA, 2009.

41 J.C. Spall. _Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control_. Wiley, Hoboken, NJ, 2003.

42 J. Sun, C.-H. Lai, and X.-J. Wu. _Particle Swarm Optimisation: Classical and Quantum Perspectives_. CRC Press, Boca Raton, FL, 2011.

43 A. Takayama. _Analytical Methods in Economics_. Harvester Wheatsheaf, New York, 1994.

44 C. Tsallis and D.A. Stariolo. Generalized simulated annealing. _Physica A_ , 233:395–406, 1996.

45 C.J.C.H. Watkins and P. Dayan. Q-learning. _Machine Learning_ , 8:279–292, 1992.

46 H.P. Williams. _Model Building in Mathematical Programming_ (4th ed.). Wiley, Chichester, 1999.

47 L.A. Wolsey. _Integer Programming_. Wiley, New York, 1998.

48 X.-S. Yang. Firefly algorithm, Lévy flights and global optimization. In M. Bramer, R. Ellis, and M. Petridis, editors, _Research and Development in Intelligent Systems XXVI_ , pp. 209–218. Springer, 2010.

An elementary introduction can be found in [8, Chapter 12].

In principle, we may stipulate that _H_ ( **x** ) = +∞ for an infeasible solution, but this is not very helpful computationally.

Tightening such bounds will speed up branch and bound methods, which are the standard solution methods for mixed-integer programs.

For the sake of notational simplicity, here we denote by _R_ the _total_ return, whereas when _R_ is the _rate_ of return, total return is 1 + _R_.

See Fig. 1.12 for a surface plot of the Rastrigin function in the bidimensional case.

The choice is not that obvious for many problems in combinatorial optimization, e.g., when solutions correspond to permutations of objects.

Affine independence in  _n_ means that the vectors ( **x** 2 − **x** 1),..., ( **x** _n_ +1 − **x** 1) are linearly independent. For _n_ = 2 this means that the three points do not lie on the same line. For _n_ = 3 this means that the four points do not lie on the same plane.

See, e.g., [21].

See Chapter 13 to see how risk measures may be included in an optimization model.

This is related to some issues with risk measures like value-at-risk; see Chapter 13.

Convexity of chance-constrained problems is discussed in [25].

See, e.g., [39] for a more detailed discussion.

Readers with a background on measure-theoretic probability will immediately recognize that this is a measurability condition with respect to a filtration.

We denote set difference by \\.

Two approaches in this vein are stochastic decomposition [19] and stochastic dual dynamic programming [35].

As we shall see later in the chapter, dynamic programming provides us with decisions in feedback form, which makes it very suitable for out-of-sample simulation and evaluation of policies.

As we shall see in Chapter 13, the same reasoning can be used to evaluate conditional value-at-risk in a normal case.

See Section 2.2.2.

See Section 9.4.

We are considering here only the _backward_ version of dynamic programming. For the shortest path and other deterministic combinatorial optimization problems, we could also apply a _forward_ equation (see, e.g., [9, Appendix D]). The natural formulation of stochastic problems is backward, but forward DP plays a major role in some approximate methods, as we shall see later.

For an extensive treatment of modeling issues in DP, see [37, Chapter 5].

We are cutting several corners here, as we assume that an optimal decision in feedback form can be found. Actually, this depends on technical conditions that we take for granted, referring the reader to the literature for a rigorous treatment.

In some engineering applications, the average cost, rather than the discounted one is of interest. However, discounted DP is the rule in finance and economics, so we will stick to it.

Of course, this is not an absolute rule, as DP is also widely used by operations researchers, too. On the contrary, stochastic programming with recourse is less common in economics.

We will see a simplified model in this vein in Section 10.7.3. For an extensive treatment of these kind of model see, e.g., [10].

26See, e.g., [37, pp. 29–31].

27The example is borrowed from [37, p. 107]; since it is problem 3.11 in that book, we will add the PW311 prefix to R functions to remark this fact.

28The treatment here follows [34], to which we refer for economic and financial application examples.

29In practice, the Jacobian matrix is not inverted, and a system of linear equations is solved; see, e.g., [7].

30See, e.g., [43].

31See [10, Chapter 7].

32See Section 14.2 for more details.

33The book by Warren Powell [37] is the recommended reading for a general presentation of ADP, and it also provided the basis for much of the following presentation.

34In the linear regression based approach that we discuss in Section 11.3.3, a batch procedure is used.

35This way of rewriting the optimality equations is associated with end-of-period value functions in [24, Chapter 12]. We prefer the terminology and the approach of [37].

36We shall consider pricing of American-style options by Monte Carlo methods in Section 11.3.3. In that case, the post-decision state variable is related to the continuation value of the option, with no increase in complexity.

37In this section we denote the state by _s_ , rather than by boldface s, as we assume that states can be enumerated.

38In some artificial intelligence or control engineering problems, we could even apply the strategy online to the actual system.

39We assume that the immediate cost is deterministic, but it could well be random; in this case, we should use the notation  ( _s_ _n_ , _a_ _n_ ), as we are dealing with an observation of a random variable.

40The whole Chapter 11 of [37] is devoted to selecting adaptive stepsizes.

# Chapter Eleven

# Option Pricing

Option pricing is one of the most important application fields for Monte Carlo methods, since option prices may be expressed as expected values under a suitable probability measure. Monte Carlo sampling is a quite flexible strategy that can be adopted when alternative numerical methods, like binomial/trinomial lattices or finite difference methods for solving partial differential equations, are inefficient or impossible to apply. For low-dimensional and path-independent options, Monte Carlo methods are not quite competitive with these alternatives, but for multidimensional or path-dependent options they are often the only viable computational strategy. Even more so when one has to deal with more complicated processes than an innocent geometric Brownian motion (GBM), or even a full-fledged term structure of interest rates. In the past, it was claimed that a significant limitation of Monte Carlo methods was their inability to cope with early exercise features. More recently, ideas from stochastic optimization and stochastic dynamic programming have been adapted to option pricing, paving the way to some Monte Carlo methods for American- and Bermudan-style options.

In this chapter we rely on the path generation methods of Chapter 6, where we have first seen how to price a vanilla call option by random sampling in Section 6.2.1. We will also make good use of the variance reduction strategies of Chapter 8, which were also illustrated in the simple case of a vanilla call option. Finally, when dealing with early exercise features, we use concepts of stochastic programming, like scenario trees, and approximate dynamic programming, which we introduced in Sections 10.5 and 10.8.

To make things a bit more interesting and realistic, we will consider simple examples of multidimensional and path-dependent options, as well as extremely basic interest rate derivatives. As a first example, we consider bidimensional spread options in Section 11.1, where we apply both naive Monte Carlo and stratified sampling. Then, in Section 11.2 we consider path-dependent derivatives, like barrier and arithmetic average Asian options. In these first two sections we consider European-style options, which lend themselves very well to sampling methods, since only the estimation of an expectation is required. The case of American- or Bermudan-style options is more challenging, as their early exercise features require the solution of an optimization problem. In Section 11.3 we show how certain optimization concepts can be adapted to option pricing. We introduce these fundamental concepts within the easy and comfortable BSM (Black–Scholes–Merton) world. This is a sensible choice from a pedagogical point of view, and sometimes allows us to compare the results obtained by sampling methods against exact prices. However, the BSM world does not capture all of the intricacies of financial markets. In Section 11.4 we illustrate an example of option pricing with stochastic volatility. Finally, we consider extremely simple interest rate derivatives Section 11.5, i.e., zero coupon bonds and vanilla options on them, in the cases of the Vasicek and Cox–Ingersoll–Ross models.

We only consider option pricing in this chapter, but related issues are dealt with in Chapters 12, where we estimate option greeks, and 13, where we measure the risk of a portfolio of options. Some background about risk-neutral pricing is provided in Section 3.9.

# 11.1 European-style multidimensional options in the BSM world

Let us consider what is arguably the simplest option depending on two risk factors, namely, a European-style _spread option_. This is an option written on two stocks, whose price dynamics under the risk-neutral measure are modeled by the following GBMs:

where the two Wiener processes have instantaneous correlation ρ. Thus, the driving process is a bidimensional geometric Brownian motion, for which we can generate sample paths as shown in Section 6.2.2. The option payoff is

i.e., there is a positive payoff when the spread between the two prices exceeds the threshold _K_. When _K_ = 0 the option is also called _exchange option_. To see where the name comes from, consider the value at maturity of a portfolio consisting of one share of _U_ and one option:

Hence, the option allows us to exchange one asset for the other at maturity. The exchange option is actually fairly simple to deal with, and indeed there is an analytical pricing formula which is a fairly straightforward generalization of the Black–Scholes–Merton formula:

where Φ(·) is the CDF of the standard normal distribution, and _V_ 0 = _V_ (0), _U_ 0 = _U_ (0). The reason why we get this type of formula is that the payoff has a homogeneous form, which allows to simplify the corresponding partial differential equation by considering the ratio _V/U_ of the two prices.1 R code implementing this formula is shown in Fig. 11.1.

**FIGURE 11.1** **Exact pricing for an exchange option.**

In this simple case, we have a benchmark against which to assess the quality of Monte Carlo estimates. In order to apply Monte Carlo, we need a path generation strategy, which is rather simple in this case. We recall from Section 6.2.2 that, to generate sample paths for two correlated Wiener processes, we need correlated standard normals. One possible approach relies on the Cholesky factor for the covariance matrix, which in this case is actually a correlation matrix:

It may be verified by straightforward matrix multiplication that Σ = **LL**  , where

Hence, to simulate tridimensional correlated Wiener processes, we must generate two independent standard normal variates _Z_ 1 and _Z_ 2 and use

(11.1)

(11.2)

to drive path generation. However, for the sake of generality, let us take advantage of the function simMVGBM that we developed in Chapter 6. The resulting R code is displayed in Fig. 11.2 for a spread option. The code is fairly simple, and the only thing worth mentioning is that the array storing the sample paths is tridimensional. To price the exchange option and check the results against the exact formula, we just need to select _K_ = 0:

**FIGURE 11.2** **Pricing a spread option by naive Monte Carlo.**

    > V0 <- 50; U0 <- 60; sigmaV <- 0.3
    > sigmaU <- 0.4; rho <- 0.7; T <- 5/12
    > r <- 0.05; K <- 0
    > EuExchange(V0,U0,sigmaV,sigmaU,rho,T,r)
    [1] 0.8633059
    > numRepl <- 200000
    > set.seed(55555)
    > EuSpreadNaiveMC(V0,U0,0,sigmaV,sigmaU,rho,T,r, numRepl)
    $estimate
    [1] 0.8536415
    $conf.int
    [1] 0.8427841 0.8644989

Now, can we improve the quality of the above estimate? In this bidimensional case, it is fairly easy to resort to stratification with respect to both random variables. However, for illustration purposes, let us pursue both single and double stratification. By single stratification, we mean stratifying with respect to one random variable, e.g., the standard normal  1 that drives the price of the first stock. Given Eqs. (11.1) and (11.2), we immediately see that the conditional distribution of  2 given  1 is normal with

However, we can obtain the same result by using a more general result about conditional distributions for the multivariate normal, which was given in Section 3.3.4.1 and can be applied in more general settings. The details are given in Example 3.7, and we urge the reader to check that we indeed obtain the same results. When stratifying a standard normal, it may be convenient to stratify the underlying uniform variable. As we have seen in Example 8.4, if we have _m_ strata, indexed by _j_ = 1,..., _m_ , and _N_ replications, we may generate _N_ _j_ = _N/m_ observations. Each stratum is an interval of the form

So, our procedure requires:

**1.** To generate uniform random variables _U_ _k_ , _k_ = 1,..., _N_ _j_ , for each stratum _j_ , and to transform them into

the observation _k_ within stratum _j_.

**2.** To invert the CDF of the standard normal and generate

**3.** To sample  2 _jk_ ~ N(ρ 1 _jk_ , 1 − ρ2).

**4.** To generate the corresponding lognormal variables and evaluate the option payoff.

All of this is accomplished by the R code in Fig. 11.3. Note that, in order to compute the overall estimate and the confidence interval, according to Eq. (8.13), we have to compute the sample mean and the sample variance within each stratum.

**FIGURE 11.3** **Pricing a spread option by single stratification.**

Double stratification requires to generate two uniform variables in a subsquare of the unit square:

The rest of the procedure is quite similar to the one for single stratification and can be implemented as shown in Fig. 11.4.

**FIGURE 11.4** **Pricing a spread option by double stratification.**

Now we may compare the accuracy of naive Monte Carlo against the two stratifications. We choose _K_ = 0 in order to use the exact formula for the exchange option, and we also note that, for a fair comparison, if we set the number of strata to 25 in simple stratification, we should set it to 5 in double stratification, since the parameter numStrat specifies the number of strata along each dimension:

    > V0 <- 50; U0 <- 45; sigmaV <- 0.2
    > sigmaU <- 0.3; rho <- 0.5; T <- 1
    > r <- 0.05; K <- 0
    > EuExchange(V0,U0,sigmaV,sigmaU,rho,T,r)
    [1] 7.887551
    > numRepl <- 10000
    > set.seed(55555)

    > EuSpreadNaiveMC(V0,U0,K,sigmaV,sigmaU,rho,T,r,numRepl)
    $estimate
    [1] 7.934347
    $conf.int
    [1] 7.771838 8.096855
    > numStrats <- 25
    > EuSpreadStratl(V0,U0,K,sigmaV,sigmaU,rho,T,r,
    numRepl,numStrats)
    $estimate
    [1] 7.899219
    $conf.int
    [1] 7.807015 7.991424
    > numStrats <- 5
    > EuSpreadStrat2(V0,U0,K,sigmaV,sigmaU,rho,T,r,
    numRepl,numStrats)
    $estimate
    [1] 7.898201
    $conf.int
    [1] 7.869630 7.926772

The effect of stratification is rather evident. Clearly, this is a fairly trivial example, and when dealing with several random variables, full stratification is not feasible. Nevertheless, in the case of multivariate normal distributions, the aforementioned results about conditioning can be exploited to stratify with respect to the most relevant dimensions. We also note that these results are the foundation of the Brownian bridge technique. In other cases, we may lack explicit results, but rejection sampling strategies have been proposed to apply the idea.

# 11.2 European-style path-dependent options in the BSM world

Pricing European-style, path-dependent options is, in principle, quite easy within a Monte Carlo framework. All we need to do is generate sample paths, under the risk-neutral measure, and collect the discounted payoff. However, when prices have to be sampled at many points in time, the process can be rather inefficient. Hence, we may try to improve the accuracy/performance trade-off by applying the variance reduction strategies of Chapter 8, or sample path generation by low-discrepancy sequences. In this section we consider two cases of path-dependent options:

**1.** An example of barrier option, namely, a down-and-out put option, to which we apply importance sampling

**2.** An arithmetic average Asian call option, to which we apply control variates and path generation by scrambled Sobol sequences

## 11.2.1 PRICING A BARRIER OPTION

In barrier options, a specific asset price _S_ _b_ is selected as a barrier value. During the life of the option, this barrier may be crossed or not, an event that may activate or cancel the option. In knock-out options, the contract is canceled if the barrier value is crossed at any time during the whole life; on the contrary, knock-in options are activated only if the barrier is crossed. The barrier _S_ _b_ may be above or below the initial asset price _S_ 0: if _S_ _b_ ≥ _S_ 0, we have an up-option; if _S_ _b_ < _S_ 0, we have a down-option. These features may be combined with the payoffs of call and put options to define an array of barrier options. For instance, a down-and-out put option is a put option that becomes void if the asset price falls below the barrier _S_ _b_ ; in this case we need _S_ _b_ < _S_ 0 and _S_ _b_ < _K_ for the option to make sense. The rationale behind such an option is that the risk for the option writer is reduced and, therefore, it is reasonable to expect that a down-and-out put option is cheaper than a vanilla one. From the option holder's viewpoint, this means that the potential payoff is reduced; however, if you are interested in options to manage risk, and not as a speculator, this also means that you may get cheaper insurance. By the same token, other combinations of features may be devised, like up-and-out call and down-and-in put options. Actually, we do not need the ability of pricing all of them, as precise relationships between their prices may be observed. Let us consider, for instance a down-and-in put option. Holding both a down-and-out and a down-and-in put option, with the same barrier level and strike, is equivalent to holding a vanilla put option. To see this, note that the barrier is either crossed or not; hence, one of the two options will be active at maturity. Thus, we have the following parity relationship:

(11.3)

where _P_ is the price of the vanilla put, and _P_ di and _P_ do are the prices for the down-and-in and the down-and-out options, respectively. Sometimes a rebate is paid to the option holder if the barrier is crossed and option is canceled; in such a case the parity relationship above is not correct.

In principle, the barrier might be monitored continuously; in practice, periodic monitoring is applied (e.g., the price could be checked each day at the close of trading). This may affect the price, as a lower monitoring frequency makes the detection of barrier crossing less likely. As a general rule for path-dependent options, continuous-time monitoring is more amenable to analytical treatment. Indeed, exact formulas are available for certain barrier options, such as a down-and-out put option with continuous-time monitoring. The following formula, where _S_ 0, _r_ , σ, _K_ , and _T_ have the usual meaning, gives the price of this option:2

where

and

R code implementing these formulas is given in Fig. 11.5. When monitoring occurs in discrete time, we should expect that the price for a down-and-out option is increased, since breaching the barrier is less likely. An approximate correction has been suggested,3 based on the idea of applying the analytical formula above with a correction on the barrier level:

**FIGURE 11.5** **Exact pricing for a down-and-out barrier option with continuous-time monitoring.**

(11.4)

where the term 0.5826 derives from the Riemann zeta function, δ _t_ is time elapsing between two consecutive monitoring time instants, and the sign ± depends on the option type. For a down-and-out put we should choose the minus sign, as the barrier level should be lowered to reflect the reduced likelihood of crossing the barrier. Notice that this barrier option is less expensive than the corresponding vanilla; the price of the barrier option tends to that of the vanilla option if _S_ _b_ → 0. By lowering the barrier according to Eq. (11.4) we increase the option price. The following snapshot, where we assume daily monitoring for an option maturing in 2 months consisting of 30 days each, illustrates the difference between a vanilla and a down-and-out put, and the effect of the correction:

    > S0 <- 50; K <- 45; Sb <- 40
    > r <- 0.05; sigma <- 0.4; T <- 2/12
    > numSteps <- 60; numDays <- 360
    > EuVanillaPut(S0,K,T,r,sigma)
    [1] 1.107108
    > EuDOPut(S0,K,T,r,sigma,Sb)
    [1] 0.1897568
    > Sbc <- Sb*exp(−0.5826*sigma*sqrt(1/numDays))
    > EuDOPut(S0,K,T,r,sigma,Sbc)
    [1] 0.2420278

If we want to price the option using Monte Carlo simulation, we must generate sample paths like

where the integer time subscript refers to time step of length δ _t_ , and _T_ = _M_ δ _t_. The naive Monte Carlo estimator is

where the indicator function _I_ is defined as

By relying on the function simvGBM to sample a GBM process, it is easy to write R code pricing the option by Monte Carlo, as shown in Fig. 11.6. Note the use of any to avoid a for loop to inspect the path. The function returns the point estimate and the confidence interval as usual, but also the number of sample paths on which the barrier is crossed, for reasons that we will clarify shortly. With the same parameters as before we obtain

**FIGURE 11.6** **Pricing a down-and-out barrier option by naive Monte Carlo.**

    > EuDOPut (S0,K,T,r,sigma,Sbc)
    [1] 0.2420278
    > numRepl <- 50000
    > set.seed(55555)
    > DownOutPutNaive(S0,K,T,r,sigma,Sb,numSteps,numRepl)
    $estimate
    [1] 0.2437642
    $conf.int
    [1] 0.2368905 0.2506378

    $numCrossed
    [1] 7835

We notice that the barrier has been crossed in 7835 replications out of 50,000. If we have to price a down-and-in put option, we may price the down-and-out and use the parity relationship of Eq. (11.3), or we may adapt the code, which is very easy to do.

The choice between these alternatives may by a matter of convenience. Let us suppose that the barrier is low and that crossing it is a rare event. If we consider pricing a down-and-in option, crossing the barrier is the "interesting" event, and if it is rare, then in most replications the payoff is just zero. Hence, we may consider using importance sampling to improve the performance. In this case, it is more convenient to price the down-and-in directly, and then use parity to price the down-and-out. One possible idea is changing the drift of the asset price in such a way that crossing the barrier is more likely.4 In this case we should decrease the drift, simulate GBM under this distorted measure, and correct the estimator by a likelihood ratio, just as we did in Section 8.6.1. Here, however, the likelihood ratio involves several random variables, and in order to figure it out we should step back and consider what we do to generate the sample path **S**. For each time step, we generate a normal variate _Z_ _j_ with expected value

(11.5)

and variance σ2 δ _t_. All of these variates are mutually independent, and the asset price path is generated by setting

Let **Z** be the vector of the normal variates, and let _f_ ( **Z** ) be its joint density. Now, let us assume that we lower the drift _r_ by an amount _b_ and generate normals with expected value

Note mat we interpret _b_ as an annualized drift component, with continuous compounding just like the risk-free rate _r_ , and we have to multiply it by the time step δ _t_. Let _g_ ( **Z** ) be the joint density for the normal variates generated with this modified expected value. Thanks to independence, both multivariate densities are just the product of _M_ univariate normal densities, and their ratio can be manipulated as follows:

where in the last equality we replace the generic drift μ according to Eq. (11.5). R code implementing importance sampling for a down-and-in put option is reported in Fig. 11.7. By setting the parameter b to 0, we essentially run naive Monte Carlo, so we may just use this function to assess the variance reduction in pricing the down-and-in put option:

**FIGURE 11.7** **Pricing a down-and-in put option by importance sampling.**

    > S0 <- 50; K <- 50; Sb <- 40; r <- 0.05; sigma <- 0.4;
    > T <- 5/12; numSteps <- 5*30; numDays <- 360

    > Sbc <- Sb*exp(−0.5826*sigma*sqrt (1/numDays))
    > numRepl <- 50000
    > set.seed(55555)
    > DownInPutIS(S0,K,T,r,sigma,Sb,numSteps,numRepl,0)
    $estimate
    [1] 3.90588
    $conf.int
    [1] 3.850881 3.960879
    $numCrossed
    [1] 18936
    > set.seed(55555)
    > DownInPutIS(S0,K,T,r,sigma,Sb,numSteps,numRepl,0.8)
    $estimate
    [1] 3.907309
    $conf.int
    [1] 3.882772 3.931846
    $numCrossed
    [1] 40303

It seems that we do gain something by applying importance sampling. However, apart from the intuition that rare and interesting events should be made more likely, it is not quite clear how the drift should be selected. In Section 13.4 we will consider a more detailed analysis to justify the selection of an importance sampling measure.

## 11.2.2 PRICING AN ARITHMETIC AVERAGE ASIAN OPTION

Barrier options are path-dependent, but to a weak degree, since the only relevant information is whether the barrier has been crossed or not. A stronger degree of path dependency is typical of Asian options, as the payoff depends on the average asset price over the option life. Different Asian options may be devised, depending on how the average is calculated. Sampling may be carried out in discrete or (in principle) continuous time, and the average itself may be arithmetic or geometric. The discrete arithmetic average is

where _t_ _j_ , _j_ = 1,..., _M_ , are the discrete sampling times. The corresponding geometric average is

If continuous sampling is adopted, we have

The chosen average _A_ , whatever it is, defines the option payoff by playing the role of either a rate or a strike. For instance, an average rate call has a payoff given by

whereas in an average strike call we have

By the same token, we may define an average rate put,

or an average strike put,

Early exercise features may also be defined in the contract. Hence, we may devise American- or Bermudan-style Asian options, even though the name does sound a bit weird.

Pricing an Asian option is, in general, a nontrivial task. The problem is not trivial not only because the payoff is more complicated than in a vanilla option, but also because when pricing the option after its inception, we must account for the average cumulated so far, and the fact that even if the sampling period δ _t_ is uniform, the time to the next price observation is different. Nevertheless, there are some analytical pricing formulas for Asian option. Consider the payoff of a discrete-time, geometric average Asian option:

Since the product of lognormal random variables is still lognormal, it is possible to find an analytical formula for this option, which looks like a modified Black–Scholes–Merton formula.5 In the formula, we are pricing the option at time _t_ , and _G_ _t_ is the current geometric average, if any observation has been already collected. If so, _t_ _m_ is the last time instant at which we observed the price of the underlying asset, _m_ is the corresponding index, _t_ _m_ +1 is the time instant of the next observation, and _t_ _M_ = _T_ is the option maturity:

(11.6)

where

The formula gets considerably simplified if we just consider the option price at its inception, i.e., at time _t_ = 0. In such a case _m_ = 0, and the resulting R implementation is illustrated in Fig. 11.8.

**FIGURE 11.8** **Exact pricing of a geometric average Asian option.**

In the case of an arithmetic average option, since the sum of lognormals is not lognormal, we have to use numerical methods. Pricing a European-style Asian option by Monte Carlo is quite easy in principle, since we have just to generate sample paths and take an average. In Fig. 11.9 we show the R code for a discrete, arithmetic average rate call option:

**FIGURE 11.9** **Naive Monte Carlo method for a European-style Asian option.**

    > S0 <- 50; K <- 50; r <- 0.05; sigma <- 0.4
    > T <- 1; numSamples <- 12; numRepl <- 50000
    > EuVanillaCall(S0,K,T,r,sigma)
    [1] 9.011476
    > set.seed(55555)
    > EuAsianNaive(S0,K,T,r,sigma,numSamples,numRepl)
    $estimate
    [1] 5.436376
    $conf.int
    [1] 5.359236 5.513516

It is interesting to note that the Asian option is cheaper than the corresponding vanilla, as there is less volatility in the average price than in the price at maturity. This explains why Asian options may provide cheaper insurance when multiple time instants are relevant for a hedger. In order to improve the quality of estimates, we may pursue variance reduction strategies, most notably control variates, or adopt path generation by low-discrepancy sequences.

### 11.2.2.1 Control variates for an arithmetic average Asian option

We know from Section 8.3 that, in order to apply this kind of variance reduction strategy, we need a suitable control variate, i.e., a random variable, with a known expected value, which is correlated with the option payoff. In the BSM framework, the following possibilities can be considered:

  Using the expected value of the arithmetic average, which is known in the case of geometric Brownian motion. Let us recall the formula

The expected value of the arithmetic average of stock prices is (under the risk-neutral measure):

This control variate does capture an essential feature of the option payoff, but not its nonlinearity.

  Using a vanilla call option, whose price is given by the BSM formula. This control variate, unlike the previous one, does capture nonlinearity in the option payoff, but it disregards the sample path.

  Using the geometric average Asian, whose price is given in Eq. (11.6). This control variate is definitely the most sophisticated and it embodies very deep knowledge, even though the kind of average is different.

The R function code in Fig. 11.10 is a generic function to implement control variates for the arithmetic average Asian option. The input argument sampleCV is actually a function that returns the value of the control variate for a given sample path; expCV is the expected value of the control variate. The script of Fig. 11.11 shows how to use the generic function to implement the three proposed strategies and produced the following (edited) output:

**FIGURE 11.10** **A generic function to price an Asian option by control variates.**

**FIGURE 11.11** **A script to compare the three control variates.**

    > EuAsianNaive(S0,K,T,r,0,sigma,numSamples,numRepl)
    $estimate
    [1] 5.436376
    $conf.int
    [1] 5.359236 5.513516
    > # CV is average price
    > EuAsianCV(S0,K,T,r,0,sigma,numSamples, numRepl,
    + SampleCV1,expCV1,numPilot)
    $estimate
    [1] 5.396385
    $conf.int
    [1] 5.364076 5.428693
    > # CV is vanilla call
    > EuAsianCV(S0,K,T,r,0,sigma,numSamples,numRepl,
    + SampleCV2,expCV2,numPilot)
    $estimate

    [1] 5.425655
    $conf.int
    [1] 5.383384 5.467925
    > # CV is geometric
    > EuAsianCV(S0,K,T,r,0,sigma,numSamples,numRepl,
    + SampleCV3,expCV3,numPilot) $estimate
    [1] 5.394813
    $conf.int
    [1] 5.390427 5.399199

We clearly see that the geometric average option does an excellent job, even though the two alternatives also reduce variance a bit with respect to naive Monte Carlo. This is easily explained by checking the correlation between the naive Monte Carlo estimator and the three control variates. This can be done by using the function of Fig. 11.12, which yields the following estimates of correlation:

**FIGURE 11.12** **Comparing the correlation of the three control variates.**

    > set.seed(55555)
    > EuAsianCompareCorr(S0,K,T,r,sigma,numSamples, 50000)
    Correlations between arithmetic Asian and average stock price 0.9179336
    vanilla call 0.8543479
    geometric Asian 0.9985351

Indeed, there is an extremely strong correlation between the arithmetic and geometric average payoffs, which explains why this control variates does such a good job.

### 11.2.2.2 Using low-discrepancy sequences

Armed with the function lowDiscrGBM to generate sample paths of a GBM process (see Fig. 9.9), it is easy to price the arithmetic average Asian option using low-discrepancy sequences. The code is given in Fig. 11.13, and we may check its performance on the same example that we have used with control variates. We may consider the value provided by the most powerful control variate, 5.394813, as a quite accurate estimate. In the following snapshot we compare the performance of Halton and Sobol sequences with different scramblings:

**FIGURE 11.13** **Using low-discrepancy sequences to price the arithmetic average Asian option.**

    > S0 <- 50; K <- 50; r <- 0.05; sigma <- 0.4
    > T <- 1; numSamples <- 12; numRepl <- 10000
    > EuAsianLOWD(S0,K,T,r,sigma, numSamples,
    + numRepl,"halton")
    [1] 5.341912
    > EuAsianLOWD(S0,K,T,r,sigma,numSamples,
    + numRepl,"sobol",0)
    [1] 5.413302
    > EuAsianLOWD(S0,K,T,r,sigma,numSamples,
    + numRepl,"sobol",1)
    [1] 5.402433
    > EuAsianLOWD(S0,K,T,r,sigma,numSamples,
    + numRepl,"sobol",2)
    [1] 5.399809

We see that, with a limited sample size, scrambled Sobol sequences provide us with a satisfactory result, whereas Halton sequences are a bit less effective. See the related discussion in Section 9.5.

# 11.3 Pricing options with early exercise features

In this section we illustrate the basic issues of pricing derivatives with early exercise features, such as American- and Bermudan-style options. Strictly speaking, since Monte Carlo methods rely on time discretization, we can only price the latter kind of options. Anyway, the price of a Bermudan-style option can be considered as useful approximation for the corresponding American-style option, and it can be refined by Richardson extrapolation. The price of this kind of derivative is related to the following optimization problem:6

(11.7)

where the function _f_ (·) is the option payoff, the expectation is taken under a risk-neutral measure  , and τ is a stopping time. Essentially, a stopping time is a random variable representing the time at which the option is exercised. This random variable is adapted to the available information, which is the sample path observed so far; in other words, the stopping time is associated with a nonanticipative exercise policy. The early exercise decision should be made by comparing the intrinsic value of the option, i.e., the payoff obtained by exercising the option immediately, and the value of continuation, i.e., the value of keeping the option alive. This has the effect of partitioning the state space in a continuation and an exercise region; see Fig. 10.8. Finding the optimal exercise policy is not trivial in general. When dealing with low-dimensional problems, alternative approaches can be applied, like binomial lattices and finite differences, which determine the optimal exercise policy by going backward in time. These methods are not feasible for high-dimensional and possibly path-dependent problems, which is the typical range of problems well-suited to Monte Carlo methods, in the case of European-style options. Indeed, years ago it was believed that Monte Carlo methods were not suitable to price options featuring early exercise opportunities. The situation has changed to a certain degree, thanks to the application of an array of approximation methods related to stochastic optimization, which we have introduced in Chapter 10:

**Simulation-based optimization.** One strategy is to specify a decision rule depending on a set of parameters, and solve a simulation-based optimization problem based on a set of sample paths. For instance, one might decide to exercise the option when it is enough in-the-money, i.e., when the intrinsic value exceeds a time-varying threshold. We may define the threshold by a set of parameters that can be optimized by the approaches of Sections 10.4. This approach may work fairly well in some cases, but it suffers from definite limitations when the shape of the early exercise region is complicated and disconnected.

**Scenario trees.** Another approach relies on a tree representation of uncertainty. As we have seen in Section 10.5, scenario trees are the standard way to represent uncertainty in stochastic programming with recourse, as they allow to deal with nonanticipative decisions. Trees are discretized stochastic processes, just like binomial lattices, but they do not recombine. Hence, the bad side of the coin is that their size tends to explode exponentially with respect to the number of time steps, but the upside is that they can deal with path-dependent and high-dimensional problems, provided that the set of early exercise opportunities is limited.

**Approximate dynamic programming.** Finally, we may rely on approximate dynamic programming methods, most notably regression-based approaches, to learn the value function; see Section 10.8. The value function is, in this case, the value of the continuation. With respect to full-fledged stochastic optimization approaches, in option pricing we do not face one of the three curses of dimensionality, since the decision variable is pretty simple: exercise or continue.

These three categories are not exhaustive, since there are alternatives, such as the stochastic mesh method. Furthermore, the boundaries among them need not be clear-cut. For instance, learning a policy by simulation can be interpreted as a form of approximate dynamic programming, focused on the policy itself rather than the value function determining the policy implicitly. Relying on the value function, in principle, allows for more flexibility, and in the following we will describe in detail only examples of the two latter approaches. However, first we have to investigate bias issues. Indeed, in Section 7.1.1 we have illustrated the pricing of a chooser option, which involves an optimal decision, and we discovered that a naive Monte Carlo approach may not work at all because of induced bias.

## 11.3.1 SOURCES OF BIAS IN PRICING OPTIONS WITH EARLY EXERCISE FEATURES

When pricing European-style options, we usually take for granted that our estimator is unbiased and concentrate on reducing its variance. When dealing with early exercise features, the picture is not so simple. To begin with, there are two clear sources of low bias:

  If we have to price an American-style option and discretize time to generate sample paths, we are restricting the exercise opportunities, which reduces the value of the option.

  When making early exercise decisions, we typically use suboptimal rules obtained by an approximate solution of an optimization problem. Hence, this also contributes to reduce the estimate of the option value.

On the other hand, however there is also a potential source of high bias. When making decisions based on a scenario tree, like the one illustrated in Fig. 11.14, we are actually using more information in the model than we actually have in the real world, as only a set of sampled scenarios is used, ruling out other realizations of the random variables involved. As a more formal clue, let us consider the problem of estimating

**FIGURE 11.14** **Scenario tree to price a Bermudan-style option.**

where α is a given number, playing the role of the intrinsic value, and _X_ is a random variable, whose expectation plays the role of the continuation value; we estimate the maximum on the basis of a sample _X_ 1,..., _X_ _m_ and the sample mean  . It is easy to see that estimator max {α,  } is biased high:7

In general, the different sources of bias interact in unclear ways, possibly resulting in an undefined situation. It is quite recommended to run the simulation in such a way that we have a clear bias, in one way or another. The scenario tree approach, that we illustrate in Section 11.3.2, yields both a low- and a high-biased estimator, which can be merged into a valid confidence interval. The value-function-based approach that we illustrate in Section 11.3.3 is a potentially more powerful approach, but its bias is unclear. What we can do is learn an approximate policy, using a set of replications, and then evaluate the option by an independent set of replications, which yields a low-biased estimator. There are other sophisticated strategies to obtain upper bounds, but they require a deeper background on stochastic processes.8

## 11.3.2 THE SCENARIO TREE APPROACH

Pricing European-style options can be accomplished by the generation of a bunch of independent scenarios, sharing only the initial state. When dealing with early exercise features, we need more care, as we cannot make any exercise decision on the basis of the future states visited along each individual sample path. As we have seen in Section 10.5, a scenario tree can be generated in order to enforce nonanticipativity of decisions in stochastic programming models. Broadie and Glasserman (BG) have proposed a pricing approach that relies on such a structure.9 The R function in Fig. 11.15 shows how a tree can be built for a trivial one-dimensional GBM process. The approach can be easily extended to multidimensional and more realistic models. A possible disadvantage of trees is that they suffer from an exponential explosion in their size when one has to branch at many time instants. Let _b_ denote the branching factor, i.e., the number of nodes branched for each node in the tree; for the sake of simplicity, we assume that _b_ is constant along the tree. If we branch _b_ = 100 nodes from each node at each time layer in the structure, we have 1 node at time _t_ = 0, 100 nodes at time _t_ = _T_ 1, 1002 nodes at time _t_ = _T_ 2, and so on. Indeed, the BG tree approach is best suited to derivatives with limited time epochs for early exercise. An advantage, however, is that the idea works very well also for high-dimensional derivatives. A further, and possibly even more significant advantage, is that the authors have proposed an approach to find both high- and low-biased estimators.

**FIGURE 11.15** **Sampling a scenario tree for option pricing.**

To understand the approach, it is important to choose a sensible notation.10 We generate the tree starting from the initial state _X_ 0. Then:

  From the initial node _X_ 0 we generate _b_ successor nodes _X_ 11,..., _X_ 1 _b_ at time _T_ 1.

  From each of these nodes _X_ 1 _i_ , _i_ = 1,..., _b_ , we branch _b_ successors at time _T_ 2, denoted by _X_ 2 _i_ 1,..., _X_ 2 _ib_.

  The process is repeated recursively to generate the successors at time _T_ 3, _X_ 3 _i_ 1 _i_ 21,..., _X_ 3 _i_ 1 _i_ 2 _b_ , up to option expiration at time _T_ _m_.

We observe that the subscript for each node is associated with its time period, whereas the superscript is associated with the path. Indeed, this approach can be applied to path-dependent options, unlike binomial lattices. Since we are sampling the tree by straightforward Monte Carlo, all branches have the same conditional probability, 1/ _b_ ; alternative scenario generation procedures, like Gaussian quadrature, would assign different probabilities to successor nodes. The early exercise decision when in state _x_ at time _t_ has to be made by a comparison between the intrinsic value _h_ _t_ ( _x_ ) and the discounted expectation of the value at future states:

(11.8)

where we assume for simplicity that the length of the time steps is always δ _t_. For a vanilla put, the intrinsic value is _h_ _t_ ( _x_ ) = max{ _K_ − _x_ , 0}, where the state _x_ is the current asset price _S_ _t_ and _K_ is the asset price; clearly, we do not care about negative intrinsic values. This is a recursive dynamic programming equation defining the option value at each state. At the terminal nodes of the tree, the exercise decision is trivial, since there is no value in waiting and the form of the option payoff is given. Going backward, we may estimate the option value by replacing the expectation by a sample mean, as successor nodes are equiprobable:

For the reasons that we have outlined in Section 11.3.1, this estimator is biased high for each node of the tree. In building this estimator, the same information is used twice: first, to decide when to exercise early, and then to estimate the value of this decision. In order come up with a corresponding low-biased estimator, we should break this link and use a sample to learn the rule, and then another independent sample to estimate its value. Since this requires sampling more and more nodes, the BG approach relies on a common trick in statistics. Given a node ( _i_ 1 _i_ 2 ··· _i_ _t_ ) at time _t_ , we set apart a single successor node ( _i_ 1 _i_ 2 ··· _i_ _t_ _k_ ) in turn, _k_ = 1,..., _b_ , and use the remaining _b_ − 1 nodes to find a low-biased estimate   of the option value at ( _i_ 1 _i_ 2 ··· _i_ _t_ ). The estimate is low-biased because, as we show below in detail, we use the other _b_ − 1 successor nodes to decide whether we exercise or not, but we assess the value of the decision on the successor node that was set apart. This results in _b_ estimates, which are then averaged to find the low-biased estimate  . Note the use of a lowercase _v_ to distinguish this estimate from the high-biased value estimate, denoted by uppercase _V_. In detail, given node _i_ 1 _i_ 2 ··· _i_ _t_ at time _t_ , we set

Then, all of the above quantities are averaged and yield the low-biased estimate

These low-biased estimates are initialized at the last time layer of the tree by setting them to option payoff. The R code of Fig. 11.16 shows how to generate lower and upper bounds based on a tree generated by the function sampleBGMTree.

**FIGURE 11.16** **Pricing using the Broadie–Glasserman tree.**

## 11.3.3 THE REGRESSION-BASED APPROACH

In Sections 10.7 and 10.8 we have illustrated different strategies for the numerical solution of the recursive equation of stochastic dynamic programming by approximating value functions. In particular, approximate dynamic programming (ADP) lends itself to pricing options with early exercise features by Monte Carlo sampling. In this section we describe an approach due to Longstaff and Schwartz,11 which should be interpreted as a way to approximate the value function of dynamic programming by a linear regression against a set of basis functions. Since we approximate the value function, what we expect is a suboptimal solution; furthermore, time is discretized. Hence, we should expect some low bias in our estimate of price. Actually, to find an accurate estimate with a well-defined bias, it is recommended to use the procedure that we outline below to learn first a set of approximate value functions, which embody a strategy for early exercise decisions; then, a more extensive set of replications should be used to assess the value of the decision rules. This is not needed when ADP is used in other optimization problems whose main output is the set of decision variables, defined by a suitable strategy; here, however, it is the value of the objective function that we are interested in.

For the sake of simplicity, we will just consider a vanilla American put option on a single non-dividend-paying stock. Clearly, the approach makes sense in more complex settings. As usual with Monte Carlo simulation, we generate sample paths ( _S_ 0, _S_ 1,..., _S_ _t_ ,..., _S_ _T_ ), where we use _t_ as a discrete time index, leaving the discretization step δ _t_ implicit. If we denote by _h_ _t_ ( _S_ _t_ ) the intrinsic value12 of the option at time _t_ , the dynamic programming recursion for the value function _V_ _t_ ( _S_ _t_ ) is

(11.9)

The expectation is taken under a risk-neutral measure   and is conditional on the current asset price _S_ _t_. In the case of a vanilla American put, we have _h_ _t_ ( _S_ _t_ ) = max{ _K_ − _S_ _t_ , 0}. Having to cope with continuous states is the only difficulty we have here, as time is discretized and the set of control actions is finite: Either you exercise immediately the option, or you cross your fingers and continue. Equation (11.9) is identical to Eq. (11.8). The difference between the regression-based and the scenario tree approaches lies in the scenario generation process: Here we do not branch a tree, which suffers from an exponential increase in complexity, but we generate independent sample paths, like we do with European-style options. Then, however, we must find a way to enforce nonanticipativity. It is important to realize that we _cannot_ make the exercise decision along individual sample paths; if we are at a given point of a sample path generated by Monte Carlo sampling, we cannot exploit knowledge of future prices along that path, as this would imply clairvoyance.13 What we can do is use our set of scenarios to build an approximation of the conditional expectation in Eq. (11.9), for some choice of basis functions  _k_ ( _S_ _t_ ), _k_ = 1,..., _K_. The simplest choice we can think of is regressing the conditional expectation against a basis of monomials:  1( _S_ ) = 1,  2( _S_ ) = _S_ ,  ( _S_ ) = _S_ 2, etc. In practice, orthogonal polynomials can also be used, as well as well as problem-specific basis functions capturing important features of each state. Note that we are using the same set of basis functions for each time instant, but the coefficients in the linear combination do depend on time:

Since the coefficients α _kt_ are not associated with specific sample paths, decisions are nonanticipative. The coefficients α _kt_ can be found by linear regression, going backward in time as customary with dynamic programming; note that the approximation is nonlinear in _S_ _t_ , but it is linear in terms of the coefficients.

In order to illustrate the method, we should start from the last time period. Assume that we have generated _N_ sample paths, and let us denote by _S_ _ti_ the price at time _t_ along sample path _i_ = 1,..., _N_. At option expiration, the value function is just the option payoff:

These values can be used, in a sense, as the _Y_ values in a linear regression, where the _X_ values are the prices at time _T_ − 1. More precisely, we estimate the following regression model for the continuation value:

where _e_ _i_ is the residual for each sample path. We may find the weights α _k,T_ −1 by the usual least-squares approach, minimizing the sum of squared residuals. Note that we are considering the discounted payoff, so that we may then compare it directly against the intrinsic value. Furthermore, we are using the state at time _T_ − 1 as the regressor. As we clarify later, we should interpret this as a _post-decision_ state, a fundamental concept in approximate dynamic programming that we have discussed in Section 10.8.2.

In the regression above, we have considered all of the generated sample paths. Actually, it is sometimes suggested to consider only the subset of sample paths for which we have a decision to make, i.e., the subset of sample paths in which the option is in the money at time _T_ − 1. In fact, if the option is not in the money, we have no reason to exercise; using only the sample paths for which the option is in the money is called the "moneyness" criterion and it may improve the performance of the overall approach.14 Denoting this subset by _I_ _T_ −1 and assuming _K_ = 3, we would have to solve the following least-squares problem:

(11.10)

The output of this optimization problem is a set of coefficients in the approximation of the continuation value. Note that the weights are linked to time periods, and not to sample paths. Using the same approximation for each sample path in _I_ _T_ −1, we may decide if we exercise or not. We stress again that this is how nonanticipativity is enforced in the regression-based approach, without the need to resort to scenario trees.

We should pause and illustrate what we have seen so far by a little numerical example. We will use the same example as the original reference, where the eight sample paths given in Table 11.1 are considered for a vanilla American put with strike price _K_ = 1.1. For each sample path, we also have a set of cash flows at expiration; cash flows are positive where the option is in the money. Cash flows are discounted back to time _t_ = 2 and used for the first linear regression. Assuming a risk free rate of 6% per period, the discount factor is _e_ −0.06 = 0.94176. The data for the regression are given in Table 11.2; _X_ corresponds to current underlying asset price and _Y_ corresponds to discounted cash flows in the future. We see that only the sample paths in which the option is in the money at time _t_ = 2 are used. The following approximation is obtained:

**Table 11.1** **Sample path and cash flows at option expiration for a vanilla American put**

**Table 11.2** **Regression data for time _t_ = 2**

Path | _Y_ | _X_
---|---|---
1 | .00 × .94176 | 1.08
2 | – | –
3 | .07 × .94176 | 1.07
4 | .18 × .94176 | 0.97
5 | – | –
6 | .20 × .94176 | 0.77
7 | .09 × .94176 | 0.84
8 | – | –

Now, based on this approximation, we may compare at time _t_ = 2 the intrinsic value and the continuation value. This is carried out in Table 11.3. Given the exercise decisions, we update the cash flow matrix. Note that the exercise decision does not exploit knowledge of the future. Consider, for instance, sample path 4: we exercise, earning $0.13; along that sample path, later we would regret our decision, because we could have earned $0.18 at time _t_ = 3. This is what nonanticipativity is all about. We should also note that on some paths we exercise at time _t_ = 2, and this is reflected by the updated cash flow matrix in the table.

**Table 11.3** **Comparing intrinsic and continuation value at time _t_ = 2, and resulting cash flow matrix**

The process is repeated going backward in time. To carry out the regression, we must consider the cash flows on each path, resulting from the early exercise decisions. Assume that we are at time step _t_ , and consider path _i_. For each sample path _i_ , there is an exercise time _t_ _e_ *, which we might set conventionally to _T_ \+ 1 if the option will never be exercised in the future. Then the regression problem (11.10) should be rewritten, for the generic time period _t_ , as:

(11.11)

Since there can be at most one exercise time for each path, it may be the case that after comparing the intrinsic value with the continuation value on a path, the exercise time _t_ * _e_ is reset to a previous period. Stepping back to time _t_ = 1, we have the regression data of Table 11.4. The discount factor _e_ −2·0.06 = 0.88692 is applied on paths 1 and 8. Since the cash flow there is zero, the discount factor is irrelevant, but we prefer using this to point out that we are discounting cash flows from time period _t_ = 3; if we had a positive cash flow at _t_ = 3 and zero cash flow at _t_ = 2, this is the discount factor we should use. Least squares yield the approximation:

**Table 11.4** **Regression data for time _t_ = 1**

This approximation may seem unreasonable, as we expect smaller payoffs for larger asset prices, yet the highest power of the polynomial has a positive coefficient here. It can be verified that, for the range of _X_ values we are considering, the function is in fact decreasing. Based on this approximation of the continuation value, we obtain the exercise decisions illustrated in Table 11.5. Discounting all cash flows back to time _t_ = 0 and averaging over the eight sample paths, we get an estimate of the continuation value of $0.1144, which is larger than the intrinsic value $0.1; hence, the option should not be exercised immediately.

**Table 11.5** **Comparing intrinsic and continuation value at time _t_ = 1, and resulting cash flow matrix**

Figure 11.17 shows R code to implement this regression-based approach in general. The function is modular in that it takes scenarios generated outside by another function. Figure 11.18 shows a script to check the toy example we have considered with the generic function. We insist again on the fact that a safer approach would first learn the exercise policy, on the basis of a limited number of replications. Then, using a brand new and possibly much larger set of replications, we should find the low-biased estimator.

**FIGURE 11.17** **Regression-based American-style option pricing.**

**FIGURE 11.18** **A script to check the example of Longstaff and Schwartz.**

Finally, it is important to interpret the regression-based approach within the framework of approximate dynamic programming. Unlike other numerical approaches for stochastic dynamic programming, which we have considered in Section 10.7, we did not consider the expectation of the value functions. The expectation is directly built into the conditional expectation, as linear regression yields a function of _S_ _t_ :

In this approach, there is no need to tackle the expectation of _V_ _t_ +1( _S_ _t_ +1) by, say, Gaussian quadrature. As we have seen in Section 10.8, this corresponds to the concept of value function for the post-decision state variable. In this case the post-decision state variable is just the current asset price, which in practice is the same as the pre-decision state variable, but it should be interpreted as the state _after the decision to continue._ The point is that our decision is whether we exercise or not, and this does not influence the state variable, unlike other dynamic programming models. Indeed, linear regression yields the value of the continuation, which is then compared against the value of immediate exercise to find the option value.

We close the section by noting that in some ADP approaches, a recursive approach to linear regression is adopted, whereby the coefficients are incrementally updated by considering one sample path at time. Here we are using a batch regression, whereby all sample paths are generated at once and we run a linear regression per time step on all of them. The relative merits of the two strategies may depend on the specific problem at hand.

# 11.4 A look outside the BSM world: Equity options under the Heston model

Option pricing within the BSM world is only a first step to grasp the remarkable potential of Monte Carlo methods for financial engineering. We have considered some more general models than geometric Brownian motion in Chapter 3, such as the Heston stochastic volatility model and mean-reverting square-root diffusions for short rates. Path generation mechanisms for such processes have been introduced in Chapter 6. All of these tools may be integrated to relax the restrictive assumptions of the BSM formula and price options in a more general and, hopefully, more realistic setting. We should not forget that in doing so we may lose market completeness. Hence, as we have pointed out in Section 3.9.3, we may have to face a nontrivial model calibration task. In this and the next section we do not consider calibration, but we just illustrate the flexibility of Monte Carlo methods, which indeed are often (not always) the only viable pricing strategy.

In Section 3.7.5 we have considered the Heston stochastic volatility model, reported below for convenience, under the risk-neutral measure:

The model integrates a GBM with nonconstant volatility and a square-root diffusion modeling squared volatility, where   is a long-term value, α measures the speed of reversion to the mean, and ξ is the volatility of the square-root diffusion. Different assumptions can be made about the instantaneous correlation ρ of the two driving Wiener processes _W_ _t_ 1 and _W_ _t_ 2. Suppose that we are interested in pricing a vanilla call option. Unlike the case of GBM, we cannot sample the price _S_ _T_ at maturity directly, but we have to go through a whole sample path generation to keep the discretization error under control. A straightforward approach to discretize the above equations is the Euler scheme

where  _t_ 1 and  _t_ 2 are standard normals with correlation ρ. Since the Euler discretization does not guarantee non-negativity, we may heuristically patch the above expressions by taking the maximum between the result and 0, as shown in the code of Fig. 11.19. Here is one sample run of the function, and the reader is invited to try and play with it, in order to see the effect of the different parameters:

**FIGURE 11.19** **Pricing a vanilla call option under the Heston model.**

    > CallStochVol(S0,K,T,r,V0,alpha,Vbar,xi,rho,numRepl,numSteps)
    $estimate
    [1] 5.754051
    $conf.int
    [1] 5.566677 5.941425

Actually, the Heston model allows for some semianalytical solutions in simple cases, but the Monte Carlo code can be adapted to more complicated options. As we mentioned, we have to generate a whole sample path, with a corresponding increase in computational effort with respect to the GBM case. Furthermore, since we are using a plain Euler scheme, the time step δ _t_ must be small enough to ensure accuracy. However, this additional cost may be not quite relevant if we want to price more exotic, possibly path-dependent, options. Asian options feature a payoff depending on the average underlying asset price. Another class of options, called lookback options, feature a payoff depending on the maximum or the minimum observed price. Consider, for instance, a lookback call option, which is characterized by the payoff

where _t_ 1, _t_ 2,..., _t_ _n_ ≡ _T_ is the collection of discrete time instants at which the price is monitored. It is very easy to adapt the code of Fig. 11.19 to price this option.

# 11.5 Pricing interest rate derivatives

In the first sections of this chapter we have used the risk-neutral valuation principle to price options on stocks, under the assumption of a constant risk-free rate. Using this principle, the price at time _t_ of a path-independent, European-style option maturing in _T_ can be written as

(11.12)

where _f_ ( _S_ _T_ ) is the payoff at maturity, depending on the stock price _S_ _T_ ,   is the pricing measure, and the expectation is conditional on the price _S_ _t_ of the underlying asset in _t_. If we step into the domain of interest rate derivatives, there are substantial complications.

  The first issue is that modeling interest rates is an intrinsically more difficult endeavor than modeling stock prices, since we should model a whole term structure, rather than a single stochastic process. Here we will skip this difficulty by adopting single-factor models depending only on the short-term (instantaneous) rate, like the Vasicek and the Cox–Ingersoll–Ross (CIR) models that we have mentioned in Chapter 3. These models rely on stochastic differential equations that are slightly more complicated than geometric Brownian motion, but we have seen in Section 6.3 how to generate sample paths for them.

  Since we are dealing with stochastic interest rates, we cannot take the discount factor outside the expectation, as is done in Eq. (11.12). Furthermore, since we are dealing with a short rate, we should account for the whole path of the process _r_ _t_ , even if the option payoff is path-independent. The risk-neutral pricing approach leads here to a formula like

(11.13)

  Another quite tricky point is that, as we have pointed out in Section 3.9.3, the market here is incomplete. The implication is that there is not a unique measure   for pricing, but several ones. This means that we have to calibrate the short-rate model against the prices of quoted derivatives. In this chapter, we assume that such calibration has already been done, and when we use a model like

we assume that the parameters define the risk-neutral measure for pricing purposes.

In order to calibrate an interest rate model, we need quoted prices of interest rate derivatives, and the simplest such assets are, in fact, bonds. This may sound odd, as bond pricing given a term structure of interest rates is an easy task, but the point here is that we want to _estimate_ a risk-neutral model for pricing purposes, accounting for the stochastic character of the interest rates. In fact, bonds are interest rate derivatives in the sense that they are assets whose value depends on that underlying risk factor. Since a coupon-bearing bond can be priced as a portfolio of zero-coupon bonds, the first interest rate derivative that we should price is a zero-coupon bond. In such a case, the option payoff is just 1, if we assume a bond with $1 face value. Hence, the price at time _t_ of a zero-coupon bond maturing at time _T_ , when the short rate in _t_ is _r_ _t_ , can be expressed as

(11.14)

This expression is fairly simple, and indeed there are analytical formulas to price zero-coupon bonds under both Vasicek and CIR models. From a pedagogical viewpoint, though, it is quite useful to find those prices by Monte Carlo simulation in order to build skills that can be put to good use when dealing with more complex (and interesting) cases. From Eq. (11.14) we immediately see that to use Monte Carlo in this setting, we should generate sample paths of the short rate process and then approximate the integral by a sum:

where δ _t_ is the discretization step, and _r_ _t_ _k_ is the short rate at the discretized time instants _t_ _k_ , _k_ = 0,..., _m_ , such that _t_ 0 = _t_ and _t_ _m_ = _T_. Note that in the sum we multiply by δ _t_ the rate observed at the _beginning_ of each time slice.

The next step in the learning process is to price a relatively simple option, namely, a call option on a zero-coupon bond. The risk-neutral principle must be applied exactly in the same way, but we have to realize that there are two maturities:

  The option maturity, _T_ _O_ , i.e., the time at which the option can be exercised, by purchasing a zero-coupon bond at the strike price _K._

  The bond maturity, _T_ _B_ ; clearly, for the option to make sense, we must have _T_ _O_ < _T_ _B_.

Therefore, the option payoff at _T_ _O_ is

(11.15)

## 11.5.1 PRICING BONDS AND BOND OPTIONS UNDER THE VASICEK MODEL

The price at time _t_ of a zero-coupon bond maturing in _T_ , with face value $1, under the Vasicek short-rate model

can be expressed as follows:

where

(11.16)

(11.17)

We insist once again that the parameters of the Vasicek model are _not_ the parameters of the real world, which could be estimated by analyzing time series of short rates, but those in the risk-neutral world. These formulas can be proved by solving the partial differential equation that we have given in Eq. (3.125),15 and are easily implemented R, as shown in Fig. 11.20.

**FIGURE 11.20** **Pricing a zero-coupon bond under the Vasicek model.**

To price the bond using Monte Carlo, we rely on the function simVasicek, listed in Fig. 6.12, to generate sample paths. In Fig. 11.20, we also show code to price the zero-coupon bond using random sampling, and a snapshot to compare the results. Running the code yields the following output:

    > ZCBVasicek(r0,rbar,gamma,sigma,T,F)
    [1] 97.26989
    > ZCBVasicekMC(r0,rbar,gamma,sigma,T,mumRepl,numSteps,F)
    $estimate
    [1] 97.23904
    $conf.int
    [1] 97.14941 97.32867

We see that the results are fairly reasonable. The price at time 0 of a European-style call option maturing at time _T_ _O_ on a zero-coupon bond maturing in _T_ _B_ is, under the Vasicek model:

(11.18)

where _r_ 0 is the current value of the short rate, Φ(·) is the CDF of the standard normal distribution, and

(11.19)

(11.20)

(11.21)

To interpret this formula, it is quite useful to note its deep similarity with the Black–Scholes–Merton price of a vanilla call option on a stock, given in Eq. (3.108), and note the following:

  In Eq. (11.18), _Z_ (0, _r_ 0; _T_ _B_ ) should be interpreted as the price of the underlying asset, i.e., the bond maturing in _T_ _B_ , whereas _Z_ (0, _r_ 0; _T_ _O_ ) should be interpreted as a discount factor from the option maturity _T_ 0 to time 0.

  The terms _d_ 1 and _d_ 2 in Eqs. (11.19) and (11.20) look much like the similar terms in the BSM formula.

  The term _S_ ( _T_ _O_ ), where _B_ ( _T_ _O_ , _T_ _B_ ) is just the function given in Eq. (11.16), plays the role of a volatility.

It is also useful to note that short-rates under the Vasicek model, which relies on an Ornstein–Uhlenbeck process, are normally distributed, and that the price of a zero-coupon bond is an exponential of these rates. Hence, the bond price is lognormally distributed, just like stock prices under the BSM model, and this is the essential reason behind the observed similarity.

In Fig. 11.21 we give R code to evaluate the exact call price, and a function to estimate it by straightforward Monte Carlo. We are cheating a bit there, as you may notice that to evaluate the payoff we are using the exact bond price. Otherwise, we should simulate sample paths of _r_ _t_ from _t_ = 0 to _t_ = _T_ _O_ to evaluate the discount factor, and then up to _r_ = _T_ _B_ to evaluate the bond price. We use a little short-cut to avoid this nightmare, but the reader will immediately appreciate the complexity of pricing a really difficult interest rate derivative.16

**FIGURE 11.21** **Pricing a bond option under the Vasicek model.**

Running the code, we again find sensible results:

    > ZCBCallVasicek(r0,rbar,gamma,sigma,TB,K,TO,F)
    [1] 3.190405
    > ZCBCallVasicekMC(r0,rbar,gamma,sigma,TB,K,TO,
    numRepl,numSteps,F)
    $estimate
    [1] 3.192955
    $conf.int
    [1] 3.140366 3.245544

## 11.5.2 PRICING A ZERO-COUPON BOND UNDER THE CIR MODEL

We have noted in Chapter 6 that, since the CIR model relies on a square-root diffusion,

the short rates are no longer normal. Nevertheless, a zero-coupon bond can be priced analytically under the CIR model, even though the formulas are more complicated than those for the Vasicek model:

where

In Fig. 11.22 we show R code that implements these formulas, as well as a Monte Carlo pricer relying on the function simCIR of Fig. 6.14 to generate sample paths of the square-root diffusion. As usual, we also verify the sensibility of the results:

**FIGURE 11.22** **Pricing a zero-coupon bond under the CIR model.**

    > ZCBCIR(r0,rbar,gamma,alpha,T,F)
    [1] 97.2596
    > ZCBCIRMC(r0,rbar,gamma,alpha,T,numRepl,numSteps,F)
    $estimate
    [1] 97.26934
    $conf.int
    [1] 97.25589 97.28279

# For further reading

  A friendly introduction to numerical methods for option pricing can be found in [4].

  The most comprehensive reference on Monte Carlo methods for option pricing is [9].

  Other books that include interesting discussions of specific points are [11] and [12].

  An early survey can be found in [2], which was later updated in [3].

  An interesting collection of papers is proposed in [8].

  The background on the interest rate derivatives that we have discussed in Section 11.5 can be found, e.g., in [16].

# References

1 T. Björk. _Arbitrage Theory in Continuous Time_ (2nd ed.). Oxford University Press, Oxford, 2004.

2 P. Boyle. Options: A Monte Carlo approach. _Journal of Financial Economics_ , 4:323–338, 1977.

3 P. Boyle, M. Broadie, and P. Glasserman. Monte Carlo methods for security pricing. _Journal of Economics Dynamics and Control_ , 21:1267–1321, 1997.

4 P. Brandimarte. _Numerical Methods in Finance and Economics: A MATLAB-Based Introduction_ (2nd ed.). Wiley, Hoboken, NJ, 2006.

5 M. Broadie and P. Glasserman. Pricing American-style securities using simulation. _Journal of Economic Dynamics and Control_ , 21:1323–1352, 1997.

6 M. Broadie, P. Glasserman, and S.G. Kou. A continuity correction for discrete barrier options. _Mathematical Finance_ , 7:325–349, 1997.

7 L. Clewlow and C. Strickland. _Implementing Derivatives Models._ Wiley, Chichester, West Sussex, England, 1998.

8 B. Dupire, editor. _Monte Carlo. Methodologies and Applications for Pricing and Risk Management._ Risk Books, London, 1998.

9 P. Glasserman. _Monte Carlo Methods in Financial Engineering._ Springer, New York, 2004.

10 M.B. Haugh and L. Kogan. Pricing American options: A duality approach. _Operations Research_ , 52:258–270, 2004.

11 P. Jaeckel. _Monte Carlo Methods in Finance._ Wiley, Chichester, 2002.

12 R. Korn, E. Korn, and G. Kroisandt. _Monte Carlo Methods and Models in Finance and Insurance._ CRC Press, Boca Raton, FL, 2010.

13 Y.K. Kwok. _Mathematical Models of Financial Derivatives._ Springer, Berlin, 1998.

14 F.A. Longstaff and E.S. Schwartz. Valuing American options by simulation: A simple least-squares approach. _Review of Financial Studies_ , 14:113–147, 2001.

15 S.M. Ross and J.G. Shanthikumar. Monotonicity in volatility and efficient simulation. _Probability in the Engineering and Informational Sciences_ , 14:317–326, 2000.

16 P. Veronesi. _Fixed Income Securities: Valuation, Risk, and Risk Management._ Wiley, Hoboken, NJ, 2010.

17 P. Wilmott. _Quantitative Finance (vols. I and II)._ Wiley, Chichester, West Sussex, England, 2000.

1See, e.g., [1, pp. 184–188] for a proof.

2See, e.g., [17, pp. 250–251].

3See [6] or [13, p. 266].

4See [15].

5See [7, pp. 118–119].

6See Section 10.2.5.

7This is a consequence of Jensen's inequality: E[ _f_ ( **X** )] ≥ _f_ (E[ **X** ]) for a random vector **X** and a convex function _f_ (·). See [9, Chapter 8] for an excellent discussion of bias issues in pricing options with early exercise.

8See, e.g., [10].

9See [5].

10The notation is based on [9, pp. 432–434].

11See [14].

12Again, we identify the intrinsic value with the value of immediate exercise.

13This point was also emphasized when we discussed the role of nonanticipativity in multistage stochastic programming.

14The criterion was suggested in the original paper by Longstaff and Schwartz, and we follow the idea, even though some authors have argued against its use.

15See, e.g., [16].

16Suitable changes of measure are in fact used to price interest rate derivatives with Monte Carlo, but this requires deeper concepts that are beyond the scope of this book.

# Chapter Twelve

# Sensitivity Estimation

The aim of most Monte Carlo simulations is to estimate the expected value of a function of several random variables. On the one hand, this function depends on the sample path, which we may associate with a random event ω; informally, we may think of ω as a sequence of (pseudo)random numbers. On the other hand, the function depends on some relevant parameters as well. If, for the sake of simplicity, we focus on a single parameter α, we may denote the function implemented by the simulation program as _f_ (α, ω). When taking the expected value, we obtain another function

(12.1)

depending on α alone. Monte Carlo is a way to find an estimate _ _(α). In many practical settings, we are also interested in the sensitivity of _g_ (·) with respect to its argument α. Formally, we would like to find

(12.2)

The standard example in financial engineering is the computation of the option greeks, which measure the sensitivity of option prices with respect to parameters like the current price of the underlying asset, volatility, etc. More generally, we might need the sensitivity with respect to parameters that are the decision variables of a stochastic optimization problem. If the objective function depends on a vector α    _p_ , certain stochastic optimization algorithms need an estimate of the gradient

In other cases, we might also want to find the Hessian matrix of second-order derivatives. Option gamma, for instance, is the second-order partial derivative of option price with respect to the underlying asset price. To keep it simple, we will consider first the case of the first-order derivatives, referring to option greeks of a plain vanilla, European-style option within the Black–Scholes–Merton world. There, we may also afford the luxury to compare our estimates with the exact result to gain confidence with methods and to see the relevant issues. To make things a tad more interesting, later we also deal with an Asian option.1

It is important to realize that even such a simple problem may hide some potential trouble. In fact, we do not really know _g_ (α), and the only thing we are able to do is to sample realizations of the random variable _Y_ (α) = _f_ (α, ω). Then, we might consider the following "limit of finite differences":

(12.3)

and estimate the expected value

(12.4)

by its sample mean. However, this raises a few issues:

**1.** In the limit of Eq. (12.3) we have differences of _random variables._ Which kind of limit should we consider? What is the role of ω?

**2.** What we want is actually the derivative of an expectation, but in Eq. (12.4) we are taking expectation of a "derivative." However, to begin with, this is not quite a standard derivative, since it involves random variables. Even if we disregard this issue, how can we be sure that the expectation and the limit commute?

To start appreciating the involved issues and gain a deeper understanding, we first consider a simple approach based on finite differences in Section 12.1. Then, we move on to more refined approaches that aim at estimating sensitivities using information provided by the same sample path that we use to estimate the expected value of the function itself, in our case the option price. In Section 12.2 we consider an approach based on pathwise derivatives, where the parameter is regarded as an attribute of the function itself. On the contrary, in the likelihood ratio (or score) method, which is dealt with in Section 12.3, the parameter is regarded as an attribute of the probability density, i.e., it is a property of the random variables, rather than the function.

We know from Chapter 11 that the price of a vanilla option is a function,

(12.5)

depending on current underlying asset price _S_ 0, strike price _K_ , time to maturity _T_ , risk-free interest rate _r_ , and volatility σ. The aim of this chapter is not to give a full picture of Monte Carlo estimation procedures for the full set of option greeks. We just want to use simple examples to introduce the aforementioned approaches. Hence, we will only consider the essential greek, i.e., the option delta:

This will suffice to appreciate the essential issues, which apply, e.g., to the estimation of vega as well. For an alternative approach to sensitivity estimation, we refer the reader to Section 10.4.2, where we deal with metamodeling approaches to simulation-based optimization; these strategies, too, can be used to estimate the gradient and the Hessian matrix of a function depending on random variables.

# 12.1 Estimating option greeks by finite differences

Using the Black–Scholes–Merton formula, it is not too difficult to prove that the option delta for a European-style vanilla call is

(12.6)

where Φ(·) is the CDF of a standard normal random variable. This formula is easily implemented as shown in Fig. 12.1. Since the task may not be this simple with exotic options, we are interested in Monte Carlo estimates of greeks. Estimating the option delta by random sampling should be a straightforward business. After all, the derivative with respect to _S_ 0 is just the limit of an increment ratio, for suitably small _h_ :

**FIGURE 12.1** **Exact Δ for a vanilla BSM option.**

where in _g_ (·) we only point out the dependence on the initial underlying asset price. If we sample the discounted option payoff starting from _S_ 0 \+ _h_ and _S_ 0, and take the sample mean of this increment ratio, the job should be done. This simple idea is reflected in the code of Fig. 12.2. The code also returns a confidence interval, and the figure includes a script to set values and check how well the function performs against the exact result:

**FIGURE 12.2** **Straightforward approach to estimate a vanilla call delta.**

    > DeltaOK
    [1] 0.583839
    > OutDeltaMCNaive
    $estimate
    [1] 0.8262371

    $conf.int
    [1] 0.4567431 1.1957310

To put it mildly, the result is extremely disappointing. The point estimate is quite far from the exact value, which is something that in practical cases we would not know. What we would know, however, is that such a wide confidence interval is far from acceptable. It is so large that it includes nonsensical values, as option delta cannot be larger than 1 for this kind of option: From a financial viewpoint, Δ is the number of stock shares that a call option writer should hold in order to hedge option risk, according to the BSM model; clearly, there is no reason to hedge by using more than one stock share per option. From a mathematical viewpoint, Eq. (12.6) shows that Δ is given as a probability, which cannot be larger than 1. Therefore, something is definitely wrong.

A first issue is related to a (deliberately) missing point in Eq. (12.3). There, we did not clarify the role of the event ω, i.e., the underlying sequence of random numbers. To be more precise, what we wrote is something like

where _f_ ( _S_ 0, ω) is the discounted option payoff starting from _S_ 0 and using the sample path ω, which in this case boils down to a single observation of a standard normal variable used to generate a lognormal price. The trouble is that we have used _independent_ sample paths in the evaluation. From a theoretical point of view, this does not look quite right when compared against the definitions in Eqs. (12.1) and (12.2). There, we have a function _f_ (·, ·) depending on two arguments, α, which we perturb, and ω, the sample path, which we use in the expectation. Here, however, we have two such sample paths. From a practical point of view, we know from Chapter 8 that when we consider differences of random variables, variance can be reduced by common random numbers. Indeed, we should use the _same_ sequence of random variables in both terms of the difference. Otherwise, variance (noise) will overwhelm the actual signal in the difference, especially when _h_ is small.

There is also another issue, related to finite difference methods in numerical analysis, which are often used to solve partial differential equations,2 including those involved in option pricing. People in the field know that sometimes an alternative finite difference should be adopted to improve accuracy. The above finite difference is called a _forward difference._ The following one is called a _central_ difference,

and it is used in the code of Fig. 12.3, where we save the sequence of standard normals in order to use them twice, for both payoffs. Now, the results are definitely more reasonable:

**FIGURE 12.3** **Improving the estimate of the option Δ by common random numbers and central differences.**

    > set.seed(55555)
    > OutDeltaMC <- VanillaDeltaMC(S0,K,T,r,sigma,dS,numRepl)
    > DeltaOK
    [1] 0.583839
    > OutDeltaMC
    $estimate
    [1] 0.5810456
    $conf.int
    [1] 0.5747994 0.5872918

It seems that by a simple trick we have solved all of our problems. However, we should dig a bit deeper to sharpen our understanding of the use of finite differences. In particular, we should investigate the potential bias of this kind of estimator. To set the problem within the correct framework, let us consider the forward difference estimator

(12.7)

and the central difference estimator

(12.8)

Here we point out the dependence on a common stream of random numbers explicitly, as we have seen that this does play a role in terms of variance. However, this does not play any role in terms of expectation. If we take the expectation of the forward difference estimator (12.7), we find

(12.9)

Assuming that _g_ (·) is suitably differentiable, we may write the Taylor expansion:

(12.10)

Rearranging the equation, we find the bias in the estimator:

(12.11)

Let us compare this with the central difference estimator. In this case, it is also useful to consider the Taylor expansion:

(12.12)

Subtracting Eq. (12.12) from Eq. (12.10) and rearranging, we find

(12.13)

since the second-order derivatives now cancel each other. This bias is of lower order with respect to Eq. (12.11). We might also consider a backward difference estimator, but Fig. 12.4 shows intuitively why the central difference estimator might be the best one.

**FIGURE 12.4** **Comparing finite differences.**

As a more interesting example, let us consider the delta of an arithmetic average, European-style, Asian option, with payoff:

(12.14)

For the sake of simplicity, we consider only the option at its inception, i.e., when it is written, and we also assume that the time intervals at which the underlying asset price is sampled along the option's life to evaluate the average price are of uniform length. The code in Fig. 12.5 is based on central finite differences and common random numbers. In this case, we cannot compare what we find numerically against an exact, analytical result. One possibility for a check is to use a simple first-order metamodel, which actually boils down to pricing the option in the neighborhood of _S_ 0, using a high-quality approach, and then fit a linear regression model. The slope of the regression line provides us with another estimate of delta. In Chapter 11 we have seen that an accurate estimate of the price of the arithmetic average Asian option is obtained by using the corresponding geometric average option as a control variate. The script of Fig. 12.6 yields the following output, which is somewhat reassuring:

**FIGURE 12.5** **Code to estimate Δ of an Asian option by central differences.**

**FIGURE 12.6** **Checking the estimate Δ of an Asian option against a regression approach.**

    > delta
    [1] 0.5742706
    > OutAsianDeltaCentral
    $estimate
    [1] 0.5727809
    $conf.int
    [1] 0.5675191 0.5780428

What is less pleasing here is that we need a significant computational effort, since we have to run simulations starting from ( _S_ 0 − _h_ ), _S_ 0, and ( _S_ 0 \+ _h_ ), if we want to get both the option price and the delta by central differences. It would be nice to run just one set of replications for pricing, and then squeeze out additional information to find the greeks. Indeed, this can be done by the two approaches we describe in the following two sections:

**1.** The pathwise derivative approach, based on perturbation analysis

**2.** The likelihood ratio approach, also known as score function approach

# 12.2 Estimating option greeks by pathwise derivatives

Let us consider again the function _f_ (α, ω); if we fix ω, i.e., if we consider a given sample path, and the function is suitably differentiable, we may take the derivative

Since this is a random variable, depending on ω, we may consider its expected value

(12.15)

Actually, we should wonder whether the inner limit exists. To see the relevance of the question, consider the payoff of a vanilla call option. We know that there is a point of nondifferentiability for _S_ _T_ = _K_ , which is potentially troublesome when we differentiate with respect to α = _S_ 0. Nevertheless, since this is a set of measure zero, we may argue that the limit exists with probability 1 and hope for the best. As we have already pointed out, however, what we are interested in is not the expectation of the limit, but the limit of an expectation:3

(12.16)

Comparing Eqs. (12.15) and (12.16) we see that if expectation and differentiation commute, we may use ∂ _f_ /∂α as an unbiased estimator of _g_ ′(α). However, this cannot be taken for granted in general. Since we fix a sample path ω and work on the dependence with respect to α, this approach is called _pathwise differentiation_.

It is possible to give a sound theoretical treatment of the above issues, which is outside the introductory scope of this book.4 However, the best way to get the essentials is by a concrete example. Let us consider a vanilla call option, with discounted payoff

(12.17)

where

(12.18)

and   ~ N(0, 1) is the essence of the sample path ω. We may regard the discounted payoff as our function _f_ (α, ω), which, in this specific case, can be rewritten as

Using the chain rule for differentiation, for a fixed sample path, we have

The second derivative is easy to find by considering Eq. (12.18) for a fixed  :

The first derivative is a bit more problematic, as the option payoff is kinky at the strike price _K_. Nevertheless, we have already observed that the trouble arises at a set of measure zero. At other points, we have

When taking the expectation in Eq. (12.15), we may disregard the only critical point and conclude

where **1** {·} is the usual indicator function for a given event. Putting everything together, we obtain the following estimator of Δ:

It is easy to implement the idea in R, as shown in Fig. 12.7. We may check the estimate against the exact result, in the same setting as the previous section:

**FIGURE 12.7** **Estimating the option Δ by a pathwise estimator.**

    > DeltaOK
    [1] 0.583839
    > set.seed(55555)
    > OutDeltaPath <- VanillaDeltaMCPath(S0,K,T,r,sigma,numRepl)
    > OutDeltaPath
    $estimate
    [1] 0.5813664
    $conf.int
    [1] 0.5750999 0.5876330

We see that the approach indeed works in this case, but it is quite useful to see a counterexample.

###   Example 12.1 Delta of a digital call

The digital call pays $1 when the underlying asset price at maturity is above the strike. Hence, the discounted payoff of a digital option is

(12.19)

This payoff is not only nondifferentiable at _S_ _T_ = _K_ , but also discontinuous. However, this is again a single point. The trouble is that elsewhere the derivative of this discounted payoff is just zero, as it is a piecewise constant function of _S_ _T_ , which is in turn a function of _S_ 0. Hence, for the above payoff, we find

since the delta of a digital call is certainly not zero.

By comparing Eqs. (12.17) and (12.19), we immediately see that pathwise differentiation cannot be used to estimate the Γ of a vanilla call, either. In the next section we consider an alternative approach that may overcome these issues. Before doing so, let us consider another example of pathwise differentiation, the delta of an Asian option. By discounting the payoff in Eq. (12.14), we are led to the function

(12.20)

where ω is a sequence of _T_ independent standard normals  _t_ , and the dependence on _S_ 0 is implicit in the arithmetic average  . Keeping the sample path ω = {  _t_ } fixed, we follow the same drill as above:

For the last derivative, we may expand the sum:

where we again keep the normals fixed. Therefore, the pathwise estimator for the Asian delta boils down to

which is readily implemented as shown in Fig. 12.8. The results we obtain by applying this function are in line with those obtained by finite differences:

**FIGURE 12.8** **Estimating Δ for an arithmetic average Asian option by a pathwise estimator.**

    > S0 <- 90
    > K <- 90
    > T <- 1
    > r <- 0.05
    > sigma <-0.4

    > numSamples <- 12
    > numRepl <- 50000
    > set.seed(55555)
    > AsianDeltaPath(S0,K,r,T,sigma,numRepl,numSamples)
    $estimate
    [1] 0.5726421
    $conf.int
    [1] 0.5673608 0.5779234

# 12.3 Estimating option greeks by the likelihood ratio method

The likelihood ratio method, also known as score function approach, casts the problem of sensitivity estimation in a quite different way with respect to pathwise differentiation. Pathwise differentiation revolves around the dependence of a function with respect to a parameter of interest. Since the functions we deal with in financial engineering are often nondifferentiable or even discontinuous option payoffs, this may raise some trouble, as we have seen in Example 12.1. On the contrary, in the likelihood ratio approach, the parameter is associated with the probability density of the underlying random variables. Hence, it is convenient to consider a function _f_ (·) depending on a vector of random variables **X** = [ _X_ 1,..., _X_ _n_ ] , and write

(12.21)

where _h_ α( **x** ) is the joint density of the random vector **X** , depending on the parameter α. If integration and differentiation commute, we may write

(12.22)

Assuming that the interchange is valid, we may use essentially the same trick we used in variance reduction by importance sampling. If we introduce the shorthand

to streamline notation a bit, we may rewrite Eq. (12.22) as

(12.23)

Note that, unlike  , _h_ α( **x** ) is a proper density, and that the ratio

is similar to a likelihood ratio, which we already met when dealing with importance sampling, and from which the name of the method is derived. Apparently, this approach is less natural than pathwise differentiation but, as we shall see shortly, it is actually simple to recast the problem in such a way that the parameter of interest is included in the density. The big advantage of this idea is that its validity relies on differentiability of probability densities, which are typically smooth, unlike option payoffs.

To illustrate the method, we use the two examples we are familiar with, the delta of vanilla and Asian call options. In the first case, the key point is to work on the density of _S_ _T_ considering _S_ 0 as a parameter. Assuming geometric Brownian motion as usual, we know from Section 3.7.4 that, under the risk-neutral measure,

This says that _S_ _T_ is lognormal, and we may write its density as5

(12.24)

where

is the density of a standard normal and

Using the chain rule, we find

which implies that the likelihood ratio is

The sensitivity estimator is obtained by plugging _x_ = _S_ _T_ and multiplying the likelihood ratio by the discounted payoff:

(12.25)

We may recast the expression as a function of the standard normal   that we use to sample _S_ _T_ starting from _S_ 0. By plugging Eq. (12.18) into (12.25) and simplifying the ratio, we end up with the estimator

(12.26)

The resulting code is displayed in Fig. 12.9. The result may be checked against the exact delta. Once again, the results are in line with the other methods:

**FIGURE 12.9** **Estimating Δ for an arithmetic average Asian option by likelihood ratios.**

    > S0 <- 50
    > K <- 55
    > T <- 1
    > r <- 0.1
    > sigma <-0.4
    > numRepl <- 50000
    > VanillaCallDelta(S0,K,T,r,sigma)
    [1] 0.583839
    > set.seed(55555)
    > VanillaDeltaMCLR(S0,K,T,r,sigma,numRepl)
    $estimate
    [1] 0.5729819
    $conf.int
    [1] 0.5592513 0.5867126

The case of the arithmetic average Asian option proceeds much along the same lines, but we have to cope with a multidimensional PDF _h_ _s_ 0 ( _S_ 1,..., _S_ _T_ ). Since the underlying GBM process is Markovian, we may factor the PDF as the product of conditional densities:

Each conditional density can be written just like in the case of the vanilla option:

where ϕ(·) is again the density of a standard normal, δ _t_ is the time elapsing between two observations in the average, and

The score function gets considerably simplified by the fact that _S_ 0 only occurs in the first conditional density. It is convenient to differentiate the logarithm of the above product, thus turning it into a sum. Using more or less the same calculations as in the vanilla call case, we find

where  1 is the first standard normal used for sample path generation, in the transition from _S_ 0 to _S_ 1. Therefore, the estimator of Δ is

which is used in the code of Fig. 12.10. Once again, the results are in line with the other methods:

**FIGURE 12.10** **Estimating Δ for an arithmetic average Asian option by likelihood ratios.**

    > S0 <- 90
    > K <- 90
    > T <- 1
    > r <- 0.05
    > sigma <-0.4
    > numSamples <- 12
    > numRepl <- 50000
    > set.seed(55555)
    > AsianDeltaLR(S0,K,r,T,sigma,numRepl,numSamples)
    $estimate
    [1] 0.5735931
    $conf.int
    [1] 0.5540027 0.5931835

# For further reading

  A deeper, yet readable treatment of bias issues in sensitivity estimation can be found in [4, Chapter 7].

  For a rigorous treatment of perturbation analysis, which is the background of pathwise derivatives, you may have a look at [3].

  The score function method is dealt with in depth in [8], even though the treatment is aimed at discrete-event systems. For discrete-time systems, see, e.g., [5].

  The interplay between simulation and optimization, where sensitivity analysis does play a role, is covered in [7] or [9], where you may also find information about stochastic approximation and stochastic gradient methods. The metamodeling approach is treated in depth by [6].

  Another high-level textbook reference for the topics of this chapter is [1, Chapters 7–8].

# References

1 S. Asmussen and P.W. Glynn. _Stochastic Simulation: Algorithms and Analysis_. Springer, New York, 2010.

2 P. Brandimarte. _Numerical Methods in Finance and Economics: A MATLAB-Based Introduction_ (2nd ed.). Wiley, Hoboken, NJ, 2006.

3 P. Glasserman. _Gradient Estimation via Perturbation Analysis_. Kluwer Academic, Boston, 1991.

4 P. Glasserman. _Monte Carlo Methods in Financial Engineering_. Springer, New York, 2004.

5 P.W. Glynn. Likelihood ratio gradient estimation for stochastic systems. _Communications of the ACM_ , 33:75–84, 1990.

6 J.P.C. Kleijnen. _Design and Analysis of Simulation Experiments_. Springer, Berlin, 2008.

7 G.C. Pflug. _Optimization of Stochastic Models: The Interface Between Simulation and Optimization_. Kluwer Academic, Dordrecht, The Netherlands, 1996.

8 R.Y. Rubinstein and A. Shapiro. _Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization via the Score Function Method_. Wiley, New York, 1993.

9 J.C. Spall. _Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control_. Wiley, Hoboken, NJ, 2003.

1See Chapter 11 for an introduction to path-dependent options.

2See, e.g., [2, Chapters 5 and 9].

3In this case we have an ordinary derivative, as the function _g_ (α) = E[ _f_ (α, ω)] only depends on α whereas in Eq. (12.15) we have a partial derivative. In many books, only ordinary derivatives are used, but we prefer to insist clearly on the dependence on ω.

4See, e.g., [4, p. 393].

5See Section 3.2.3.1.

# Chapter Thirteen

# Risk Measurement and Management

In this chapter we deal with a fundamental key issue in finance, i.e., risk. Actually, risk is a multidimensional concept, and there is a long list of risk factors that are relevant in finance. Risk factors directly related to financial markets are the first that come to mind, such as volatility in stock prices, interest rates, and foreign exchange rates, but they do not exhaust the list at all. For instance, after the subprime mortgage crisis, everyone is well aware of the role of credit risk, and the increasing interplay between financial markets and energy/commodity markets contributes to complicate the overall picture. In fact, the impact of derivative markets on oil and commodity (most notably, food) prices is controversial. The growth of derivative markets casts some doubts on seemingly obvious causal links: If one thinks about interest rate derivatives, it may seem natural assume that the value of those derivatives depends on prevailing conditions on money and capital markets. However, if one realizes the sheer volume of derivatives traded on regulated exchanges and over the counter, some doubts may arise about tails wagging dogs.

The above risk factors are often addressed by Monte Carlo methods, but we should mention that there are other risk factors that are less amenable to statistical approaches. The increasing role of information/communication technologies and globalization has introduced other forms of risk, such as operational risk, political risk, and regulatory risk. Ensuring business continuity in the face of operational disruptions is quite relevant to any financial firm, but it involves a different kind of risk analysis, and it is certainly difficult to attach probability distributions to political risk factors. Despite these difficulties, it is fundamental not to miss the interplay between financial and nonfinancial stakeholders. In times of general concern about the nature of financial speculation and the abuse of derivatives, it is refreshing to keep in mind that risk affects not only financial institutions, but also nonfinancial stakeholders, like firms using derivatives for hedging purposes. A global player is typically exposed, among other things, to foreign exchange risk. Choosing the right hedging instrument (e.g., forward/futures contracts, vanilla or exotic options) requires an assessment of the amount of exposure that should be covered. In such a context, things are made difficult by volume risk, which is related to how good or bad the business is going to be. Given this level of complexity, it is clear that we cannot adequately address the full set of risk dimensions in a single chapter. We will only deal with a limited subset of the above risk factors, through a few selected examples aimed at illustrating the potential of Monte Carlo applications for risk measurement and management.

Last but not least, we should remark that risk management is sometimes confused with risk measurement. It is certainly true that a sensible way to measure risk is a necessary condition to manage risk properly. But coming up with a number gauging risk, however useful, is not the same as establishing a policy, e.g., to hedge financial or commodity risk. In the first part of the chapter we deal with risk measures. Emphasis will be placed on quantile-based risk measures, such as value-at-risk (V@R) and conditional value-at-risk (CV@R). In the second part of the chapter, we move on and deal with risk management; we also illustrate a few links with certain stochastic optimization models dealt with in Chapter 10.

We start in Section 13.1 by discussing general issues concerning risk measurement, most notably the coherence of risk measures, Then, Section 13.2 is essentially devoted to V@R, a widely used and widely debated quantile-based risk measure. First we define V@R, then we illustrate a few of its shortcomings. Despite all of the criticism concerning its use, V@R is still a relevant risk measure, and it provides us with a good opportunity to appreciate Monte Carlo simulation in action. It is not too difficult to build a crude simulation model to estimate V@R, but there are several issues that we must take into account. As we have pointed out in Section 7.3, estimating quantiles raises some nontrivial statistical issues. Therefore, in Section 13.3 we estimate shortfall probabilities, from which other risk measures can be estimated. Furthermore, when a portfolio includes derivatives, it may be rather expensive to reprice them several times. Thus, we explore the possibility of using option greeks to speed up the computation. Another thorny issue is that in risk measurement we are concerned with extreme bad tails, where rare events are relevant. This may require the adoption of suitable variance reduction strategies, as we illustrate in Section 13.4. In the second half of the chapter we move from risk measurement to risk management. In Section 13.5 we take a brief detour to discuss stochastic programming models involving risk measures; in particular, we illustrate the difficulty of dealing with V@R, whereas the problem of CV@R optimization has a surprisingly simple solution, at least in principle. In Section 13.6 we illustrate how Monte Carlo methods can be used to assess the merits of a risk management policy. As a simple but instructive example, we consider delta hedging of a vanilla call option and compare it against a simpler stop-loss strategy. As expected, continuous-time delta hedging yields the Black–Scholes–Merton price for the option, assuming very idealized, not quite realistic, market conditions. Nevertheless, a simulation model is flexible and allows us to analyze the impact of additional features, like transaction costs, stochastic volatilities, and modeling errors. Finally, in Section 13.7 we discuss the interplay between financial and nonfinancial risk factors in a foreign exchange risk management problem.

# 13.1 What is a risk measure?

When setting up a portfolio of financial assets, stocks or bonds, or when writing an option. risk is the first and foremost concern. This is evident in basic portfolio management models in the Markowitz style, where expected return is traded off against variance. We have introduced this model in Section 3.1.1, and here we just recall that, if we consider a universe of _n_ assets with random rate of return _R_ _i_ , _i_ = 1,..., _n_ , and if **w**    _n_ is the vector collecting portfolio weights _w_ _i_ , then the variance of the portfolio return is

where the matrix ∑    _n,n_ collects the covariances σ _ij_ = Cov( _R_ _i_ , _R_ _j_ ) of the asset returns. Typically, the actual risk measure considered is the standard deviation σ _p_ , which has the same unit of measurement of expected return; the reason for minimizing variance is purely computational, as this is equivalent to minimizing standard deviation and results in an easy quadratic programming problem. Clearly, standard deviation can be considered as a risk measure: the smaller, the better. Standard deviation does capture the dispersion of a probability distribution, but is it really a good risk measure?

Before answering the question, we should actually clarify what a risk measure is. Standard deviation maps a random variable into a real number. Hence, we may define a risk measure as a function ρ( _X_ ) mapping a random variable _X_ into  . According to our convenience, the random variable _X_ may represent the value of a portfolio, or profit, or loss. Furthermore, when we consider loss, we may take the current wealth as a reference point, or its expected value in the future. Whatever the modeling choice is, a larger value of ρ( _X_ ) is associated with a riskier portfolio. By the way, standard deviation of portfolio return _R_ _p_ is just the same as the standard deviation of portfolio loss _L_ _p_ = − _R_ _p_. Indeed, standard deviation is symmetric, as it penalizes both the upside and the downside potential of a portfolio. Hence, it may be a suitable measure if we assume a symmetric distribution for asset returns; it is a bit less so, if we include assets with skewed, i.e., asymmetric returns. Even if we accept the hypothesis that stock returns are, e.g., normally distributed, and this is not quite the case, it is certainly not true that the same property holds for derivatives written on those assets, because of the nonlinearity of the payoff functions, which induces a nonlinearity in option values before maturity as well.

Given the limitations of standard deviation, we should look somewhere else to find alternative risk measures, but which ones make sense? What are, in principle, the desirable features of a risk measure? A plausible list of desiderata derives from intuitive properties that a risk measure should enjoy. Such a list has been compiled and leads to the definition of _coherent_ risk measure. Note that in the following statement we assume that _X_ is related to portfolio value or profit (the larger, the better); if we interpret _X_ as loss, the properties can be easily adjusted by flipping some inequalities and changing a few signs:

  **Normalization.** If the random variable is _X_ ≡ 0, it is reasonable to set ρ(0) = 0; if you do not hold any portfolio, you are not exposed to any risk.

  **Monotonicity.** If _X_ 1 ≤ _X_ 2, with probability 1, then ρ( _X_ 1) ≥ ρ( _X_ 2). In plain English, if the value of portfolio 1 is never larger than the value of portfolio 2, then portfolio 1 is at least as risky as portfolio 2.1

  **Translation invariance.** If we add a fixed amount to the portfolio, this will reduce risk: ρ( _X_ \+ α) = ρ( _X_ ) − α.

  **Positive homogeneity.** Intuitively, if we double the amount invested in a portfolio, we double risk. Formally: ρ(α _X_ ) = αρ( _X_ ), for α ≥ 0.

  **Subadditivity.** Diversification is expected to decrease risk; at the very least, diversification cannot increase risk. Hence, it makes sense to assume that the risk of the sum of two random variables should not exceed the sum of the respective risks: ρ( _X_ \+ _Y_ ) ≤ ρ( _X_ ) + ρ( _Y_ ).

The two last conditions may be combined into a convexity condition:

Convexity is a quite important feature when tackling optimization problems involving risk measures, as convex optimization problems are relatively easy to solve. We are dealing here only with a single-period problem; tackling multiperiod problems may complicate the matter further, introducing issues related to time consistency, which we do not consider here.2

These are the theoretical requirements of a risk measure, but what about the practical ones? Clearly, a risk measure should not be overly difficult to compute. Unfortunately, computational effort may be an issue, if we deal with financial derivatives whose pricing itself requires intensive computation. Another requirement is that it should be easily communicated to top management. A statistically motivated measure, characterizing a feature of a probability distribution, may be fine for the initiated, but a risk measure expressed in hard monetary terms can be easier to grasp. This is what led to the development of value-at-risk, which we describe in the next section. A further advantage of such a measure is that it sets all different kind of assets on a common ground. For instance, duration gives a measure of interest rate risk of bonds, and option greeks, like delta and gamma, tell something about the risk of derivatives, but it is important to find a measure summarizing all risk contributions, irrespective of the nature of the different positions.

Before closing the section, we should at least mention that in financial economics there is another approach to deal with risk aversion and decision under risk, based on expected utility theory. Utility functions, despite all of their limitations and the criticisms surrounding them, could be used in principle, but there are at least two related difficulties:

  It is difficult to elicit the utility function of a decision maker.

  If we are dealing with a portfolio of a mutual fund, whose risk aversion should we measure? The fund manager? The client? And how can we aggregate the risk aversion of several clients?

The last observation clearly points out that we need a hopefully _objective_ risk measure, related to the risk in the portfolio, rather than the subjective attitude of a decision maker toward risk. This is necessary if risk measures are to be used as a tool by regulators and managers.

# 13.2 Quantile-based risk measures: Value-at-risk

The trouble with standard deviation as a risk measure is that it does not focus on the bad tail of the profit/loss distribution. Hence, in order to account for asymmetry, it is natural to look for risk measures related to quantiles. The best-known such measure is value-at-risk. It is common to denote value-at-risk as VaR, where the last capital letter should avoid confusion with variance.3 We follow here the less ambiguous notational style of V@R. Informally, V@R aims at measuring the maximum portfolio loss one could suffer, over a given time horizon, within a given confidence level. Technically speaking, it is a quantile of the probability distribution of loss. Let _L_ _T_ be a random variable representing the loss of a portfolio over a holding period of length _T_ ; note that a negative value of loss corresponds to a profit. Then, V@R at confidence level 1 − α can be defined as the smallest number V@R1−α such that

(13.1)

This definition is also valid for discrete probability distributions. If _L_ _T_ is a continuous random variable and its CDF is invertible, we may rewrite Eq. (13.1) as

(13.2)

In the following discussion we will mostly assume for simplicity that loss is a continuous random variable, unless a discrete one is explicitly involved. For instance, if we set α = 0.05, we obtain V@R at 95%. The probability that the loss exceeds V@R is α.4

Actually, there are two possible definitions of V@R, depending on the reference wealth that we use in defining loss. Let _W_ 0 be the initial portfolio wealth. If _R_ _T_ is the random (rate of) return over the holding period, the future wealth is

Its expected value is

where μ is the expected return. The absolute loss over the holding period is related to the initial wealth:

The quantile of absolute loss at level 1 − α is the _absolute_ V@R at that confidence level. We find the _relative_ V@R if we take the expected future wealth as the reference in defining loss:

If we work with a short time period, say, one day, drift is dominated by volatility;5 hence, the expected return is essentially zero and the two definitions boil down to the same thing. Since certain bank regulations require the use of a risk measure in order to set aside enough cash to be able to cover _short-term_ losses, in this section we consider absolute V@R and drop the superscript from the definition of loss. Nevertheless, we should keep in mind that relative V@R may be more relevant to longer term risk, as the one faced by a pension fund. Furthermore, even if the underlying risk factors do not change at all, the sheer passage of time does have an effect on bonds and derivatives.

The loss _L_ _T_ is a random variable depending on a set of risk factors, possibly through complicated transformations linked to option pricing. To illustrate V@R using a simple example, let us consider a portfolio of stock shares and assume that the risk factor is portfolio return itself. If the holding period return has a continuous distribution, Eq. (13.2) implies that loss will exceed V@R with a low probability,

where we may indifferently write "≥" or ">," since the distribution involved is continuous. This can be rewritten as follows:

(13.3)

where we have defined

The return _r_ α will be negative in most practical cases and is just the quantile at level α of the distribution of portfolio return, i.e., the worst-case return with confidence level α. Thus, absolute V@R is

(13.4)

whereas relative V@R is

(13.5)

Computing V@R is very easy if we assume that the rate of return is normally distributed with

in which case there is no difference between absolute and relative V@R, and loss _L_ _T_ = − _W_ 0 _R_ _T_ is normal as well, _L_ _T_ ~ N(0, _W_ 02σ2). We may take advantage of the symmetry of the normal distribution, as the critical return _r_ α is, in absolute value, equal to the quantile _r_ 1−α. Then, to compute V@R1−α, we may use the familiar standardization/destandardization drill for normal variables:

where _Z_ is a standard normal variable. We have just to find the standard quantile _z_ 1−α and set

###   Example 13.1 Elementary V@R calculation

We have invested $100,000 in Quacko Corporation stock shares, whose daily volatility is 2%. Then, V@R at 95% level is

We are "95% sure" that we will not lose more than $3289.71 in one day. V@R at 99% level is

Clearly, increasing the confidence level by 4% has a significant effect, since we are working on the tail of the distribution.

The assumption of normality of returns can be dangerous, as the normal distribution has a relatively low kurtosis; alternative distributions have been proposed, featuring fatter tails, in order to better account for tail risk, which is what we are concerned about in risk management. Nevertheless, the calculation based on the normal distribution is so simple and appealing that it is tempting to use it even when we should rely on more realistic models. In practice, we are not interested in V@R for a single asset, but in V@R for a whole portfolio. Again, the normality assumption streamlines our task considerably.

###   Example 13.2 V@R in multiple dimensions

Suppose that we hold a portfolio of two assets. The portfolio weights are _w_ 1 =   and _w_ 2 =  , respectively. We also assume that the returns of the two assets have a jointly normal distribution; the two daily volatilities are σ1 = 2% and σ2 = 1%, respectively, and the correlation is ρ = 0.7. Let the time horizon be _T_ = 10 days; despite this, we assume again that expected holding period return is zero. To obtain portfolio risk, we first compute the variance of the holding period return:

Hence, σ _p_ = 0.05011. If the overall portfolio value is $10 million, and the required confidence level is 99%, we obtain

Once again, we stress that the calculations in Example 13.2 are quite simple (probably _too_ simple) since they rely on a few rather critical assumptions. To begin with, we have scaled volatility over time using the square-root law, which assumes independence of returns over time. If you are not willing to believe in the efficient market hypothesis, this may be an uncomfortable assumption. Then, we have taken advantage of the analytical tractability of the normal distribution, and the fact the return of stock shares was the only risk factor involved. Other risk factors may be involved, such as inflation and interest rates, and the portfolio can include derivatives, whose value is a complicated function of underlying asset prices. Even if we assume that the underlying risk factors are normally distributed, the portfolio value may be a nonlinear function of them, and the analytical tractability of the normal distribution is lost. Needless to say, Monte Carlo methods play a relevant role in estimating V@R in such cases, as we will illustrate later. However, we should also mention that a completely different route may be taken, based on historical V@R. So far, we have relied on a _parametric_ approach, based on a theoretical, not necessarily normal, probability distribution. One advantage of the normal distribution is that it simplifies the task of characterizing the joint distribution of returns, since we need only a correlation matrix. However, we know that correlations may not fully capture dependence between random variables, and this is especially true under the stress conditions that are relevant in risk management. The copula framework of Section 3.4 may be adopted to improve accuracy. Alternatively, rather than assuming a specific joint distribution, we may rely on a _nonparametric_ approach based on historical data. The advantage of historical data is that they should naturally capture dependence. Hence, we may combine them, according to _bootstrapping_ procedures, to generate future scenarios and estimate V@R by historical simulation.

Whatever approach we use for its computation, V@R is not free from some fundamental flaws, which depend on its definition as a quantile. We should be well aware of them, especially when using sophisticated computational tools that may lure us into a false sense of security. For instance, a quantile cannot distinguish between different tail shapes. Consider the two loss densities in Fig. 13.1. In Fig. 13.1(a) we observe a normal loss distribution and its 95% V@R, which is just its quantile at probability level 95%; the area of the right tail is 5%. In Fig. 13.1(b) we observe a sort of truncated distribution, obtained by appending a uniform tail to a normal density, which accounts for 5% of the total probability. By construction, V@R is the same in both cases, since the areas of the right tails are identical. However, we should not associate the same measure of risk with the two distributions. In the case of the normal distribution there is no upper bound to loss; in the second case, there is a clearly defined worst-case loss. Whether the risk for density (a) is larger than density (b) or not, it depends on how we measure risk exactly; the point is that V@R does not indicate any difference between them. In order to discriminate between the two cases, we may consider the expected value of loss _conditional_ on being on the right (bad) tail of the loss distribution. This conditional expectation yields the midpoint of the uniform tail in the truncated density; the conditional expected value may be larger in the normal case, because of its unbounded support. This observation has led to the definition of alternative risk measures, such as _conditional value-at-risk_ (CV@R), which is the expected value of loss, conditional on being to the right of V@R.

**FIGURE 13.1** **Value-at-risk can be the same in quite different situations.**

Risk measures like V@R or CV@R could also be used in portfolio optimization, by solving mathematical programs with the same structure as problem (10.12), where variance is replaced by such measures. The resulting optimization problem can be rather difficult. In particular, it may lack the convexity properties that are so important in optimization. It turns out that minimizing V@R, when uncertainty is modeled by a finite set of scenarios (which may be useful to capture complex distributions and dependencies among asset prices), is a nasty nonconvex problem, whereas minimizing CV@R is, from a computational viewpoint, easier as it yields a convex optimization problem.6

There is one last issue with V@R that deserves mention. Intuitively, risk is reduced by diversification. This should be reflected by any risk measure ρ(·) we consider and, as we have seen before, the subadditivity condition is needed to express such a requirement. The following counterexample is often used to show that V@R lacks this property.

###   Example 13.3 V@R is not subadditive

Let us consider two corporate bonds, _A_ and _B_ , whose issuers may default with probability 4%. Say that, in the case of default, we lose the full face value, $100 (in practice, we might partially recover the face value of the bond). Let us compute the V@R of each bond with confidence level 95%. Since loss has a discrete distribution in this example, we should use the more general definition of V@R provided by Eq. (13.1). The probability of default is 4%, and 1 −0.04 = 0.96 > 0.95; therefore, we find

Now what happens if we hold both bonds and assume independent defaults? We will suffer:

  A loss of $0, with probability 0.962 = 0.9216

  A loss of $100, with probability 2 × 0.96 × 0.04 = 0.0768

  A loss of $200, with probability 0.042 = 0.0016

Now the probability of losing $0 is smaller than 95%, and

Hence, with that confidence level, V@R( _A_ + _B_ ) = 100 > V@R( _A_ ) + V@R( _B_ ), which means that risk, as measured by V@R, may be increased by diversification.

The counterexample shows that V@R lacks one of the fundamental properties of a coherent risk measure. We should mention that if we restrict our attention to specific classes of distributions, such as the normal, V@R is subadditive. Nevertheless, the above considerations suggest the opportunity of introducing other risk measures, like CV@R. It can be shown that CV@R is a coherent risk measure.

# 13.3 Issues in Monte Carlo estimation of V@R

Using Monte Carlo methods to estimate V@R is, in principle, quite straightforward. Let _V_ ( **S** ( _t_ ), _t_ ) denote the value of a portfolio at time _t_ , depending on a vector of risk factors _S_ ( _t_ )    _n_ , with components _S_ _i_ ( _t_ ), _i_ = 1,..., _n_. The risk factors could be the underlying asset prices in a portfolio including derivatives, but we might also consider interest rates or exchange rates. After a time step of length δ _t_ , the risk factors are changed by a shock

To streamline notation, we will use _S_ _i_ ≡ _S_ _i_ ( _t_ ), _i_ = 1,..., _n_ , to denote the current value of the risk factors, and δ _i_ = δ _S_ _i_ to denote their shocks; these variables are collected into vectors **S** and δ, respectively. If we assume a joint distribution for δ, Monte Carlo estimation of V@R is straightforward, in principle:

**1.** We sample independent observations of the shocks δ.

**2.** We reprice each asset included in the portfolio for the new values of the risk factors, and we assess the corresponding loss

**3.** Given a set of loss observations, we estimate the required quantile.

A quick and dirty way to estimate the quantile would be to sort, say, 1000 observations of loss in decreasing order, and to report the one in position 50 if, for instance, we want V@R at 95% confidence level. For a slightly more careful discussion of quantile estimation, see Section 7.3, where we also underline the related difficulties. An alternative approach, proposed in [6, 7], is to fix a suitable loss threshold _x_ and to estimate the probability of losing more than this value, i.e., the _shortfall probability:_

(13.6)

Given _M_ replications, _k_ = 1,..., _k_ we can estimate the above probability asz

(13.7)

where **1** {·} is the indicator function for the relevant event. Then, we may build confidence intervals for this probability based on parameter estimation for the binomial distribution (using, e.g., the R function bino.fit). Furthermore, on the basis of the probability estimates corresponding to a few selected values of the threshold _x_ , we may build an empirical CDF, which enables us to come up with an estimate of V@R. Given the role of shortfall probabilities, in the following we will only be concerned with them.

Unfortunately, the first two steps in the above procedure are not quite trivial, either. Specifying a sensible distribution for multiple factors, especially under stress conditions, is far from trivial. Even if we pretend, as we do in the rest of this section, that a multivariate normal is a sensible model for the underlying risk factors, repricing each derivative contract in a real life portfolio may be a daunting task, possibly itself requiring expensive Monte Carlo runs. Using option greeks, it is possible to develop an approximation which may help in estimating V@R. If we assume that we own a portfolio of derivatives and that the only risk factors are the underlying asset prices, we might rely on a delta-gamma approximation to develop linear or quadratic models.7 Let us collect the first-order sensitivities into the vector

(13.8)

We should note that Δ _i_ should not be interpreted as the delta of a single option in the portfolio; rather, it is the sensitivity of the overall portfolio to a single risk factor. The portfolio may consist of long or short positions in any number of vanilla call or put options, or even exotic derivatives. Thus, Δ _i_ will be, in general, a linear combination of option deltas. Then, loss can be approximated by the linear model

(13.9)

A similar line of reasoning may be pursued for bonds, using their durations. Clearly, _L_ boils down to a normal random variable, if the underlying shocks are jointly normal. This makes computations quite simple, but possibly oversimplified. In fact, we are missing one point which may be quite relevant for both options and bonds, the passage of time: In general, long option positions lose value, which is measured by the option theta, which is usually negative. Let Θ be the portfolio theta, which may be positive or negative, depending on the sign of the positions. As we pointed out in the previous section, there are cases in which δ _t_ is very small and the effect of time can be ignored; for the sake of generality, here we do not neglect this issue. Furthermore, a linear approximation may be inaccurate, and we may include second-order sensitivities, related to option gammas, as a remedy. If we collect the second-order sensitivities into the symmetric matrix

the resulting second-order approximation is

(13.10)

The sensitivity with respect to time plays the role of a constant, but the quadratic term makes the overall approximation non-normal, even when δ is jointly normal.

To check the quality of delta–gamma approximations, let us compare the shortfall probability estimates obtained by:

**1.** A straightforward Monte Carlo procedure based on full repricing of a portfolio of derivatives. The R code is displayed in Fig. 13.2. Note that shocks in the value of underlying assets are generated according to normal distribution, rather than from a lognormal, to ease the comparison with the approach that we describe in the next section. The probability measure we use, of course, is the real one, as risk-neural measures are relevant for pricing purposes only.

**FIGURE 13.2** **Estimating a shortfall probability by straightforward Monte Carlo for a portfolio of call and put options.**

**2.** The corresponding estimate by a delta-gamma approximation, shown in Fig. 13.3. This function relies on R code to evaluate greeks; see Fig. 3.51.

**FIGURE 13.3** **Estimating a shortfall probability by a delta–gamma approximation.**

Here and in the following section we aim at replicating an experiment carried out in [7]. In that paper, the shortfall probability P{ _L_ > _x_ } is estimated, among other things, for a portfolio consisting of a short position in ten calls and five puts, all written on ten stock shares with similar characteristics; note that the number of calls and puts is negative. The input data are shown in Fig. 13.4, where we give a script to compare the two estimates. In the output lists out1 and out2 we also include vectors of losses corresponding to each scenario. Note that, by resetting the state of random generators, we are comparing estimates based on the same 3000 scenarios. It is interesting to produce a scatterplot of the true loss against the delta–gamma estimates and to check their correlation:

**FIGURE 13.4** **Comparing two estimates of shortfall probability.**

    > plot(out1$loss,out2$loss,pch=20)
    > cor(out1$loss, out2$loss)
    [1] 0.9979958

The correlation between losses is quite high and indeed the plot in Fig. 13.5 shows that the approximation looks fairly good. The two point estimates also look in fairly good agreement:

**FIGURE 13.5** **A scatterplot of true loss against the delta–gamma approximation.**

    > out2$estimate
    [1] 0 . 3206667

    > out2$estimate
    [1] 0 .33

A more careful check shows that there are some problems, though:

    > cor(out1$loss>50,out2$loss>50)
    [1] 0.9470662
    > cor(out1$loss>150,out2$loss>150)
    [1] 0.9334365
    > cor(out1$loss>250,out2$loss>250)
    [1] 0.9035132

Here we are comparing indicator variables detecting when the loss threshold is exceeded. These correlations get smaller and smaller when the threshold is increased. This seems to suggest that the quality of the delta–gamma approximation tends to deteriorate under extreme scenarios. Hence, we may be forced to resort to more sophisticated approximations, when available, or to full option repricing. Nevertheless, this kind of approximation may help improving the efficiency of Monte Carlo methods, as we illustrate in the next section.

# 13.4 Variance reduction methods for V@R

In this section we consider variance reduction methods, like importance sampling, to measure the risk of a portfolio of derivatives. The approach we describe was originally proposed in [7], and it is an excellent illustration of clever variance reduction strategies, taking advantage of the delta–gamma approximation we have discussed in the previous section. The treatment follows [6, Chapter 9], where more details can be found, as well as an extension to heavy-tailed distributions. As in the previous section, the objective is to estimate the probability _P_ { _L_ > _x_ } that loss _L_ exceeds a given threshold _x._ It would be convenient to approximate loss by the quadratic function of Eq. (13.10), but, as we have seen, this may be an inaccurate approximation for market scenarios under stress. Furthermore, risk measurement has to cope with tail risk, which implies that many replications might be actually useless, when exceeding the loss threshold is a rare event. As we have seen in Chapter 8, this is a typical case for importance sampling and, possibly, other variance reduction strategies. The main issue with importance sampling is how to find a suitable change of probability measure. To this end, we may take advantage of the quadratic delta-gamma approximation.

Let us assume, for the sake of simplicity, that δ ~ **N** ( **0** , ∑), where δ is the vector collecting the changes δ _S_ _i_ in the underlying risk factors. As we have seen in Section 5.5.2, sampling such a multivariate normal may be accomplished by choosing a matrix **C** such that **CC**  = ∑, generating a vector of independent standard normals **Z** ~ **N** ( **0** , **I** ), and then computing

(13.11)

Given the selected matrix **C** , the delta–gamma quadratic approximation of Eq. (13.10) can be rewritten as

(13.12)

where _a_ = −Θ δ _t_ and **b** = − **C**  Δ. Standard choices for the matrix **C** are the lower triangular Cholesky factor or the (symmetric) square root of the covariance matrix. However, a different choice can be advantageous. In fact, the quadratic form is a function of standard normals, but it is not normal itself, as it involves squares _Z_ _j_ 2 and products _Z_ _i_ _Z_ _j_ of standard normals. If we could get rid of these products, the quadratic form would be a much simpler function of standard normals, paving the way for useful analysis. This would be the case if the Hessian matrix **C**  Γ **C** in Eq. (13.12) were diagonal. In order to find a matrix **C** such that **CC**  = ∑ and the resulting Hessian is diagonal, we must find a way to transform the more familiar factorizations of the covariance matrix ∑. Let the square matrix **G** be one such factorization, i.e., a matrix such that **GG**  = ∑, like the lower triangular Cholesky factor of ∑. Since the matrix −1/2 **G  ΓG** is symmetric, it can be diagonalized:

(13.13)

where **U** is an orthogonal matrix (i.e, **U**  **U** = **UU**  = **I** , where **I** is the identity matrix) collecting the unit eigenvectors of −  **G**  Γ **G**. The diagonal matrix Λ consists of the corresponding eigenvalues:

Note that we are not diagonalizing a covariance matrix and that these eigenvalues might also be negative. Now, let us consider the matrix

(13.14)

This matrix meets our two requirements:

**1.** It factors the covariance matrix:

**2.** The resulting Hessian matrix is diagonal:

Therefore, if we sample the factor shocks using the matrix **C** of Eq. (13.14), we may rewrite the quadratic approximation _Q_ in Eq. (13.12) as

(13.15)

This form of _Q_ includes squared standard normals, which is a bit annoying, but at least it is the sum of independent terms. We may analyze its distribution by evaluating its characteristic function, or we may just complete the square,

which hints at a noncentral chi-square random variable. In fact, we recall from Section 3.2.3.2 that the chi-square distribution is obtained as a sum of squares of independent standard normals. In the expression above the standard normals _Z_ _i_ are shifted and there are additional transformations, but the resulting distribution can be analyzed by using moment and cumulant generating functions (see Section 8.6.2). We refrain from doing so, as this requires a deeper background, but we give the result of the analysis. The cumulant generating function for _Q_ is

(13.16)

On the basis of this knowledge, we might even compute the exact probability P{ _Q_ > _x_ }, which immediately suggests a control variate estimator to improve variance in Eq. (13.7):

(13.17)

A potential issue with this approach is that the correlation between _L_ and _Q_ may be weak just where it matters most, when loss is high, as we have seen in the previous section; hence, we will not pursue this approach here.8 Intuition suggests that importance sampling should work better in this specific case, and that we should sample in such a way that large losses, exceeding the threshold _x_ , are more likely. This means that we should change the distribution of the normals _Z_ _i_. If we look again at Eq. (13.15), we observe immediately that we may increase loss as follows:

  When _b_ _i_ > 0, we should have a positive _Z_ _i_ as well, which suggests increasing the expected value of _Z_ _i_ from 0 to a positive number.

  When _b_ _i_ < 0, we should have a negative _Z_ _i_ as well, which suggests decreasing the expected value of _Z_ _i_ from 0 to a negative number.

  When λ _i_ > 0, since it multiplies _Z_ _i_ 2, we could increase loss by increasing the variance of _Z_ _i_.

This intuition is made more precise in [7] by resorting to exponential tilting. In Section 8.6.3 we have seen that:

**1.** Exponential tilting transforms a normal variable _X_ into another normal variable.

**2.** The likelihood ratio is given by exp {−θ _X_ \+  (θ)}, where θ > 0 is the tilting parameter and  (θ) is the cumulant generating function (see Eq. 8.24).

Now the problem is how to choose a suitable random variable playing the role of _X_ in the likelihood ratio. Such a variable is the loss _Q_ in the delta–gamma approximation of Eq. (13.15). If we used _Q_ , the application of exponential tilting to the estimation of the shortfall probability would involve the importance sampling estimator _e_ −θ _Q_ + (θ) **1** ( _Q_ > _x_ }, which yields

(13.18)

where Eθ[·] denotes expectation under the importance sampling measure parameterized by the tilting coefficient θ. Note that, for θ > 0, the tilted measure gives more weight to large losses, and the likelihood ratio in Eq. (13.18) compensates for that. To better see why we may hope to reduce variance, let us consider the second-order moment of the estimator:

(13.19)

(13.20)

(13.21)

(13.22)

where:

  The equality in Eq. (13.19) follows from familiar properties of the exponential function and the fact that 12 = 1.

  We move back to the original probability measure in Eq. (13.20), which requires multiplying the estimator by the inverse of the likelihood ratio.

  The inequality in Eq. (13.21) follows from the fact that if _Q_ > _x_ , then _e_ −θ _Q_ < _e_ −θ _x_ for θ > 0.

  The inequality in Eq. (13.22) follows from the fact that **1** { _Q_ > _x_ } ≤ 1.

Clearly, this bound is reduced by increasing θ. However, since _Q_ may be a poor replacement of the true loss _L_ , we should apply the idea as follows:

(13.23)

Again, if we consider the second-order moment of this estimator (be sure to notice the change of measure in the expectation),

(13.24)

we see that it is small if θ > 0 and if _Q_ is large when the event { _L_ > _x_ } occurs. The correlation between _Q_ and _L_ may not be as large as we wish, but it should be enough for the idea to work. However, now we face the problem of sampling _Q_ and _L_ under the tilted measure, which may not be trivial in general. If we assume that the shocks δ are normally distributed, what we actually do is sample a vector of independent standard normals **Z** and set δ = **CZ** , where the matrix **C** is given by the diagonalization process that we have described before. These shocks may be used to reprice the portfolio exactly, rather than by the quadratic approximation. Then, the estimator is actually a function of the normal variables **Z** , and a key issue is what their distribution is under the tilted measure. We have seen in Example 8.10 a simple case in which by tilting a normal, we end up with another normal. This nice result, as shown in the original paper [7], holds here as well. Under the tilted measure with parameter θ, the distribution of the driving normals **Z** is still normal, **Z** ~ N(μ(θ), ∑(θ)), where the covariance matrix is diagonal. The components of μ(θ) and the diagonal entries of ∑(θ) are related to the _n_ eigenvalues λ _i_ of the Hessian matrix of the quadratic approximation:

(13.25)

By the way, these results agree with our intuitive discussion about how the probability of a large loss can be increased under the importance sampling measure: The sign of the expected value of each normal variable depends on the sign of the corresponding coefficient _b_ _i_ , and its variance is increased if the corresponding eigenvalue λ _i_ is positive. A quick look at these expressions shows that we have constraints on the tilting parameter:

(13.26)

These constraints are actually required to ensure the existence of the cumulant generating function in Eq. (13.16). Furthermore, we need some clue about a good setting of θ. Equation (13.22) provides us with an upper bound on the second-order moment of the estimator. This lower bound is minimized by minimizing the argument of the exponential in the expression of the upper bound,

Taking advantage of the convexity property of the cumulant generating function, we enforce the first-order optimality condition and find the minimum by solving the equation

(13.27)

The reader may notice that this is related to the result of equation (8.26).

What we have achieved may be summarized by the following procedure:

**Step 1.** Calculate the sensitivities (greeks) of the option portfolio.

**Step 2.** Using diagonalization of the Hessian matrix, find the parameters _a_ , _b_ _i_ , and λ _i_ in the quadratic approximation _Q_ of Eq. (13.15).

**Step 3.** Find the optimal tilting parameter θ by solving Eq. (13.16).

**Step 4.** For each replication:

  Sample **Z** from a multivariate normal with parameters given by Eq. (13.25).

  Evaluate the delta–gamma approximation _Q_ on the basis of Eq. (13.15), as well as the likelihood ratio _e_ −θ _Q_ + (θ).

  Evaluate the shock δ = **CZ** on the risk factors **S** , as in Eq. (13.11), reprice the portfolio to find _V_ ( **S** \+ δ, _t_ \+ δ _t_ ), and evaluate the loss _L_.

  Calculate the importance sampling estimator

**Step 5.** Return the average of the estimator.

To accomplish all of the above in the simple example involving vanilla call and put options (see Fig. 13.4), we use again the functions to evaluate option greeks, but we also need some auxiliary functions depicted in Fig. 13.6.

**FIGURE 13.6** **Some auxiliary functions to implement importance sampling for estimation of a shortfall probability.**

  The function psi evaluates the cumulant generating function for a given value of the tilting parameter tilt.

  The function psiPrime evaluates the derivative  '(θ).

  The function findTilt finds the optimal value of θ by solving Eq. (13.27). To this end, we use the function uniroot to find a positive root bounded by Eq. (13.26).

Armed with these functions, we may implement the procedure as shown in Fig. 13.7. Note that we are assuming normal shocks on the underlying asset prices, which is not quite consistent with lognormality of prices in the GBM model. We should generate normals _Z_ _i_ and compute their exponential, but using a simple Taylor expansion we see

**FIGURE 13.7** **Estimating a shortfall probability by importance sampling.**

Thus, the approximation is not too bad for small shocks. Unfortunately, it is large shocks that we are interested in, and we make them larger using exponential tilting. Indeed, we should change our definition of the greeks, in order to express the sensitivity of option prices to the underlying normal shocks, rather than asset prices. We avoid doing so for the sake of simplicity. Anyway, we compare results for functions based on the same approximation; thus, the comparison should be fair. In order to do so for the toy example, we use the script in Fig. 13.8, which produces the following output:

**FIGURE 13.8** **Comparing crude Monte Carlo and importance sampling.**

    estimate MC = 0.03497667
    estimate IS = 0.03534474
    percentage error crude MC = 3.194342 %
    percentage error imp. samp. = 1.079004 %

In order to avoid issues with the non-normality of the estimator, rather than producing a confidence interval using one sequence of replications, we run each function 100 times, producing 100 averages of replications. Then, we build a confidence interval based on these averages and evaluate a percentage error. In fact, we do see a significant reduction in variance when we use importance sampling. Clearly, this is just meant as an illustration and not as a conclusive evidence. We also mention that the variance reduction becomes really impressive when importance sampling is integrated with stratification; see [6].

# 13.5 Mean–risk models in stochastic programming

Estimating V@R, or any other risk measure, does not necessarily mean that we are actually _managing_ risk. To this end, we need to set up a strategy, or a decision model. For instance, the classical mean-variance portfolio optimization model is a decision support for managers interested in trading off expected return against variance/standard deviation. Since we have other risk measures at our disposal, it is natural to generalize the idea by replacing variance with, say, V@R or CV@R. This leads to mean-risk decision models. In this section we are not concerned with the appropriateness of each risk measure, but with the computational viability of each class of models. Minimizing variance subject to linear constraints, including a lower bound on expected return, is a straightforward convex quadratic programming model, because we may express variance as an explicit function of portfolio weights. In some specific cases, also minimizing V@R and CV@R may be quite simple. However, in general, we have to use the tools of stochastic programming to generate scenarios and solve the corresponding optimization problem. Needless to say, scenario generation methods, including Monte Carlo, play a key role here.9

Unfortunately, the minimization of V@R is not a convex optimization problem in general. One reason is its lack of subadditivity. Another reason is the interplay with Monte Carlo sampling. To see why, let us consider two assets, with sampled returns _R_ _i_ _s_ , where _i_ = 1, 2 refers to assets and _s_ is the scenario index. For a given scenario, the portfolio return is linear function of portfolio weights _w_ and 1 − ω:

If we see V@R as a function of _w_ , what we obtain is the envelope of linear affine functions. To see why, consider the worst-case return as a function of _w_ :

Thus, we find the lower envelope of a family of affine functions, which is clearly a continuous piecewise linear function, but nondifferentiable and not necessarily convex. Actually, we should select the worst case at confidence level α, but this does not change the nature of the resulting function. To visualize a picture and further investigate the issue, let us try the code in Fig. 13.9. Here we consider two assets with normally distributed returns. Of course, by using the concepts of Section 13.2, we may evaluate V@R as a function of _w_ explicitely, but let us consider a sample of randomly generated returns. In order to estimate V@R, we use the naive quantile estimation approach of Eq. (7.9). If we generate 100 scenarios and plot the V@R at 95% level, by repeating the random sampling four times, we obtain the plots in Fig. 13.10. There, we notice:

**FIGURE 13.9** **Plotting V@R as a function of portfolio weights: The case of two assets and discretized scenarios.**

**FIGURE 13.10** **Plotting V@R in a two-asset portfolio, for normally distributed returns: the case of 100 random scenarios.**

  There is quite some sampling variability; this is not quite surprising, as there is a significant volatility in the distributions of return, and we are collecting a statistic affected by tail behavior.

  The resulting function is nonconvex and nondifferentiable: an optimization nightmare.

If we increase the sample size to 10,000, the plot looks more reasonable, as we see in Fig. 13.11. As a check, if we set portfolio weight to _w_ = 0 and _w_ = 1, V@R should be as follows:

**FIGURE 13.11** **Plotting V@R in a two-asset portfolio, for normally distributed returns: the case of 10000 random scenarios.**

    > qnorm(0.95)*0.4-0.12
    [1] 0.5379415

    > qnorm(0.95)*0.3-0.10
    [1] 0.3934561

This is in good agreement with the picture, and we see that, by increasing the number of scenarios, we get closer to a smooth and convex function. Indeed, V@R is subadditive for a multivariate normal distribution, but, in a general case, anything can happen.

Since CV@R looks like a complication of V@R, it seems reasonable to expect that it is even a more difficult beast to tame. On the contrary, CV@R is much better behaved, which may be not quite surprising after all, since CV@R is a coherent risk measure and V@R is not. More surprisingly, minimizing CV@R may even lead to a (stochastic) linear programming model formulation.10 Let _f_ ( **x** , **Y** ) be a loss or cost function, depending on a vector of decision variables x and a vector of random variables **Y** with joint density _g_ **Y** ( **y** ), and consider function _F_ 1−α( **x** , ζ) defined as

where [ _z_ ]+ = max{ _z_ , 0}, and ζ    . is an auxiliary variable. It can be shown that minimization of CV@R, at confidence level 1 − α, is accomplished by the minimization of _F_ 1−α( **x** , ζ) with respect to its arguments. Furthermore, the resulting value of the auxiliary variable ζ turns out to be the corresponding V@R.

If we discretize the distribution of Y by a set _S_ of scenarios, characterized by realizations **y** s and probabilities π _s_ , _s_   _S_ , we may recast the problem as a stochastic programming problem:

Furthermore, if the loss _f_ ( **x** , **y** _s_ ) in scenario _s_ is a linear function, and the same applies to the additional constraints depending on the specific model, the minimization of CV@R boils down to the solution of a linear programming model. This important result should be tempered by the difficulty in getting a quantile-based estimate right using a limited number of scenarios. See, e.g., [5] for some critical remarks on the coherence of risk measures and their estimates.

# 13.6 Simulating delta hedging strategies

In this section we illustrate how Monte Carlo simulation may be used to check the effectiveness of hedging strategies. As an illustrative example, we consider delta hedging in its simplest form. We know from Chapter 3 that the price of a call option written on a non-dividend-paying stock is essentially the cost of a delta hedging strategy and that the continuous-time hedging strategy requires holding an amount Δ of the underlying asset. To summarize the essence of the reasoning, we have a portfolio whose value is a function Π( _S_ ) of the underlying asset price _S_. To hedge against changes in _S_ , we may hold an amount _h_ of the underlying asset. Then, the overall value of the portfolio is

If we look for first-order immunization, we set the first-order derivative to zero:

Delta hedging is a commonly mentioned risk management strategy, but it is open to criticism:

  We have a perfect hedge for infinitesimal perturbations, rather than a good hedge for larger shocks.

  We should rebalance the hedge frequently, incurring transaction costs.

  Delta hedging as a pricing argument in the BSM model neglects not only transaction costs, but stochastic volatilities, jumps, market impact, herding behavior, etc.

In a word, naive delta hedging is subject to modeling errors. Hence, we should probably at least check its effectiveness and robustness by simulation experiments in a realistic setting. Needless to say, this is a job for Monte Carlo methods. For illustration purposes we consider hedging a vanilla call option in the BSM world. Clearly, in real life the net factor exposure due to a whole portfolio of options should be considered.

It may also be instructive to compare delta hedging against another strategy, the stop-loss strategy.11 Stop loss is simpler than delta hedging and model-free. The idea is that we should have a covered position (hold one share) when the option is in the money, and a naked position (hold no share) when it is out of the money. So, we should buy a share when the asset price goes above the strike price _K_ , and we should sell it when it goes below. Ideally, if we buy at _K_ and sell at _K_ , the cash flows cancel each other, assuming that we disregard the time value of money. This strategy makes intuitive sense, but it is not that trivial to analyze in continuous time.12 Nevertheless, it is easy to evaluate its performance in discrete time by Monte Carlo simulation. The problem with executing the strategy in discrete time is that we cannot really buy or sell at the strike price: We buy at a price larger than _K_ , when we detect that the price went above that critical value, and we sell at a price which is slightly lower. Hence, we cannot claim that cash flows due to buying and selling at _K_ cancel each other. So, even without considering transaction costs, which would affect delta hedging as well, we see a potential trouble with the stop-loss strategy.

An R function to estimate the average cost of a stop-loss strategy is given in Fig. 13.12. The function receives the matrix paths of sample paths, possibly generated as we have seen in Chapter 6, by the function simvGBM. Note that in this case, unlike option pricing, the real drift mu must be used in the simulation, as we check hedging in the real world, and not under the risk-neutral measure. Note that the true number of steps (time intervals) is one less the number of columns in matrix paths, which includes the initial price. If we need to buy shares of the underlying stock, we may need to borrow money, which should be taken into account. But, since we assume deterministic and constant interest rates, we will not account for borrowed money, since we can simply record cash flows from trading and discount them back to time _t_ = 0, having precomputed discount factors in the vector discountFactors. We use a state variable, covered, to detect when we cross the strike price going up or down. Since cash flow is negative when we buy, and positive when we sell, the option "price" is evaluated as the average total discounted cash flow, with a change in sign. We should also pay attention to what happens at maturity: If the option is in the money, the option holder will exercise her right and we will also earn the strike price, which should be included in the cash flow stream.

**FIGURE 13.12** **Evaluating the cost of a stop-loss hedging strategy.**

Since vectorizing R code is often beneficial, we also show a vectorized version of this code in Fig. 13.13. The main trick here is using a vector oldPrice, which is essentially a shifted copy of paths, to spot where the price crosses the critical level, going up or down. The time instants at which we go up are recorded in vector upTimes, where we have a negative cash flow; a similar consideration applies to downTimes.

**FIGURE 13.13** **Vectorized code for the stop-loss hedging strategy.**

Now let us check if the two functions are actually consistent, i.e., if they yield the same results, and whether there is any advantage in vectorization:

    > S0 <- 50
    > K <- 52
    > mu <- 0.1;
    > sigma <- 0.4;
    > r <- 0.05;
    > T <- 5/12;
    > numRepl <- 100000
    > numSteps <- 10
    > set.seed(55555)
    > paths <- simvGBM(S0,mu,sigma,T,numSteps, numRepl)
    > system.time(cost <- stopLoss(paths,K,r,T))
    user system elapsed
    4.93 0.00 4.93
    > system.time(costV <- stopLossV(paths,K,r,T))
    user system elapsed
    0.20 0.00 0.21
    > cost$mean
    [1] 4.833226
    > costV$mean
    [1] 4.833226

Indeed, using the function system.time we observe that there is a significant difference in CPU time. Now we should compare the cost of the stop-loss strategy against the cost of delta hedging, as well as the theoretical option price. A code to estimate the average cost of delta hedging is displayed in Fig. 13.14. The code is similar to the stop-loss strategy, but it is not vectorized. The only vectorization we have done is in calling BSMdelta once to get the option Δ for each point on the sample path. Note that Δ must be computed using the current asset price and the current time to maturity. The current position in the stock is updated given the new Δ, generating positive or negative cash flows that are discounted back to time _t_ = 0.

**FIGURE 13.14** **Evaluating the performance of delta hedging.**

Figure 13.15 displays a script to compare performances of the two hedging strategies, in terms of both expected value and standard deviation of the present value of hedging cost. In the first pair of runs, we are only using 10 hedging steps, which has an impact on hedging errors; in the second pair this is increased to 100 steps. We also compute the exact option price using the BSM formula, for comparison purposes. By running the script, we obtain

**FIGURE 13.15** **A script to compare hedging strategies.**

    true price = 4.732837
    StopLoss 10 steps: mean = 4.833226 sd = 4.542682
    DeltaHedging 10 steps: mean = 4.729912 sd = 1.380229
    StopLoss 100 steps: mean = 4.807347 sd = 4.074755
    DeltaHedging 100 steps: mean = 4.734969 sd = 0.4529393

We may observe the following:

  The average cost of the stop-loss strategy is larger than the average cost of delta hedging.

  What is more significant, however, is the standard deviation of the hedging cost. This is converging to zero with delta hedging, whose expected cost converges to the BSM option price.

Indeed, the price of an option is related to its hedging cost. In the idealized BSM world, the market is complete and we may hedge perfectly the option risk. In other words, we are replicating it exactly. Of course, the introduction of stochastic volatility, transaction costs, modeling errors, etc., make hedging less than perfect. As we mentioned, delta hedging has been often criticized as an over-simplified, possibly misleading, approach. The important message is that by Monte Carlo methods we may assess all of the above effects, for this and other hedging policies.

# 13.7 The interplay of financial and nonfinancial risks

Let us consider a firm that is subject to currency exchange risk, but also to _volume_ risk.13 To make the problem as simple as possible, let us assume that the current exchange rate is 1.0, expressed in any currency exchange ratio you like. There is considerable uncertainty on the exchange rate at some future time _T_ , and let us assume that it is uniformly distributed between 0.7 and 1.3. The firm has to buy a significant amount of the foreign currency for its activity, say, 50,000. If the rate increases, the firm will have to pay much more for that amount of currency; if, on the contrary, the rate decreases, that will be good news. Let us say that the base case is when the rate stays at the current level. Considering the two extreme scenarios, if the rate goes up and turns out to be 1.3, the firm will incur a loss given by

On the contrary, it will have a corresponding gain if the rate turns out to be 0.7.

Let us assume that the firm can hedge using two types of derivative:

  A long position in a forward contract, where we assume that the forward price is just 1.0.14

  A long position in at-the-money call options. Unlike forward contracts, this requires an upfront payment. Let us assume that the price of each option is 0.1; in other words, we have to pay 10% of the nominal amount we hedge, in the domestic currency.

The decision that the firm has to make is twofold:

  They have do decide how much to hedge.

  They have to decide the hedge mix, i.e., how much to hedge with forward contracts vs. options.

Hence, if we only consider the scenarios (0.7, 1.0, 1.3), the corresponding outcomes would be

  (15,000, 0, − 15,000) with 0% hedging.

  (0, 0, 0) with 100% hedging using forward contracts.

  (−5000, −5000, 10,000) with 100% hedging using options; note that in this case we have to pay 5000 for the options, which expire worthless in the first two outcomes, whereas we have to subtract the option price from the option payoff in the third case.

With no hedging there is considerable uncertainty, which is completely eliminated by forward contracts. With options, we limit the downside a bit, while retaining some upside. The choice is a matter of risk aversion, as well as probabilities, assuming that we are willing to associate probabilities with each event.

In many practical settings, however, there is still another complication: volume risk. We may not really know precisely the amount we have to hedge. Say that 50,000 is the volume in the nominal scenario, i.e., the base case, but the amount we need to hedge is uniformly distributed between 10,000 and 90,000. For the sake of simplicity, we assume that the two sources of risk are independent but, clearly, Monte Carlo methods may be applied in more complicated and realistic settings. To see why volume risk is relevant, let us check the outcome resulting from 100% hedging with forward contracts in the following scenario: exchange rate is 0.7 and required volume is 10,000. Now we have to buy (and arguably to sell immediately) 40,000 useless units of foreign currency, with a loss of

Apart from the loss, we are here in a probably bad business setting, since the reduced requirements is likely to come from lost business. With 100% coverage using call options, which give the right but not the obligation to buy, the loss would be just the foregone option premium, 5000.

Deciding the best course of action is by no means a trivial task, even in this idealized setting. To support the decision makers, it would be nice to have a picture of the profit/loss scenarios as a function of the decision variables. Then we might also want to plot a given risk measure. All of these important tasks can be accomplished by Monte Carlo sampling.

# For further reading

  An elementary introduction to risk aversion and related issues can be found in [3].

  The original treatment of coherent risk measures can be found in [1].

  For Monte Carlo applications to credit risk, see [6, Chapter 9].

  The reformulation of CV@R in a way suitable for stochastic optimization was originally proposed in [9, 10].

# References

1 P. Artzner, F. Delbaen, J.-M. Eber, and D. Heath. Coherent measures of risk. _Mathematical Finance_ , 9:203–228, 1999.

2 P. Artzner, F. Delbaen, J.-M. Eber, D. Heath, and H. Ku. Coherent multiperiod risk adjusted values and Bellman's principle. _Annals of Operations Research_ , 152:5–22, 2007.

3 P. Brandimarte. _Quantitative Methods: An Introduction for Business Management._ Wiley, Hoboken, NJ, 2011.

4 P.P. Carr and R.A. Jarrow. The stop-loss start-gain paradox and option valuation: a new decomposition into intrinsic and time value. _Review of Financial Studies_ , 3:469–492, 1990.

5 F.J. Fabozzi and R. Tunaru. On risk management problems related to a coherence property. _Quantitative Finance_ , 6:75-81, 2006.

6 P. Glasserman. _Monte Carlo Methods in Financial Engineering._ Springer, New York, 2004.

7 P. Glasserman, P. Heidelberger, and P. Shahabuddin. Variance reduction techniques for estimating value-at-risk. _Management Science_ , 46:1349–1364, 2000.

8 J.C. Hull. _Options, Futures, and Other Derivatives_ (8th ed.). Prentice Hall, Upper Saddle River, NJ, 2011.

9 R.T. Rockafellar and S. Uryasev. Optimization of conditional value-at-risk. _The Journal of Risk_ , 2:21–41, 2000.

10 R.T. Rockafellar and S. Uryasev. Conditional value-at-risk for general loss distributions. _Journal of Banking and Finance_ , 26:1443–1471, 2002.

1Since we are comparing random variables, the inequality should be qualified as holding almost surely, i.e., for all of the possible outcomes, with the exception of a set of measure zero. The unfamiliar reader may consider this as a technicality.

2The essence of time consistency of a multiperiod risk measure is that if a portfolio is riskier than another portfolio at time horizon τ, then it is riskier at time horizons _t_ < τ as well. See, e.g., [2].

3Arguably, the lowercase letter in the middle should also avoid confusion with VAR, which usually refers to vector autoregressive models.

4We stick to the statistical convention that α is the small area associated with the tail, but sometimes the opposite notation is adopted.

5Intuitively, _drift_ is related to expected return, and _volatility_ is related to standard deviation. On a short time interval of length δ _t_ , since drift scales linearly with δ _t_ , whereas volatility is proportional to √δ _t_ , drift goes to zero more rapidly than does volatility. See the square-root rule, which we considered in Section 3.7.1.

6We illustrate this later in Section 13.5.

7To be precise, if we assume that the risk factors are just the underlying asset prices, we may rely on the standard formulas of Section 3.9.2 for option greeks. Otherwise, option senstivities must be determined, and estimated, by a careful analysis.

8See [6, Section 9.2].

9See Section 10.5.3.

10In this section, we rely on results from Rockafellar and Uryasev [9, 10], which we take for granted, thereby cutting a few corners.

11See [8, pp. 300–302].

12See [4].

13For a real life example of such a situation, please refer to the following business case, which has been an inspiration for the section: M.A. Desai, A. Sjoman, and V. Dessain. _Hedging Currency Risks at AIFS._ Case no. 9-205-026, Harvard Business School Publishing.

14Therefore, we are assuming that there is no difference between the interest rates in the two currencies.

# Chapter Fourteen

# Markov Chain Monte Carlo and Bayesian Statistics

We have introduced Bayesian parameter estimation in Section 4.6, as a possible way to overcome some limitations of orthodox statistics. The essence of the approach can be summarized as follows:

posterior ∝ prior × likelihood,

where the prior collects information, possibly of subjective nature, that we have about a set of parameters before observing new data; the likelihood measures how likely is what we observe, on the basis of our current knowledge or belief; and the posterior merges the two above pieces of information in order to provide us with an updated picture. This is an informal restatement of Eq. (4.12), which we rewrite below for convenience:

(14.1)

The parameters θ _j_ , _j_ = 1,..., _q_ , are regarded as random variables within the Bayesian framework, and _p_ (θ1,..., θ _q_ ) is their prior. The prior can be a probability density function (PDF) or a probability mass function (PMF), or a mixed object, depending on the nature of the parameters involved. For the sake of simplicity, in this chapter we will not deal systematically with both cases, and we will refer to either one according to convenience. We have a set of independent observations, i.e., realizations of a random variable taking values _x_ _i_ , _i_ = 1,..., _n_. Note that here we are considering _n_ observations of a scalar random variable, whose distribution depends on _q_ parameters; the concepts can be generalized to multivariate distributions. We know from maximum-likelihood estimation that an important role is played by the likelihood function _f_ _n_ ( _x_ 1,..., _x_ _n_ | θ1,..., θ _q_ ), for given parameters. Since we are considering independent observations, the likelihood is the product of PDF values corresponding to each observation _x_ _i_. Bayes' theorem yields the posterior density _p_ _n_ (θ1,..., θ _q_ | _x_ 1,..., _x_ _n_ ), summarizing our knowledge about the parameters, merging the possibly subjective prior and the empirical evidence.

Armed with the posterior density, we may proceed to find estimates of the parameters θ1,..., θ _q_ , e.g., by finding their expected values. We may also find credible intervals, which are the Bayesian counterpart of confidence intervals, and test hypotheses about parameters. All of this requires knowledge of the posterior density, but there is a missing piece in Eq. (14.1). There, we state that the posterior is _proportional_ to the product of prior and likelihood, but to fully specify the posterior density, we should also find a normalization constant _B_ such that the PDF integrates to 1. Hence, we should divide the posterior by

Unless there is some special structure to be exploited, the calculation of this multidimensional integral is difficult. As we have seen in Section 4.6, there are nice cases in which the form of the posterior is easy to find. If we deal with observations from a normal distribution and the prior is normal, then the posterior is normal too, and figuring out the normalizing constant is easy. When the posterior belongs to the same family of distributions as the prior, for a given kind of likelihood, we speak of _conjugate_ distributions; the prior is also called the conjugate prior. In less lucky cases, one possibility to find the normalization constant, needless to say, is the numerical evaluation of the multidimensional integral by Monte Carlo methods. However, there is a possibly better idea, if what we need is just the ability to _sample_ from the posterior density. As it turns out, there are ways to sample from a density which is known up to a constant, and in fact we have already met one: acceptance–rejection.

In Section 14.1 we show that we may apply acceptance–rejection sampling when we miss a normalization constant, but the resulting approach need not be the most efficient. There is a class of sampling approaches, collectively known as Markov chain Monte Carlo (MCMC) methods, that can be used. The idea, as the name suggests, is to simulate a Markov chain, whose stationary distribution is the posterior density. MCMC methods can be introduced as a tool for computational Bayesian statistics, but they have a wider application range, as they can be used as random sample generators for difficult cases that cannot be tackled using standard approaches. We review a few basic facts about the stationary distribution of Markov chains Section 14.2. Then, we outline the Metropolis–Hastings algorithm and some of its variations in Section 14.3. Finally, in Section 14.4 we show how the simulated annealing algorithm, a stochastic search algorithm for both global and combinatorial optimization that we introduced in Section 10.3, can be cast and analyzed within the MCMC framework.

# 14.1 Acceptance–rejection sampling in Bayesian statistics

Acceptance–rejection sampling is a general purpose approach to sample from a distribution with a given PDF or PMF, whereas the inverse transform method works with the CDF. As we have seen in Section 5.4, given a target density _f_ (·), we choose an instrumental density _g_ (·) and proceed as follows:

**1.** Generate _Y_ ~ _g._

**2.** Generate _U_ ~ U(0, 1), independent of _Y._

**3.** If

where _c_ is an upper bound on the ratio _f_ / _g_ , accept and return _X_ = _Y_ ; otherwise, reject and repeat the procedure.

If you look at the proof of the validity of the method, in Section 5.4, you will notice that what really matters is that _c_ is an upper bound on the ratio of _f_ / _g_ , and we may also work with an unscaled density _f_ (·), i.e., a density known up to a constant. In the Bayesian framework, we have an unscaled posterior

where **x** is the vector of (given) observations and θ the vector of (random) parameters. To sample a vector of parameters from the unsealed posterior, we have to find a candidate density _g_ (θ) and an upper bound _M_ on the ratio:

Then, we sample a candidate value   randomly from _g_ (·), and we accept it with probability

Improvements to this basic strategy have been proposed, but they do not solve one basic difficulty: the number of iterations that are needed to accept a sampled value can be large. We know that this is given by _M_ ; hence, the closer the instrumental density to the target, the better. If we are working with a distribution with bounded support, like a beta distribution, we may use a uniform as the instrumental density. However, the choice of the target is more difficult if the target has unbounded support, in which case the candidate should be a heavy-tailed distribution. Unfortunately, when we are dealing with multiple parameters, acceptance–rejection may require the generation of many proposals before one is accepted. Because of this practical limitation, alternative methods, described in the following, have been proposed. Nevertheless, acceptance–rejection is a viable approach for the case of a single parameter.

# 14.2 An introduction to Markov chains

Before we introduce MCMC, we need to refresh some essential concepts pertaining to discrete-time Markov chains. MCMC methods are indirect sampling strategies, based on the simulation of a Markov process, whose long-run state density is the target density. We have already met Markov processes in the book, since processes described by Itô stochastic differential equations are Markovian. Here, though, we focus on their long-term state distribution. Within a Bayesian framework, the state of the stochastic process is related to the value of the parameters θ. When the parameters are real numbers, we have to deal with a continuous-state process. However, the necessary intuition to understand MCMC methods can be built on the basis of the much simpler discrete-state case, which is referred to as Markov chain. A discrete-time Markov chain is a process with a state variable

taking values on a discrete set, which may be identified with the set of integer numbers, or a subset of it, or a finite discrete set (e.g., the credit rating of a bond). We may represent the process as a graph, which is where the name "chain" comes from. A simple example is shown in Fig. 14.1. There, the state space is the set { _A, B, C, D_ }, and nodes correspond to states. Directed arcs are labeled by transition probabilities. For instance, if we are in state _C_ , there is a probability 0.7 to be in state _B_ at the next step, and a probability 0.3 to be in state _A._ Note that these probabilities depend only on the current state, and not on the whole past history. Indeed, the essential property of Markov processes is related to their limited memory: The only information that is needed to characterize the future system evolution is the current state. Formally, we may express this in terms of the conditional transition probability

**FIGURE 14.1** **A discrete-state Markov chain.**

Hence, we just need to know the one-step transition probabilities to characterize the chain. A further simplification occurs if the process is stationary, i.e., if the transition probabilities do not depend on time. In such a case, we may describe the chain in terms of the transition probabilities

which may be collected into the one-step transition probability matrix **P**. For the chain in Fig. 14.1, we have

Such a matrix has rather peculiar properties, related to the fact that after a transition we must land somewhere within the state space, and therefore each and every row adds up to 1:

where for the sake of simplicity we assume a finite state space of size _N._ Let π0    _N_ denote the initial state distribution at step 0, i.e.,

The total probability theorem implies that

which can be rewritten as

where π1    _N_ is a vector with components π _i_ ,1 = P{ _X_ 1 = _i_ }, _i_ = 1,..., _N._ Note how, given the shape of the transition probability matrix, we write the relationship in terms of premultiplication by a _row_ vector. Thanks to the Markov property, we may easily compute multiple step transition probabilities:

Here, in the first line we have used the fact that the event { _X_ 2 = _j_ } can be expressed as the union of the disjoint events { _X_ 2 = _j, X_ 1 = _i_ }; then we have used conditioning and the Markov property. Since the final expression can be interpreted as an entry in the row-by-column matrix product, the two-step transition probability matrix can be expressed as

By the same token, if we consider an arbitrary number _m_ of steps,

Now imagine that we start with distribution π0 and we let the system run for a long time. What happens to the limit

We might guess that, after such a long time, the memory of the initial state is lost, and that the system reaches a steady-state (or long-run) distribution π. We might also guess that this distribution should be an equilibrium distribution characterized by the system of linear equations

(14.2)

This system can be interpreted by saying that if we start with the state distribution π, the next state has the same distribution. Hence, π is an eigenvector of **P** associated with the eigenvalue 1. To be precise, since we premultiply **P** by a row vector, we should use the term _left_ eigenvector. To make sense as a probability distribution, this should be a unit (normalized) and non-negative eigenvector. It turns out that, in fact, we may find a non-negative eigenvector, provided that some conditions are satisfied for **P**. Indeed, we cannot take for granted that a stationary distribution exists, independent from the initial state, as a few things may go wrong:

  The state space could be, in some sense, disconnected, as shown in Fig. 14.2. Clearly, in such a case, if we start in one region, we will stay there and the memory of the initial state cannot be lost completely. We rule out this case by considering only _irreducible_ chains.

**FIGURE 14.2** **A nonirreducible periodic Markov chain.**

  The chain could be periodic, as the one shown in Fig. 14.3. As we see, despite some uncertainty between states _C_ and _D_ , states _A, B_ , and _E_ will be repeated every four steps. This kind of behavior rules out the existence of a stationary, long-run distribution. Hence, we have to refer to _aperiodic_ chains.

**FIGURE 14.3** **A periodic Markov chain.**

We will refrain from giving a formal definition of irreducible and aperiodic chains, as intuition is all we need for our limited purposes.

Another very useful way to interpret the equilibrium condition of Eq. (14.2) can be obtained by focusing on a single equation,

which gives the equilibrium probability π _i_ of state _i_ , and by rewriting it as follows:

The last equality can be interpreted intuitively in terms of equilibrium of "probability flows:"

  The term   is the probability flow out of state _i_ to states _j_ = 1,..., _N_ (there is some flow from _i_ to itself if the transition from _i_ to itself is possible).

  The term   is the probability flow from states _j_ = 1,..., _N_ into state _i._

Thus, equating the two flows for every state _i_ establishes a _global_ balance condition, which is associated with equilibrium. For some Markov chains, a stronger equilibrium condition is satisfied, corresponding to a _detailed_ , rather than global balance:

(14.3)

This condition enforces a probability flow equilibrium for any pair of states, and it is easy to see that if we can find a vector π, whose components add up to 1, satisfying Eq. (14.3), then this vector will also satisfy Eq. (14.2). To prove this, it is sufficient to sum the detailed balance over states _i_ :

The detailed balance conditions are quite interesting for a few reasons:

  The resulting chain is time reversible, i.e., we cannot distinguish sample paths going forward and backward in time.

  This structure facilitates the solution of the equilibrium equations, leading to nice structures, such as product form queueing networks.

These concepts are beyond the scope of the book, but from our viewpoint, detailed balance conditions are useful when we have to go the other way around: Given a target equilibrium distribution, how can we build a Markov chain that has that equilibrium distribution? This is what we need in MCMC methods applied to Bayesian statistics, where the target equilibrium distribution is the posterior, with the additional twist that the posterior is known up to a multiplicative constant. This would be difficult in general, but, by taking advantage of detailed balance equations, it can be easily accomplished, as we show in the next section.

# 14.3 The Metropolis–Hastings algorithm

In Bayesian inference, we need to sample from the posterior density, which is hard to find computationally, with the exception of conjugate priors. A computational breakthrough was achieved by realizing that we may create a discrete-time Markov process, whose states are values of the parameters of interest, such that its long-run distribution can be shaped according to our needs. In other words, we need to define a state space based on the values of the parameters, and compute suitable transition probabilities. This is a daunting task in general, but it turns out that we may take advantage of the simple structure of time-reversible Markov chains, whose transition probabilities satisfy the detailed balance equations (14.3).

The original idea was devised in 1953 and is known as the Metropolis–Hastings algorithm.1 The intuition is as follows. We want to find a Markov chain with steady-state probabilities π _i_ , for discrete states _i_ = 1,..., _N_ (the idea can be easily adapted to continuous distributions). To this end, we need suitable transition probabilities _P_ _ij_ , satisfying the detailed balance equations. A first point is that whatever _P_ _ii_ we choose, it will do the job, as the trivial balance condition

is clearly satisfied. Unfortunately, in general, the equilibrium condition does not hold. For instance, if

then we should restore the balance by rejecting some of the transitions from _j_ to _i_ , whereas all of the transitions from _i_ to _j_ can be accepted. A similar consideration applies if π _i_ _P_ _ij_ > π _j_ _P_ _ji_. The trick can be done, for any desired steady-state distribution π and any transition matrix **P** , by adjusting the transition probabilities using a sort of acceptance-rejection strategy. For each pair of states _i_ and _j_ , we adjust the transition probability multiplying it by an acceptance probability

(14.4)

Hence, whenever we are in state _i_ and a candidate transition from _i_ to _j_ is randomly selected in the simulation, it is actually accepted with probability α _ij_ , which implicitly defines an adjusted probability matrix  , where  , for _i_ ≠ _j_ , and

It is not difficult to show that the pair consisting of π and   satisfies the detailed balance conditions, since:

Therefore, we can easily build a Markov chain with the desired long-run distribution by adjusting any candidate transition matrix **P**. Now, the reader should be a bit puzzled, for a couple of reasons:

**1.** In computational Bayesian statistics, we do not really know the posterior. Hence, how can we select it as the long-run distribution? A look at Eq. (14.4) shows where the trick is: Only the _ratio_ of the probabilities is involved; therefore, even if they are known up to a scaling constant, we can still simulate the chain, and sample from the underlying long-run distribution.

**2.** It may seem very weird that, whatever **P** we choose, it will work. Indeed, so far we did not say anything about efficiency. Not all transition matrices are equally efficient, and only with a suitable choice, we may hope to devise a reasonably fast algorithm.

The Metropolis–Hastings algorithm takes advantage of the above ideas:

  Given a target distribution π, possibly known up to a normalization constant, we select a candidate transition matrix **P**.

  We simulate the resulting Markov chain for a suitably long time horizon.

  The resulting state sequence _X_ _t_ is a random sample from the target distribution.

An important point to bear in mind is that, unlike the random sampling approaches that we have described so far in the book, the sample produced by this procedure is _not_ independent, since the visited states are correlated. This must be taken into due account when performing output analysis, which may require the batching method that we have discussed in Section 7.1. Furthermore, since the target density is achieved in the long run, we should discard the initial portion of the sample path, which is called the _burn-in_ phase.

We have discussed the Metropolis–Hastings algorithm in a discrete-state setting, but when the parameter to be estimated is continuous, we should simulate a continuous-state Markov process. We will not analyze in detail this extension, as it would not provide us with much additional insight.2 The major change is that densities and integrals are involved, rather than probability mass functions and sums. Since the probability of a single point within a continuous state is always zero, we should work with subsets within the state space, over which we integrate the density. Hence, the equilibrium condition of Eq. (14.2) is replaced by the following one:

(14.5)

for any (measurable) subset _B_ of the state space. Here, π( **x** ) is the equilibrium density, so that the integral on the left-hand side is the long-run probability of being in set _B._ The role of the transition matrix is played by the transition kernel function _P_ ( **x** , _B_ ), which gives the probability of moving from state **x** to set _B_. The interpretation of Eq. (14.5) is the same as Eq. (14.2).

Now, in order to be more concrete, let us apply the above ideas to a Bayesian inference for a single-parameter problem. The state space in the Markov process is the set of possible values of parameter θ and, given a vector of observations with value **x** , we want to sample from the posterior density _p_ _n_ (θ | **x** ). To this aim, we have to define a transition kernel satisfying the detailed balance equation, which in practical terms requires the definition of a transition density  (θ, θ') that satisfies the detailed balance condition

By integrating the transition density  (θ,θ') for θ' in a set _B_ , we obtain the corresponding transition kernel, _P_ (θ, _B_ ). The transition density is used to move from state θ to another state θ' by random sampling.3 In order to satisfy the balance condition, we use the Metropolis–Hastings trick: Start with any transition density _q_ (θ, θ'), and enforce the balance condition by multiplying it by

(14.6)

Once again, we notice that we only need the posterior up to a constant, as whatever constant we multiply the posterior by, it gets canceled in the above ratio. The algorithm runs as follows:

**1.** Choose an initial state θ0 and repeat the following for _k_ = 1,..., _M_ :

**(a)** Select a candidate θ' by sampling from the transition density _q_ (θ _k_ −1, θ').

**(b)** Accept the candidate with probability α(θ _k_ −1, θ'); if accepted, set θ _k_ = θ'; otherwise, set θ _k_ = θ _k_ −1.

The random acceptance mechanism is simulated by sampling a uniform variable _U_ ~ U(0, 1) and accepting the candidate if _U_ ≤ α(θ _k_ −1, θ').

Last but not least, to get the whole thing going we need a last piece of the puzzle: the transition density _q_ (θ, θ'). There are two basic possibilities:

**1.** The _random walk_ candidate density, whereby we choose a density function _h_ (·), symmetric with respect to the origin, and set

With this choice, we have _q_ (θ, θ') = _q_ (θ', θ), because of symmetry. Hence, Eq. (14.6) simplifies to

(14.7)

The implication is that if we move from state θ to a state θ' with a larger (unscaled) posterior, the move is always accepted, as suggested by intuition. The acceptance probability is less than 1 when we move "downhill."

**2.** The _independent_ density, whereby we choose the next candidate state according to a PDF

which does not depend on the current state θ. With such a choice, Eq. (14.6) can be rewritten as

(14.8)

Here, the acceptance probability depends on the ratio of the posterior densities, but also on the ratio of the candidate densities. The former ratio tends to favor uphill moves in terms of the posterior; the latter one tends to favor moves toward the tails of the candidate density, where _h_ (θ') is small. This feature may induce a better exploration behavior.

Finding a good candidate density requires finding a satisfactory trade-off between two requirements. On the one hand, we want to explore the state space, which means that, for a given θ, we should be free to move outside its close neighborhood. On the other hand, we do not want to generate candidates that are too likely to be rejected. As the reader can imagine, striking a satisfactory balance may take some experimentation. Another issue is the potential bias introduced by the arbitrary choice of θ0. As we have mentioned, this issue may be overcome by allowing a burn-in, i.e., by letting the system run for a while, before starting the collection of relevant statistics.

###   Example 14.1 Implementing random walk MCMC in R

Let us implement the above ideas in R, using a mixture of two normals as the target density. We assume that the posterior is proportional to

which is not a proper density, as we multiply it by 20, rather than dividing by √2π. As the candidate density, we use a random walk transition density N(0, 3). We first run a 1000 step burn-in phase, where no statistics are collected, starting from some arbitrary initial state θ0; then we generate a sample of size 50,000, whose histogram is plotted against the true density. The R code implementing a naive version of this random walk Metropolis–Hastings algorithm is displayed in Fig. 14.4, and it produces the histogram plotted in Fig. 14.5. As we see, there is a good agreement between the true density and the sampled one.

**FIGURE 14.4** **An illustration of the Metropolis–Hastings algorithm.**

**FIGURE 14.5** **Histogram produced by the sampler ofFig. 14.4.**

## 14.3.1 THE GIBBS SAMPLER

The Metropolis–Hastings algorithm is the starting point for several types of samplers, most notably the Gibbs sampler, which may be useful when we are dealing with multiple parameters, i.e., with a Markov chain featuring a vector state variable. Consider a vector of parameters

Also let us denote by θ- _j_ the vector of parameters with θ _j_ removed:

Since sampling directly from the multivariate distribution can be difficult, we may consider sampling each parameter in turn, while keeping the remaining ones fixed. This means that we should use a transition kernel based on a density _q_ (θ _j_ , θ' _j_ |θ- _j_ ) The idea can also be applied to blocks of parameters, rather than single ones, and is known as blockwise Metropolis–Hastings algorithm.

The Gibbs sampler is a specific case of blockwise Metropolis–Hastings algorithm, where we use the exact conditional density as the candidate density. One possible version of the algorithm is as follows:

  Let _f_ (θ) be the joint density and _f_ _j_ (θ _j_ | θ- _j_ ) be the conditional density for component _j_ = 1,..., _m_.

  Given the current state θ( _k_ ) at step _k_ , we generate the next state θ( _k_ +1) iteratively:

**–** Sample θ1( _k_ +1) ~ _f_ 1(θ1 | θ2( _k_ ),..., θ _m_ ( _k_ )).

**–** Sample θ2( _k_ +1) ~ _f_ 2(θ2 | θ1( _k_ +1), θ3( _k_ ),..., θ _m_ ( _k_ )).

**–**...

**–** Sample θ _m_ ( _k_ +1) ~ _f_ _m_ (θ _m_ | θ1( _k_ +1), θ2( _k_ +1),..., θ _m_ −1( _k_ +1)).

Thus, we see that Gibbs sampling is an iterative sampling strategy that should be contrasted with one-step sampling from a multivariate distribution. Furthermore, since we are using the true conditional density as the candidate, there is no need to check acceptance, as the acceptance probability is always 1. An alternative version of the Gibbs sampler, rather than cycling systematically through each variable, selects one dimension at a time randomly for sampling. The approach makes sense when conditional distributions are available, which is the case, e.g., for hierarchical models.

###   Example 14.2 An illustration of Gibbs sampling

As a very simple illustration, let us implement a Gibbs sampler for a bivariate normal distribution

We know from Section 3.3.4.1 that the two conditional distributions are:

The code in Fig. 14.6 implements the corresponding Gibbs sampler. The function receives the sample size, the length of the burn-in period, and the correlation. The output is used by the script to produce the scatterplot matrix, with histograms on the diagonal, shown in Fig. 14.7. The result is in fact what we would expect from such a bivariate normal distribution.

**FIGURE 14.6** **A simple Gibbs sampler.**

**FIGURE 14.7** **Scatterplot matrix, with histograms on the diagonal, produced by the Gibbs sampler ofFig. 14.6.**

# 14.4 A re-examination of simulated annealing

The Metropolis–Hastings algorithm should ring some bells for readers familiar with simulated annealing, which we described in Section 10.3.2. In both algorithms we generate a candidate state, which is accepted with some probability. With the random walk candidate density, the acceptance probability in the Metropolis–Hastings algorithm is given by Eq. (14.7); uphill moves are always accepted, whereas the acceptance of downhill moves is random. When applying simulated annealing to a minimization problem,

the roles of uphill and downhill moves are swapped:

  Given the current solution  , with cost  , an alternative (candidate) solution   is sampled in the neighborhood of  .

  The candidate is accepted with probability

(14.9)

where   and _T_ is a control parameter playing the role of a temperature.

The acceptance probability is 1 if Δ _f_ < 0, i.e., the candidate solution improves the current one; otherwise, we have an uphill move, which is accepted with a probability depending on _T_. When _T_ is large, it is relatively easy to accept an uphill move, whereas the system tends to favor only improvements when the temperature is low.

The similarity with the Metropolis–Hastings algorithm is in fact a deep one, as simulated annealing can be regarded as a MCMC optimization method, whose aim is to sample from a distribution concentrated around the global optimizers. Let us define the set of global optimizers

The optimal solution need not be unique, so let us denote the cardinality of this set by | _M_ |. Consider the target PMF:

(14.10)

for a given parameter λ > 0. Note that we may speak of a PMF if the set _S_ is finite or enumerable, as it happens in combinatorial optimization problems; we assume that this is the optimization problem we are dealing with, rather than a continuous and nonconvex problem. Note that this PMF favors low-cost solutions, for which the negative exponential is larger. In fact, when λ → +∞, the above PMF tends to

where the indicator function **1** _M_ ( **x** ) is **1** if **x** is optimal, zero otherwise. To see this, it is sufficient to multiply both numerator and denominator in Eq. (14.10) by _e_ −λ _f_ *, which yields:

where \ denotes set difference. By letting λ → +∞, we obtain a very desirable distribution assigning the same positive probability to optimal solutions, and zero to nonoptimal solutions.

Now we have to define a candidate density to generate an alternative candidate solution  . A simple approach is to sample randomly the neighborhood   of the current solution; the candidate PMF assigns a uniform probability,  , with each solution in that neighborhood. Then, given the desired target PMF, the acceptance probability is

If the neighborhood structure is the same for each solution, then the above expression boils down to

which is just Eq. (14.9), provided that we substitute λ = 1/ _T_. This is a reassuring result, as it suggests a convergence guarantee of simulated annealing to a globally optimal solution. However, we need to find a suitable cooling schedule for the temperature. Furthermore, a cynical reader might argue that, if the set of feasible solutions is finite, any complete enumeration algorithm will find an optimal solution in _finite_ time, rather than in the long run. The objection does not necessarily apply if we are considering a continuous nonconvex optimization model, possibly arising from a difficult model calibration problem.

# For further reading

  Bayesian statistics can be a challenging topic, but friendly introductions are provided in [3] and [9]. These books also offer R code.

  A quite readable introduction to computational Bayesian statistics can be found in [4], where variations of acceptance–rejection strategies are also discussed. Among other things, applications to Bayesian linear regression are discussed.

  MCMC methods are not only aimed at Bayesian statistics; a generic treatment can be found in [11, Chapter 10].

  Some further applications of MCMC are discussed in [8, Chapter 6], which includes MATLAB code.

  Readers interested in financial applications might want to learn about the link between Bayesian statistics and portfolio management. The Black–Litterman model, originally introduced in [1], may be interpreted as a Bayesian approach; see also [7].

  The role of Bayesian methods in risk management is illustrated in [2].

  There are a few books on Bayesian econometrics, too; a recent one is [5].

# References

1 F. Black and R. Litterman. Global portfolio optimization. _Financial Analysts Journal_ , 48:28–43, 1992.

2 K. Böecker, editor. _Rethinking Risk Measurement and Reporting: Volumes I and II_. Risk Books, London, 2010.

3 W.M. Bolstad. _Introduction to Bayesian Statistics_ (2nd ed.). Wiley, Hoboken, NJ, 2007.

4 W.M. Bolstad. _Understanding Computational Bayesian Statistics_. Wiley, Hoboken, NJ, 2010.

5 E. Greenberg. _Introduction to Bayesian Econometrics_ (2nd ed.). Cambridge University Press, Cambridge, 2012.

6 W.K. Hastings. Monte Carlo sampling methods using Markov chains. _Biometrika_ , 89:731–743, 1970.

7 G. He and R. Litterman. The intuition behind Black–Litterman model portfolios. Technical report, _Investment Management Research_ , Goldman & Sachs Company, 1999. A more recent version can be downloaded from <http://www.ssrn.org>.

8 D.P. Kroese, T. Taimre, and Z.I. Botev. _Handbook of Monte Carlo Methods_. Wiley, Hoboken, NJ, 2011.

9 J.K. Kruschke. _Doing Bayesian Data Analysis: A Tutorial with R and BUGS_. Academic Press, Burlington, MA, 2011.

10 N.A. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller, and E. Teller. Equations of state calculations by fast computing machines. _Journal of Chemical Physics_ , 21:1087–1092, 1953.

11 S. Ross. _Simulation_ (2nd ed.). Academic Press, San Diego, CA, 1997.

1See [10] for an early reference. The idea was refined and improved in [6].

2See, e.g., [4, pp. 129–130] for a readable proof.

3Some authors use the notation _q_ (θ' | θ) to clarify the role of the two arguments.
**Index**

absolute loss

acceptance–rejection

ACF, _see_ autocorrelation, function

action (in dynamic programming)

additive model

ADP, _see_ dynamic programming, approximate

antithetic sampling

APT, _see_ arbitrage pricing theory

AR, _see_ autoregressive

arbitrage

opportunity

pricing theory

arc (in a network)

ARCH

model

ARIMA

ARMA

asset pricing

asset–liability management

association, linear

autocorrelation

function

sample

partial, _see_ partial autocorrelation

autocorrelogram

autocovariance

autoregressive

conditionally heteroskedastic, _see_ ARCH

integrated moving-average, _see_ ARIMA

model

moving-average, _see_ ARMA

process

backshift operator

balance condition for Markov chains

detailed

global

bandwidth

barrier option, _see_ option, barrier

basis

function

of a lattice

batch method

batching

Bayes' theorem

Bayesian

computational statistics

estimation

parameter estimation

statistics

view

bequest, utility from

Bernoulli

random variable

trial

beta

distribution

function

variate generation

bias

big- _M_ constraint

binomial

coefficient

distribution

lattice

model

random variable

black swan

Black–Scholes–Merton formula

partial differential equation

world

Boltzmann constant

bond

bootstrap

bootstrapping

bond yields

Box–Jenkins model

Box–Muller method

branching factor

Brownian bridge

BSM, _see_ Black–Scholes–Merton

Buffon's needle

burn-in

càdlàg function

capital asset pricing model

CAPM, _see_ capital asset pricing model

cardinality constraint

Cartesian product

CDF, _see_ cumulative distribution function

central limit theorem

for quantiles

chain rule

chance-constrained model

chi-square

distribution

noncentral

random variable

test

variable

Cholesky

factor

factorization

CIR, _see_ Cox–Ingersoll–Ross

code vectorization

coefficient of determination

coefficient of variation

collocation

method

node

combinatorial optimization

common random numbers

complete market

concave

function

optimization problem

problem

concordance

conditional

density

expectation

Monte Carlo

probability

value-at-risk, _see also_ value-at-risk, conditional

variance

conditioning

confidence interval

half-length

confidence level

conjugate family (of priors)

conjugate prior

consumption–saving problem

continuation

region

value

continuous-time model

continuously compounded interest rate

contour plot

control variable

control variate

convex

function

polyhedral

strictly

hull

optimization

optimization problem

set

convexity

cooling schedule

copula

_t_

estimation

Gaussian

normal

product

Student's _t_

theory

correlation

coefficient

instantaneous

negative

Pearson's

sample

testing significance

cost-to-go

counting process

covariance

matrix

sample

coverage (in confidence intervals)

covered position

Cox–Ingersoll–Ross model

Cramér–Rao bound

credible interval

cross-entropy method

cross-sectional

dependence

dependency

model

cumulant generating function

cumulative distribution function

empirical

joint

curse of dimensionality

CV@R, _see_ value-at-risk, conditional

decision rules

decision variable

binary

logical

semicontinuous

default

degrees of freedom

delta hedging

density

candidate

instrumental

target

derivative

financial

over-the-counter

derivative-free method

destandardization

deterministic equivalent

detrending

diagonal model

differencing

second-order

differential equation

digital shift

dimensionality reduction

direction number

discount factor

subjective

discrepancy

star

discrete-event model

discrete-time model

discretization error

distribution, _see also specific distribution_

cumulative function

beta

bimodal

binomial

conditional

cumulative function joint

discrete empirical

elliptical

lognormal

multivariate normal

right-skewed

skewed

uniform

disturbance

diversification

DP, _see_ dynamic programming

drift

duration

dynamic programming

approximate

numerical

stochastic

early exercise

boundary

efficient market hypothesis

eigenvalue

eigenvector

elliptical distribution

EMH, _see_ efficient market hypothesis

empirical distribution

endogenous uncertainty

envelope theorem

epsilon-greedy policy

equilibrium condition

Erlang

distribution

random variable

error

absolute

discretization

relative

sampling

estimation error

estimator

biased

consistent

controlled

efficient unbiased

high-biased

low-biased

point

stratified

unbiased

Euler discretization scheme

Euler–Maruyama discretization scheme, _see_ Euler

excess return

exclusive OR

exogenous uncertainty

expected value

of a function

explicit scheme

exploitation

exploration

exponential

distribution

memoryless property

random variable

variable

exponential tilting

exponential twisting, _see_ exponential tilting

_F_ distribution

factor analysis

factor model

factorial

factorization, Cholesky

fat solution

fat tail

Faure sequence

feasible

region

set

feedback policy

Feynman–Ka  formula

financial asset

finite difference

central

forward

firefly algorithm

frequentist probability

Fubini theorem

function

2-increasing

grounded

indicator

piecewise linear

functional equation

fundamental pricing equation

gamma

distribution

function

and factorials

random variable

GARCH

Gauss–Hermite

quadrature

Gaussian

copula

distribution

process

quadrature, _see also_ quadrature, Gaussian

white noise

GBM, _see_ geometric Brownian motion

generalized autoregressive conditionally heteroskedastic model, _see_ GARCH

generalized feedback shift registers

generalized inverse

genetic algorithm

geometric

distribution

random variable

geometric Brownian motion

bidimensional

multidimensional

GFSR, _see_ generalized feedback shift registers

Gibbs sampler

global optimization

goodness of fit

test

graph optimization

Gray code

grounded function

half-length (of a confidence interval)

Halton sequence

Hardy–Krause variation

hedging strategy

Hessian matrix

Heston model

heteroskedasticity

homoskedasticity

hypercube

hypothesis

alternative

null

i.i.d. (independent and identically distributed)

implicit scheme

importance sampling

income uncertainty

incomplete market

increment

stationary and independent

independence

vs. lack of correlation

independent increment

indicator function

inferential statistics

inner product

integer programming

integral

multidimensional

Riemann

stochastic

integrating factor

integration error

interest rate

continuously compounded

instantaneous

modeling

interior point methods

interpolation

piecewise linear

intrinsic value

inverse problem

inverse transform method

It

isometry

lemma

process

stochastic differential equation

stochastic integral

Jacobian

determinant

matrix

Jarque–Bera test

Jensen's inequality

joint

probability density function

probability mass function

jump

jump–diffusion process

Kendall's tau

kernel density

kernel-based estimation

Koksma–Hlawka theorem

Kolmogorov–Smirnov

measure of fit

test

Korobov point set

kurtosis

excess

labor income

lack of memory (exponential distribution)

Lagrange multiplier

Lagrange polynomial

Lagrangian function

large numbers

strong law of

Latin hypercube sampling

lattice

binomial

calibration

generator

projection regular

rank- _r_ rule

structure in LCG

law of iterated expectations

law of one price

LCG, _see_ linear congruential generator

least-squares

nonlinear problem

Levenberg–Marquardt method

Lévy flight

Lévy–It  decomposition

Lévy process

likelihood

function

log-function

method of maximum

likelihood ratio

derivative estimation

method

linear congruential generator

linear programming

canonical form

mixed integer

standard form

linear regression

probabilistic view

linearity

local improvement

local maxima

local search

best-improving

first-improving

log-likelihood function

log-return

logarithm

lognormal

distribution

random variable

longitudinal

dependence

model

loss function

low-discrepancy sequence

scrambled

LP, _see_ linear programming

lurking variable

margin

marginal

cumulative distribution function

density

distribution

probability density function

market

complete

incomplete

market portfolio

Markov

chain

aperiodic

continuous-time

discrete time

discrete-time

irreducible

long-run distribution

time reversible

decision process

process

Markov chain Monte Carlo

Markovian

dynamic system

process

Markowitz portfolio model

mathematical programming

matrix

positive definite

positive semidefinite

transposition

maximization problem

maximum-likelihood

estimation

estimator

method

properties of estimator

MCMC, _see_ Markov chain Monte Carlo

MDP, _see_ Markov decision process

mean reversion

mean reverting process

mean–risk decision model

mean–variance

efficient portfolio

portfolio optimization

median

memoryless property (exponential distribution)

Mersenne

number

twister generator

metaheuristic

metamodel

metamodeling

method of batches

method of moments

Metropolis–Hastings algorithm

blockwise

MILP, _see_ linear programming, mixed integer

Milstein discretization scheme

MLE, _see_ maximum-likelihood estimator

mod (math operator)

mode

model calibration

model identification

modulus (in random number generation)

moment

first-order

sample

second-order

moment generating function

normal distribution

moment matching

monomial basis

monotonicity

monotonicity (requirement for variance reduction)

Monte Carlo integration

Monte Carlo simulation as an integration problem

structure of

moving-average model

moving-average process

multiplicative model

multiplier (in random number generation)

naked position

neighborhood structure

Nelder–Mead method

network optimization

newsvendor problem

Newton's method

Newton–Cotes quadrature formula

node (in a network)

node (in a quadrature formula)

nonanticipativity

noncentrality parameter

nonconvex

optimization

optimization problem

set

nonlinear least squares

nonlinear programming

nonparametric test

nonstationary process

normal

bivariate distribution

copula

distribution

mixture of

standard

moment generating function

multivariate conditional distribution

multivariate variable

random variable

standard distribution

standard random variable

standard variable

variate

numerical integration, _see_ quadrature

numerical optimization

objective function

OLS, _see_ ordinary least squares

optimal control, continuous-time

optimality equation

optimization

combinatorial

constrained problem

finite-dimensional

model

nonconvex

unconstrained problem

under uncertainty

option

American-style

as-you-like-it

Asian

arithmetic average

geometric

geometric average

Asian-style

barrier

Bermudan-style

call

chooser

continuation value

deeply out-of-the-money

delta

digital call

down-and-in put

down-and-out put

European-style

exchange

gamma

greek

holder

in-the-money

intrinsic value

lookback

lookback call

multidimensional

out-of-the-money

path dependent

path-dependent

pricing

sensitivity

spread

theta

vanilla call

vega

writer

order statistics

ordinary least squares

Ornstein–Uhlenbeck process

orthodox statistics

orthogonal

matrix

polynomial

overfitting

overflow

PACF, _see_ partial autocorrelation function

panel

data

model

parameter estimation

Bayesian

parity

for barrier options

put–call

partial autocorrelation function

sample

partial differential equation

particle swarm optimization

passive portfolio management

pathwise derivative estimation

pathwise differentiation

PCA, _see_ principal component analysis

PDE, _see_ partial differential equation

PDF, _see_ probability density function

penalty function

periodicity (in random number generators)

permanent shock

perturbation analysis

PMF, _see_ probability mass function

point estimate

point estimator

Poisson

distribution

process

compound

inhomogeneous

random variable

polar

coordinate

rejection

policy iteration

polyhedral set

polyhedron

polynomial

interpolation

primitive

polynomial regression

portfolio

benchmark

compression

tracking

position

covered

naked

positive homogeneity

post-decision

value function

posterior

density

distribution

predictor-corrector method

prime number

principal component

principal component analysis

prior

density

distribution

probability

density function

joint

mass function

marginal

measure

risk-neutral

process

adapted

standard Wiener

Wiener

nondifferentiability of

product rule

projection regular (lattice)

pseudorandom, _see also_ random number

number generator

state

PSO, _see_ particle swarm optimization

put–call parity

Pythagorean theorem

_Q_ -factor

_Q_ -learning

QP, _see_ quadratic programming

Q-Q plot

quadratic form, convex

quadratic programming

quadrature

formula

closed

composite

Gauss–Laguerre

Gauss–Legendre

Newton–Cotes

open

order

Gauss–Hermite

Gaussian

product rule

trapezoidal rule

quantile

estimated

estimation

of normal distribution

quantum PSO

quasi–Monte Carlo

quasi–random number

Radon–Nikodym derivative

random number, _see also_ pseudorandom

random number generator

combined

generalized feedback shift registers

Mersenne twister

multiple recursive

multiplicative

separate stream

Tausworthe

Wichman–Hill

random sample

random shift

random shock

random variable, _see also specific random variable_

continuous

discrete

independent

jointly continuous

lognormal

normal

standard normal

uniform

random walk

rank correlation

Rastrigin function

rate

continuously compounded

risk-free

recourse

function

matrix

simple

variable

rectangle rule

recursive equation

recursive functional equation

rejection region

rejection sampling, _see_ acceptance-rejection

replicating portfolio

replication

residual

response surface

return

continuously compounded

rate of

Richardson extrapolation

risk

aversion

idiosyncratic

management

measure

coherent

specific

systematic factor

risk-neutral

measure

option pricing

pricing

probability measure

SACF, _see_ autocorrelation function, sample

sample

correlation

covariance

mean

partial autocorrelation function

path

variance

sampling vs. simulation

scale parameter

scatterplot

scenario

extreme

generation

tree

score function

scrambling

Faure–Tezuka

of a low-discrepancy sequence

Owen

SDE, _see_ stochastic differential equation

seasonality

seed

setting in random number generators

semiannual compounding

semicontinuous variable

sensitivity estimation

shape parameter

Shapiro–Wilk test

shift (in random number generation)

short rate

short selling

shortest path problem

shortfall probability

simplex

algorithm

method

search

Simpson's rule

simulated annealing

generalized

simulation-based optimization

skewness

negative

Sklar's theorem

small world PSO

Sobol sequence

SP, _see_ stochastic programming

SPACF, _see_ partial autocorrelation function, sample

Spearman's rho

spline

cubic

natural

square-root diffusion

square-root matrix

square-root rule

stability

in sample

out of sample

stable distribution

standard deviation

of return

standard normal distribution

standard Wiener processes

standardization

star discrepancy

state transition

equation

function

state variable

post-decision

stationary and independent increments

stationary increment

steepest descent

stochastic calculus

stochastic differential

stochastic differential equation

stochastic dynamic optimization

stochastic dynamic programming

stochastic integral

stochastic optimization

stochastic process

mean reverting

nonstationary process

predictable

stationary

weakly stationary

stochastic programming

multistage with recourse

two-stage

with chance constraints

with recourse

stochastic search

stochastic volatility

stop-loss hedging

stopping time

strategic oscillation

stratification

stratified sampling

stream, in random number generators

strike price

Student's _t_

distribution

multivariate distribution

random variable

subadditivity

subjective

knowledge

probability

sufficient statistics

surface plot

tabu search

tail dependence

tail risk

Taylor expansion

_t_ distribution multivariate

term structure of interest rates

TEV, _see_ tracking error variance

tilting parameter

time bucket

time consistency

time series

analysis

model

total probability theorem

tracking error variance

tracking portfolio

transaction cost

transient phase

transition

density

kernel

matrix

probability

matrix

transitory shock

translation invariance

transposition

uncorrelated variables

uniform

random variable

unit hypercube

unk-unks (unknown unknowns)

utility

expected

function

quadratic

utility function

logarithmic

power

value function

value-at-risk

absolute

conditional, _see also_ conditional, value-at-risk

historical

relative

Van der Corput sequence

VAR, _see_ vector autoregressive model

V@R, _see_ value-at-risk

variance

of a sum of random variables

variance reduction

Vasicek model

vector autoregressive model

vector transposition

volatility

cluster

stochastic

volume risk

weakly stationary process

wealth

Weierstrass theorem

weight function

weights (in a quadrature formula)

white noise

Gaussian

whitening

Wichman–Hill generator

Wiener process

bidimensional

correlated

differential

generalized

lack of differentiability

standard

yield curve

zero-coupon bond
Wiley Handbooks in

**FINANCIAL ENGINEERING AND ECONOMETRICS**

Advisory Editor

**Ruey S. Tsay**

_The University of Chicago Booth School of Business, USA_

The dynamic and interaction between financial markets around the world have changed dramatically under economic globalization. In addition, advances in communication and data collection have changed the way information is processed and used. In this new era, financial instruments have become increasingly sophisticated and their impacts are far-reaching. The recent financial (credit) crisis is a vivid example of the new challenges we face and continue to face in this information age. Analytical skills and ability to extract useful information from mass data, to comprehend the complexity of financial instruments, and to assess the financial risk involved become a necessity for economists, financial managers, and risk management professionals. To master such skills and ability, knowledge from computer science, economics, finance, mathematics and statistics is essential. As such, financial engineering is cross-disciplinary, and its theory and applications advance rapidly.

The goal of this Handbook Series is to provide a one-stop source for students, researchers, and practitioners to learn the knowledge and analytical skills they need to face today's challenges in financial markets. The Series intends to introduce systematically recent developments in different areas of financial engineering and econometrics. The coverage will be broad and thorough with balance in theory and applications. Each volume will be edited by leading researchers and practitioners in the area and covers state-of-the-art methods and theory of the selected topic.

### Published Wiley Handbooks in Financial Engineering and Econometrics

Bauwens, Hafner, and Laurent · _Handbook of Volatility Models and Their Applications_

Brandimarte · _Handbook in Monte Carlo Simulation: Applications in Financial Engineering, Risk Management, and Economics_

Chan and Wong · _Handbook of Financial Risk Management: Simulations and Case Studies_

James, Marsh, and Sarno · _Handbook of Exchange Rates_

Viens, Mariani, and Florescu · _Handbook of Modeling High-Frequency Data in Finance_

Szylar · _Handbook of Market Risk_

### Forthcoming Wiley Handbooks in Financial Engineering and Econometrics

Bali and Engle · _Handbook of Asset Pricing_

Chacko · _Handbook of Credit and Interest Rate Derivatives_

Cruz, Peters, and Shevchenko · _Handbook of Operational Risk_

Florescu, Mariani, Stanley, and Viens · _Handbook of High-Frequency Trading and Modeling in Finance_

Jacquier · _Handbook of Econometric Methods for Finance: Bayesian and Classical Perspectives_

Longin · _Handbook of Extreme Value Theory and Its Applications to Finance and Insurance_

Starer · _Handbook of Equity Portfolio Management: Theory and Practice_

Veronesi · _Handbook of Fixed-Income Securities_

redpajama_set_name: RedPajamaBook