text:
Journal cover Journal topic
Hydrology and Earth System Sciences An interactive open-access journal of the European Geosciences Union
Journal topic
Hydrol. Earth Syst. Sci., 22, 4867-4873, 2018
https://doi.org/10.5194/hess-22-4867-2018
Hydrol. Earth Syst. Sci., 22, 4867-4873, 2018
https://doi.org/10.5194/hess-22-4867-2018

Research article 18 Sep 2018

Research article | 18 Sep 2018

# Cross-validation of bias-corrected climate simulations is misleading

Douglas Maraun1 and Martin Widmann2 Douglas Maraun and Martin Widmann
• 1Wegener Center for Climate and Global Change, University of Graz, Brandhofgasse 5, 8010 Graz, Austria
• 2School of Geography, Earth and Environmental Sciences, University of Birmingham, Birmingham, B15 2TT, UK
Abstract

We demonstrate both analytically and with a modelling example that cross-validation of free-running bias-corrected climate change simulations against observations is misleading. The underlying reasoning is as follows: a cross-validation can have in principle two outcomes. A negative (in the sense of not rejecting a null hypothesis), if the residual bias in the validation period after bias correction vanishes; and a positive, if the residual bias in the validation period after bias correction is large. It can be shown analytically that the residual bias depends solely on the difference between the simulated and observed change between calibration and validation periods. This change, however, depends mainly on the realizations of internal variability in the observations and climate model. As a consequence, the outcome of a cross-validation is also dominated by internal variability, and does not allow for any conclusion about the sensibility of a bias correction. In particular, a sensible bias correction may be rejected (false positive) and a non-sensible bias correction may be accepted (false negative). We therefore propose to avoid cross-validation when evaluating bias correction of free-running bias-corrected climate change simulations against observations. Instead, one should evaluate non-calibrated temporal, spatial and process-based aspects.

1 Introduction

Bias correction is a widely used approach to postprocess climate model simulations before they are applied in impact studies (Gangopadhyay et al.2011; Girvetz et al.2013; Hagemann et al.2013; Warszawski et al.2014). A wide range of different correction methods has been developed, ranging from simple additive or multiplicative corrections to quantile-based approaches. For reviews of bias correction see , and the book by .

The performance of a bias correction is typically evaluated against independent observational data, which have not entered the calibration of the correction function. For instance, , and apply the holdout method, i.e. they calibrate the method on a calibration period and evaluate it on a non-overlapping validation period. Some authors even apply a full cross-validation, most often by permuting calibration and validation period (Gudmundson et al.2012).

Cross-validation is a well-known and widely used statistical concept to assess the skill of predictive statistical models . It has been successfully applied in the atmospheric sciences, e.g. in weather forecasting and perfect predictor experiments of downscaling methods .

In climate change applications, however, the setting is typically different from a weather forecasting or perfect predictor setting: here, the model is running free, i.e. only external forcings are common to observation and simulation. Internal climate variability on all scales is independent and not synchronized. In this setting, the aim is not to assess predictive power, e.g. on a day-by-day or season-by-season basis, as in weather forecasting – in fact, by construction it cannot be assessed. Importantly, observed and simulated long-term trends may also differ substantially, just because of different random realizations of long-term modes of variability. Prominent examples of such modes are the Pacific Decadal Oscillation (Mantua et al.1997) and the Atlantic Multidecadal Oscillation (Schlesinger and Ramankutty1994).

These differences have crucial implications for the application of cross-validation or any evaluation on independent data. Our results build upon a recent study by , who demonstrated that in a climate change setting cross-validation of marginal aspects is not able to identify bias correction skill. Here, we additionally show that the outcome of a cross-validation is essentially random and independent of the sensibility of the bias correction. We will demonstrate these consequences for the holdout method, but they can of course be generalized to any type of cross-validation.

We will discuss the specific context of climate change simulations in Sect. 2. An analytical derivation of the cross-validation problem will be given in Sect. 3, a modelling example in Sect. 4. We will close with a discussion of the implications of our findings.

2 Cross-validation in the climate context

Cross-validation was developed to quantify the predictive skill of statistical models in the 1930s, and has become widely used with the advent of modern computers . It has become a standard tool in weather and climate forecasting .

The first major aim of cross-validation is to eliminate artificial skill: if the statistical model is evaluated on the same data that are used for calibration, the performance to predict new data will almost certainly be lower than the estimated skill. Hence, the model is calibrated only on a subset of the data, and evaluated on another – ideally independent – subset of the data. This so-called holdout method, however, uses each data point only either for calibration or validation and thus suffers from relatively high sampling errors.

The second major aim of cross-validation is therefore to use the data optimally. To this end, the holdout method, i.e. training and validation, is repeated on different subsets of the data. The simplest approach is the so-called split sample method, where the data are just split once into two subsets. More advanced k-fold cross-validation splits the data set into k non-overlapping blocks; in each fold, k1 blocks are used for calibration and the remaining block is used for validation.

In weather and climate predictions, the aim is to predict the weather, i.e. internal variability, with a given lead time (say, 3 days or a season) at a desired timescale (say, 6 h or a season). A typical evaluation assesses how well certain meteorological aspects are predicted: in weather forecasting, one may for instance be interested in the overall prediction accuracy, measured by the root-mean squared error between predicted and observed daily time series. In a seasonal prediction, one may be interested in the bias of the predicted mean, or in the bias of the predicted wet-day frequency over a season. In this context, a cross-validation makes perfect sense if the validation blocks are long compared to the prediction lead time (and process memory).

Downscaling and bias correction methods are typically tested in perfect predictor or perfect boundary condition experiments , where predictors or boundary conditions are taken from reanalysis data. The aim of the downscaling in this context is not to predict internal variability ahead into the future, but rather to predict the local weather conditional on the state of the large-scale weather (i.e. to simulate the correct local long-term weather statistics). Still, in such a setting cross-validation makes perfect sense: the choice of reanalysis data as predictors/boundary conditions synchronizes simulated and observed local variability on timescales beyond a few weeks, such that the evaluation framework is similar to the case of seasonal prediction.

In free-running climate simulations, however, the situation is fundamentally different: here, any predictive power results only from external (e.g. anthropogenic) forcing at very long timescales, but internal variability is not synchronized at any timescale. Yet long-term modes of internal climate variability, such as the PDO and the AMO , often mask forced climate trends even at multidecadal timescales . Thus, much of the difference between observed and simulated trends is not caused by model errors, but rather by random fluctuations of the climate system. This fact has strong implications for the evaluation of simulated trends , but it is also the reason why cross-validation of bias correction fails in this context.

As any cross-validation consists of repeated holdout evaluations, we will in the following only consider the holdout method. In Sect. 5 we will discuss how the following results generalize to a full cross-validation.

3 Analytical derivation

Consider a simulated time series xi and an observed time series yi. Assume that an evaluation addresses the representation of some statistic such as the long-term mean. Over the calibration period, we denote the simulated and observed means as ${\stackrel{\mathrm{‾}}{x}}_{\text{cal}}$ and ${\stackrel{\mathrm{‾}}{y}}_{\text{cal}}$, respectively. Correspondingly, we denote them as ${\stackrel{\mathrm{‾}}{x}}_{\text{val}}$ and ${\stackrel{\mathrm{‾}}{y}}_{\text{val}}$ over the validation period. Then an estimate for the bias over the calibration period is given as

$\begin{array}{}\text{(1)}& \text{BIAS}={\stackrel{\mathrm{‾}}{x}}_{\text{cal}}-{\stackrel{\mathrm{‾}}{y}}_{\text{cal}}.\end{array}$

Applying the bias estimate to the validation period, one obtains an estimate of the corrected mean over the validation period:

$\begin{array}{}\text{(2)}& {\stackrel{\mathrm{‾}}{x}}_{\text{val}}^{\text{corr}}={\stackrel{\mathrm{‾}}{x}}_{\text{val}}-\text{BIAS}={\stackrel{\mathrm{‾}}{x}}_{\text{val}}-{\stackrel{\mathrm{‾}}{x}}_{\text{cal}}+{\stackrel{\mathrm{‾}}{y}}_{\text{cal}}.\end{array}$

The remaining residual bias is then

$\begin{array}{}\text{(3)}& {\text{BIAS}}_{\text{res}}={\stackrel{\mathrm{‾}}{x}}_{\text{val}}^{\text{corr}}-{\stackrel{\mathrm{‾}}{y}}_{\text{val}}={\stackrel{\mathrm{‾}}{x}}_{\text{val}}-{\stackrel{\mathrm{‾}}{x}}_{\text{cal}}+{\stackrel{\mathrm{‾}}{y}}_{\text{cal}}-{\stackrel{\mathrm{‾}}{y}}_{\text{val}}.\end{array}$

This residual bias can be expressed in terms of the observed and simulated climate change signals. The change signal from calibration to validation period is defined as

$\begin{array}{}\text{(4)}& \mathrm{\Delta }x={\stackrel{\mathrm{‾}}{x}}_{\text{val}}-{\stackrel{\mathrm{‾}}{x}}_{\text{cal}}\end{array}$

for the model and

$\begin{array}{}\text{(5)}& \mathrm{\Delta }y={\stackrel{\mathrm{‾}}{y}}_{\text{val}}-{\stackrel{\mathrm{‾}}{y}}_{\text{cal}}\end{array}$

for the observations. Thus, the residual bias is given as

$\begin{array}{}\text{(6)}& {\text{BIAS}}_{\text{res}}=\mathrm{\Delta }x-\mathrm{\Delta }y.\end{array}$

For variables such as precipitation, one often considers relative changes. Here a corresponding derivation holds. The relative error is defined as

$\begin{array}{}\text{(7)}& \text{RE}={\stackrel{\mathrm{‾}}{x}}_{\text{cal}}/{\stackrel{\mathrm{‾}}{y}}_{\text{cal}},\end{array}$

and the corrected mean over the validation period is given as

$\begin{array}{}\text{(8)}& {\stackrel{\mathrm{‾}}{x}}_{\text{val}}^{\text{corr}}={\stackrel{\mathrm{‾}}{x}}_{\text{val}}/\text{RE}={\stackrel{\mathrm{‾}}{x}}_{\text{val}}\cdot {\stackrel{\mathrm{‾}}{y}}_{\text{cal}}/{\stackrel{\mathrm{‾}}{x}}_{\text{cal}}.\end{array}$

The residual relative error results in

$\begin{array}{}\text{(9)}& {\text{RE}}_{\text{res}}={\stackrel{\mathrm{‾}}{x}}_{\text{val}}^{\text{corr}}/{\stackrel{\mathrm{‾}}{y}}_{\text{val}}=\frac{{\stackrel{\mathrm{‾}}{x}}_{\text{val}}\cdot {\stackrel{\mathrm{‾}}{y}}_{\text{cal}}}{{\stackrel{\mathrm{‾}}{x}}_{\text{cal}}\cdot {\stackrel{\mathrm{‾}}{y}}_{\text{val}}}.\end{array}$

The relative change signal from calibration to validation period is defined as

$\begin{array}{}\text{(10)}& \mathrm{\Delta }x={\stackrel{\mathrm{‾}}{x}}_{\text{val}}/{\stackrel{\mathrm{‾}}{x}}_{\text{cal}}\end{array}$

for the model and

$\begin{array}{}\text{(11)}& \mathrm{\Delta }y={\stackrel{\mathrm{‾}}{y}}_{\text{val}}/{\stackrel{\mathrm{‾}}{y}}_{\text{cal}}\end{array}$

for the observations. Hence, the residual relative error is

$\begin{array}{}\text{(12)}& {\text{RE}}_{\text{res}}=\mathrm{\Delta }x/\mathrm{\Delta }y.\end{array}$

The residual bias or relative error could further be tested for significance, i.e. whether the bias-corrected statistic ${\stackrel{\mathrm{‾}}{x}}_{\text{val}}^{\text{corr}}$ is significantly different from the observed statistic ${\stackrel{\mathrm{‾}}{y}}_{\text{val}}$ over the validation period. Thus, a holdout evaluation will yield a positive result (in the sense of rejecting the null hypothesis, i.e. a non-zero residual bias) if the simulated change Δx is different from the observed change Δy, and a negative result (i.e. a residual bias compatible with zero) if simulated and observed changes are indistinguishable.

Assume now that a given bias correction may or may not be sensible. Note in this context that it is completely irrelevant to explicitly define what constitutes a sensible bias correction (but for a brief discussion see Sect. 4). Thus, in principle four cases are possible.

1. True negative: the bias correction is sensible, and the (bias-corrected) climate model simulates a trend closely resembling the observed trend.

2. False positive: the bias correction is sensible, but due to internal climate variability, the (bias-corrected) climate model simulates a trend different from the observed trend.

3. False negative: the bias correction is not sensible, but the (bias-corrected) climate model for some reason simulates a trend similar to the observed trend. This case corresponds to the example given in .

4. True positive: the bias correction is not sensible, and the (bias-corrected) climate model simulates a trend different from the observed trend.

The crucial point is that for typical record lengths, much of the difference between simulated and observed changes Δx and Δy will be caused by internal climate variability. Thus the result of a cross-validation, i.e. which of the four cases occurs, is mostly random and says only little about the sensibility of the cross-validation.

considered case 3: as the difference between simulated and observed trends on typical timescales of a few decades is dominated by internal variability, the holdout method is not suitable to identify a non-sensible bias correction. The reverse conclusion is that the holdout method – and consequently also a cross-validation – is not able to corroborate whether a bias correction is sensible.

Yet the discussion above implies an even stronger conclusion: because case 2 might randomly occur, a sensible bias correction may be rejected by a cross-validation. Thus, even more importantly, cross-validation in the given context is not just useless, but even misleading.

4 Empirical demonstration

Figure 1 Maps of relative changes in boreal mean summer (JJA) precipitation, 1981–2005 relative to 1956–1980. (a) EC-EARTH, (b) E-OBS, version 15.0. Square: case 1 (true negative); circle: case 2 (false positive); triangle: case 3 (false negative); diamond: case 4 (true negative).

To further illustrate the analytic findings, we will give examples of the four cases in an exaggerated modelling example. We consider mean summer (JJA) precipitation at four locations. As observational reference we select the E-OBS data set . As calibration period we use 1956–1980, as evaluation period 1981–2005.

We need to select two examples where the given bias correction is sensible, and two where it is not. Finding a convincing example of a sensible bias correction has to rely on process understanding . A major precondition is that the climate model simulates a realistic present climate and a credible climate change . The former condition mainly involves a realistic representation of the large-scale circulation . We therefore consider the following set-up: as examples of a sensible bias correction, we consider summer mean precipitation at two locations in Norway. Summer mean precipitation in Norway is dominated by large-scale precipitation, which can sensibly be assumed to be realistically simulated by current-generation general circulation models (GCMs). Specifically we choose a transient simulation of EC-EARTH , a model which has been demonstrated to suffer from minor biases in the synoptic-scale atmospheric circulation over Europe only . We assume that other potential problems such as mislocations or scale gaps (Maraun2013a) are negligible for the considered locations and timescales. In this setting, we argue that a bias correction is in principle sensible. Two slightly different locations have been selected (Fig. 1): in the Børgefjell region north-east of Trondheim observed and simulated trends are very similar (case 1). Further north, around the town of Bodø, the two trends are very different as observed precipitation has decreased whereas simulated precipitation has increased. It is reasonable to assume that the observed negative trend is caused by internal climate variability and not a forced change. Thus, in principle the bias correction is sensible even though the validation rejects it (case 2). Here, the key point is not whether the bias correction in this particular example is really sensible or not, but that case 2 is indeed a possible – and misleading – outcome of a bias correction.

In the following we show two examples where a bias correction is not sensible. A discussion about the question when a bias correction makes no sense would go very much beyond the scope of this piece. Therefore, we follow the logic of and select examples where model simulation and observation are taken from geographically far away and climatically rather different regions. The underlying idea is that for such cases, the model does not represent the target variable such that a bias correction is without doubt not sensible. Specifically, we consider the following two cases (see Fig. 1): first, mapping simulated boreal summer mean precipitation from the sub-tropical Maputo area (Mozambique, close to the South African border) to the Taiga region of the Norwegian–Finnish boarder. Here, observed and simulated trends are randomly similar. This example is equivalent to that given in , where a non-sensible bias correction is not identified by the validation (case 3). Second, we map summer mean precipitation from the tropical climate at Belén in the Amazon delta to the mild and maritime climate of northern Portugal. Here, observed and simulated trends are randomly very different: positive in the Amazon delta, negative in Portugal. The non-sensible bias correction is thus correctly identified (case 4). In these two examples, it was a priori obvious that a bias correction is not sensible. In real applications, of course, such a priori reasoning will be much more difficult and has to rest upon process understanding (see discussion above). The key point, again, is that case 3 is a possible outcome of a bias correction.

Figure 2 Time series of boreal summer (JJA) precipitation. (a) case 1 (true negative); (a) case 2 (false positive); (c) case 3 (false negative); (d) case 4 (true negative). Black: E-OBS; red: raw EC-EARTH; blue: bias-corrected EC-EARTH. The straight horizontal lines depict the long-term means over calibration and validation periods, respectively.

Figure 2 shows observed and simulated time series, the latter before and after bias correction, for the four cases we considered. Figure 2a and b show the sensible examples, Fig. 2c and d the non-sensible examples. In Fig. 2a and c, observed and simulated trends randomly agree, and in Fig. 2b and d they randomly disagree. As shown analytically in Sect. 3, the residual bias vanishes in cases (a) and (c), where the relative trends in observations and simulations are similar, and it does not vanish in (b) and (d), where the relative trends in observations and simulations disagree. The relevant cases are (b) and (c): in the former, the bias correction is in principle sensible, but the holdout method would suggest that it was not sensible (false positive). In the latter, the bias correction is not sensible, but the holdout method would suggest that it was sensible (false negative). These examples clearly illustrate our previous reasoning: the holdout method, and thus also cross-validation, yields misleading results when it is used to assess the sensibility of bias-corrected climate change simulations against observations.

5 Conclusions

We have demonstrated both analytically and with a modelling example that cross-validation of free-running bias-corrected climate change simulations against observations is misleading. The underlying reasoning is as follows: the result of a cross-validation – a significant or non-significant residual bias in the validation period – depends on the difference between observed and simulated changes between calibration and validation periods. For typical lengths of calibration and validation periods, these differences depend mostly on the realizations of internal variability in the observations and climate model. These differences therefore do not allow for conclusions about the sensibility of a bias correction. As in any setting of significance testing, four cases are possible: true negative, false positive, false negative and true negative. The actual outcome in a given application is mostly random.

The relevance of internal variability in the discussed cross-validation context depends on the relative strength of internal variability and forced trends, and the length of the calibration and validation period compared to the periodicity of the dominant modes of internal climate variability. In tropical climates, interannual variability such as that of El Niño–Southern Oscillation dominates climate variability. Thus, relatively short periods of a few decades may suffice to obtain stable estimates of forced changes between calibration and validation period, and therefore to assess whether a bias correction performs well given the observed changes. In mid-latitude climates, however, the dominant modes of internal variability have periodicities of several decades such that stable estimates of forced changes will in general not be obtained for the typical calibration and validation periods of a few decades only. The strength of internal variability depends also on the chosen variable (e.g. higher for precipitation than for temperature) and index (higher for extremes than for long-term means). Spatial aggregation will reduce regional short-term internal variability (Maraun2013b), but will not affect long-term modes of variability, which typically have coherent spatial patterns.

We have derived these conclusions for the mean and the holdout method, where the bias correction is calibrated against one part of the data and validated against its complement. Yet the results can in principle be transferred to other statistics such as variances or individual quantiles, and to a full cross-validation. The residual mean bias, however, is always zero in a full cross-validation, as long as the individual folds have the same length. The reason is that changing the calibration and validation period changes the sign of the residual bias. When averaging the residual bias across the different folds, it cancels out. For the variance or similar statistics, the outcome depends on the way the cross-validation is carried out: if the residual bias is calculated for each fold separately and then averaged (as suggested in the classical literature), the behaviour is as for the mean. If the residual bias is calculated over a concatenated cross-validated time series (as is typically done in the atmospheric sciences), the bias correction in cases (b) and (d) will yield extremely high residual biases (because the shift in the mean is not removed in the variance calculation).

The consequence of these findings is that cross-validation should not be used when evaluating bias correction of free-running climate simulations against observations. In fact, a framework for evaluating bias correction of climate simulations is still missing and not trivial. As discussed in , we propose to evaluate non-calibrated temporal, spatial and process-based aspects of the simulated time series. Whether a bias correction makes sense for climate change projections depends in particular on the realism and the credibility of the underlying climate model simulation. A process-based evaluation is thus a key prerequisite for a successful bias correction .

Data availability
Data availability.

E-OBS data are available from the ECA&D website at https://www.ecad.eu/ (last access: 21 July 2017). EC-EARTH data are available from the ESGF nodes accessible via https://esgf.llnl.gov/nodes.html (last access: 17 September 2018).

Author contributions
Author contributions.

DM and MW developed the idea for the study. DM conducted the analytical derivations, carried out the analysis, and wrote the manuscript. MW commented on the manuscript. DM and MW discussed the results.

Competing interests
Competing interests.

The authors declare that they have no conflict of interest.

Acknowledgements
Acknowledgements.

This study has been inspired by discussions in EU COST Action ES1102 VALUE.

Edited by: Carlo De Michele
Reviewed by: Uwe Ehret and Seth McGinnis

References

Bhend, J. and Whetton, P.: Consistency of simulated and observed regional changes in temperature, sea level pressure and precipitation, Clim. Change, 118, 799–810, 2013. a

Deser, C., Knutti, R., Solomon, S., and Phillips, A.: Communication of the role of natural variability in future North American climate, Nat. Clim. Change, 2, 775–779, 2012. a

Dosio, A. and Paruolo, P.: Bias correction of the ENSEMBLES high resolution climate change projections for use by impact models: Evaluation on the present climate, J. Geophys. Res. Atmos., 116, D16106, https://doi.org/10.1029/2011JD015934, 2011. a

Efron, B. and Gong, G.: A leisurely look at the bootstrap, the jackknife, and cross-validation, Am. Stat., 37, 36–48, 1983. a, b

Gangopadhyay, S., Pruitt, T., Brekke, L., and Raff, D.: Hydrologic projections for the Western United States, EOS, 92, 441–442, 2011. a

Girvetz, E., Maurer, E., Duffy, P., Ruesch, A., Thrasher, B., and Zganjar, C.: Making climate data relevant to decision making: the important details of spatial and temporal downscaling, The World Bank, 2013. a

Gudmundsson, L., Bremnes, J. B., Haugen, J. E., and Engen-Skaugen, T.: Technical Note: Downscaling RCM precipitation to the station scale using statistical transformations – a comparison of methods, Hydrol. Earth Syst. Sci., 16, 3383–3390, https://doi.org/10.5194/hess-16-3383-2012, 2012. a

Hagemann, S., Chen, C., Clark, D. B., Folwell, S., Gosling, S. N., Haddeland, I., Hanasaki, N., Heinke, J., Ludwig, F., Voss, F., and Wiltshire, A. J.: Climate change impact on available water resources obtained using multiple global climate and hydrology models, Earth Syst. Dynam., 4, 129–144, https://doi.org/10.5194/esd-4-129-2013, 2013. a

Haylock, M., Hofstra, N., Klein Tank, A., Klok, E., Jones, P., and New, M.: A European daily high-resolution gridded data set of surface temperature and precipitation for 1950–2006, J. Geophys. Res., 113, D20119, https://doi.org/10.1029/2008JD010201, 2008. a

Hazeleger, W., Severijns, C., Semmler, T., Ştefănescu, S., Yang, S., Wang, X., Wyser, K., Dutra, E., Baldasano, J., Bintanja, R., Bougeault, P., Caballero, R., Ekman, A., Christensen, J., van den Hurk, B., Jimenez, P., Jones, C., Kållberg, P., Koenigk, T., Mc Grath, R., Miranda, P., van Noije, T., Palmer, T., Parodi, J., Schmith, T., Selten, F., Storelvmo, T., Sterl, A., Tapamo, H., Vancoppenolle, M., Viterbo, P., and Willen, U.: EC-Earth: a seamless earth-system prediction approach in action, B. Am. Meteorol. Soc., 91, 1357–1363, 2010. a

Jolliffe, I. and Stephenson, D., eds.: Forecast Verification: A Practitioner's Guide in Atmospheric Science, Wiley, 2003. a, b

Laprise, R.: Comment on “The added value to global model projections of climate change by dynamical downscaling: A case study over the continental US using the GISS-ModelE2 and WRF models” by Racherla et al., J. Geophys. Res., 119, 3877–3881, 2014. a

Li, H., Sheffield, J., and Wood, E.: Bias correction of monthly precipitation and temperature fields from Intergovernmental Panel on Climate Change AR4 models using equidistant quantile matching., J. Geophys. Res., 115, D10101, https://doi.org/10.1029/2009JD012882, 2010. a

Mantua, N. J., Hare, S., Zhang, Y., Wallace, J., and Francis, R.: A Pacific interdecadal climate oscillation with impacts on salmon production, B. Am. Meteorol. Soc., 78, 1069–1079, 1997. a, b

Maraun, D.: Bias Correction, Quantile Mapping and Downscaling: Revisiting the Inflation Issue, J. Climate, 26, 2137–2143, 2013a. a

Maraun, D.: When will trends in European mean and heavy daily precipitation emerge?, Env. Res. Lett., 8, 014004, https://doi.org/10.1088/1748-9326/8/1/014004, 2013b. a, b

Maraun, D.: Bias Correcting Climate Change Simulations – a Critical Review, Curr. Clim. Change Rep., 2, 211–220, https://doi.org/10.1007/s40641-016-0050-x, 2016. a

Maraun, D. and Widmann, M.: The representation of location by a regional climate model in complex terrain, Hydrol. Earth Syst. Sci., 19, 3449–3456, https://doi.org/10.5194/hess-19-3449-2015, 2015. a

Maraun, D. and Widmann, M.: Statistical Downscaling and Bias Correction for Climate Research, Cambridge University Press, Cambridge, 2018. a, b

Maraun, D., Widmann, M., Gutierrez, J., Kotlarski, S., Chandler, R., Hertig, E., Wibig, J., Huth, R., and Wilcke, R.: VALUE: A Framework to Validate Downscaling Approaches for Climate Change Studies, Earth's Future, 3, 1–14, 2015. a, b

Maraun, D., Shepherd, T., Widmann, M., Zappa, G., Walton, D., Hall, A., Gutierrez, J. M., Hagemann, S., Richter, I., Soares, P., and Mearns, L.: Towards process-informed bias correction of climate change simulations, Nat. Clim. Change, 7, 764–773, https://doi.org/10.1038/NCLIMATE3418, 2017. a, b, c, d, e, f, g, h, i

Mason, S.: Understanding forecast verification statistics, Meteorol. Appl., 15, 31–40, 2008. a, b

Michaelsen, J.: Cross-validation in statistical climate forecast models, J. Clim. Appl. Meteorol., 26, 1589–1600, 1987. a

Piani, C., Haerter, J., and Coppola, E.: Statistical bias correction for daily precipitation in regional climate models over Europe, Theor. Appl. Climatol., 99, 187–192, 2010a. a

Piani, C., Weedon, G., Best, M., Gomes, S., Viterbo, P., Hagemann, S., and Haerter, J.: Statistical bias correction of global simulated daily precipitation and temperature for the application of hydrological models, J. Hydrol., 395, 199–215, 2010b. a

Schlesinger, M. and Ramankutty, N.: An oscillation in the global climate system of period 65–70 years, Nature, 367, 723–726, 1994. a, b, c, d

Stone, M.: Cross-validatory choice and assessment of statistical predictions, J. Roy. Stat. Soc. B, 32, 111–147, 1974. a, b

Teutschbein, C. and Seibert, J.: Bias correction of regional climate model simulations for hydrological climate-change impact studies: Review and evaluation of different methods, J. Hydrol., 456, 12–29, 2012.  a

Themeßl, M. J., Gobiet, A., and Leuprecht, A.: Empirical-statistical downscaling and error correction of daily precipitation from regional climate models, Int. J. Climatol., 31, 1530–1544, 2011. a

van Oldenborgh, G., Doblas Reyes, F.-J., Drijfhout, S., and Hawkins, E.: Reliability of regional climate model trends, Environ Res. Lett., 8, 014055, https://doi.org/10.1088/1748-9326/8/1/014055, 2013. a

Warszawski, L., Frieler, K., Huber, V., Piontek, F., Serdeczny, O., and Schewe, J.: The Inter-Sectoral Impact Model Intercomparison Project (ISI–MIP): Project framework, Proc. Nat. Acad. Sci., 111, 3228–3232, https://doi.org/10.1073/pnas.1312330110, 2014. a

Wilks, D. S.: Statistical Methods in the Atmospheric Sciences, Academic Press/Elsevier, 2 Edn., 2006. a, b

Zappa, G., Shaffrey, L., and Hodges, K.: The ability of CMIP5 models to simulate North Atlantic extratropical cyclones, J. Climate, 26, 5379–5396, 2013. a


text:
# Intensity of the diffraction pattern of the double slit

I am trying another approach for my last unanswered question. (Bounty still on for 3 days. Anyone? Please?) Note that this is not the same question but a greatly simplified version concerning a much more general situation. As this may also be viewed as a problem of probability distributions, I am also going for the quantum tags Many textbooks talk about the "brightness" distribution of the double slit. The one here (Lang/Pucker, Mathematische Methoden der Physik) gives the following example:

$$A((x,y),(x_0,y_0))=\frac{i k \sqrt{(x-x_0)^2+(y-y_0)^2}}{\sqrt{(x-x_0)^2+(y-y_0)^2}}$$ is a spherical (circular) wave.

The double slit is modelt by two of these wave at $(0,-0.5)$, $(0,0.5)$. Let $k=20$, Distance to the screen $a=10$. Then $|A((a,y),(0,-0.5))+A((a,y),(0,0.5))|^2$ is claimed to be the "intensity" $I(y)$ on the screen. The distance of the maxima with these parameters is around 3, with notable values at y>10.

I have multiple issues with that:

1. "Intensity" is a directed quantity. "Intensity on the screen" implies that I could integrate over $I(y)$ to get a power. This doesn't seem to be the case, as the flux is not perpendicular to the screen. This can be easily shown by integrating over a closed circle around a single wave and varying the center. The "power" will vary notably, too. This can be fixed using a directed surface/line element in the integral, but the theory does not give directions for a double slit (for the single slit the direction is trivial).

2. This is modeled 2D, presumably because the slits are assumed to be long. Then I would also assume an intensity dependency of 1/r, not 1/r^2 in this example. Thus, I propose a power of $1/2$ to the denominator of A.

3. Even if I choose the distance to the screen much larger than the distance between the slits (I assume then I may take the flux to be perpendicular to the circumference of a circle around the center between the slits), the result does not equal the power transmitted by the two waves separately (which would be $4\pi$). With the parameters above, I get a full power of $4.6\pi$ (after the correction proposed in 2., <<1 without it), but I can change it arbitrarily when I vary the parameters (e.g $0.76\pi$ for a slit distance of .5 instead of 1). The code I used is shown below.

In the end, I would like to place another slit on the screen and calculate the pattern behind. Consequently, understanding what is happening here is crucial for me.

Here is my code in Mathematica. Note that it uses a circular screen.

r = 10;
L = .31415;
SlotRange = {-0.5, 0.5};
SlotStepW = (SlotRange[[2]] - SlotRange[[1]]);
ImageRange = {-Pi, Pi};
ImageStepW = (ImageRange[[2]] - ImageRange[[1]])/1000;
beam = Table[
Sum[
N[
Exp[
I (2 Pi/L Sqrt[(r Sin[Phi] -
x0)^2 + (r Cos[Phi])^2])]/(Sqrt[(r Sin[Phi] -
x0)^2 + (r Cos[Phi])^2])^(1/2)
]*SlotStepW,
{x0, SlotRange[[1]], SlotRange[[2]], SlotStepW}
],
{Phi, ImageRange[[1]], ImageRange[[2]], ImageStepW}
];
ListPlot[Abs[#]^2 & /@ beam, DataRange -> ImageRange, PlotRange -> All]
energy = (Plus @@ (Abs[#]^2 & /@ beam))*ImageStepW*r/Pi

-
In order to be able to add the amplitudes like this, you must make a paraxial approximation. This is equivalent to a screen distance much larger than the slit spacing, and a wavelength much smaller. Otherwise you must treat the waves vectorially. Try using a linear screen with parameters like a=10000, k=20000 – user2963 Sep 7 '12 at 14:23
So you agree that this example from the book is flawed? For my true problem, however, I cannot vary my parameters that way :) Probably I have to work vectorially, then. – mcandril Sep 8 '12 at 17:51

format: plain


text:
GRE Math Formulas Cheat Sheet

Before walking into the GRE, it is a good idea to know the following formulas/tidbits. In fact, not knowing the information below can seriously hurt your chances of answering a question correctly.

At the same time, while this is a very useful cheat sheet, do not just memorize formulas without actually applying them to a question. Often students will see a question and will assume that a certain formula is relevant. This is not always the case. So make sure you practice using the formulas so you will know when they pertain to a question.

Interest

Simple Interest: V=P\bigg(1+\dfrac{rt}{100}\bigg), where P is principal, r is rate, and t is time Compound Interest: V=P\bigg(1+\dfrac{r}{100n}\bigg)^{nt}, where n is the number of times compounded per year

Work Rates

{\dfrac{1}{T}}={\dfrac{1}{time_1}}+{\dfrac{1}{time_2}}, where T is the time to completion of a task when two workers are combining effort.

Sets

{A+B}-{({A}\bigcup{B})}

Distance, Rate, and Time

D=rt, Distance=Rate \times Time

Circles

Area=\pi{r}^2

Circumference=2\pi{r}

Arc Length={\dfrac{x}{360}}2{\pi}r

Area of sector={\dfrac{x}{360}}{\pi}r^2

Squares

Perimeter = 4s, where s = side

Area = s^2

Rectangles

Area = l \times w, where l = length and w = width

Perimeter = 2l+2w

Trapezoids

{\dfrac{Base1+Base2}{2}} \times \text{height}

Polygons

Total degrees = 180(n-2), where n = # of sides

Average degrees per side or degree measure of congruent polygon = 180\dfrac{(n-2)}{n}

The Distance Formula

\sqrt{\bigg({x_2}-{x_1}\bigg)^2+\bigg({y_2}-{y_1}\bigg)^2}

Prime numbers and integers

1 is not a prime. 2 is the smallest prime and the only even prime.

An integer is any counting number including negative numbers (e.g. -3, -1, 2, 7… but not 2.5)

Fast Fractions

{\dfrac{1}{x}}+{\dfrac{1}{y}}={\dfrac{x+y}{xy}} i.e. {\dfrac{1}{2}}+{\dfrac{1}{5}}={\dfrac{2+5}{2*5}}={\dfrac{7}{10}}

Divisibility

3 : sum of digits divisible by 3

4 : the last two digits of number are divisible by 4

5 : the last digit is either a 5 or zero

6 : even number and sum of digits is divisible by 3

8 : if the last three digits are divisible by 8

9: sum of digits is divisible by 9

Combinations and Permutations

nCr=\dfrac{n!}{r!(n-r)!}

n is the total number, r is the number you are choosing

nPr=\dfrac{n!}{(n-r)!}

Probability

\text{\it{Probability of event}} = \dfrac{\text{\it{number of ways that fit the requirement}}}{\text{\it{number of total ways}}}


text:
Opened 11 years ago

Closed 11 years ago

# Provide Python hooks for Mercurial to call trac-admin

Reported by: Owned by: Remy Blank Remy Blank normal plugin - mercurial plugin/mercurial normal review eric@…

## Description

### in reply to:  2 comment:4 by Remy Blank, 11 years ago

I ended up using the incoming hook instead of changegroup to avoid the need for xargs. Are there any downsides to that?

Yes, a pretty major one. The incoming hook is called once for every new changeset. The changegroup hook is called once with the first new changeset (and the other ones can be determined from the changelog). So with the former, you have to call trac-admin once for every changeset, whereas the latter allows a single call to trac-admin with a list of revisions. Considering that the setup phase of a Trac environment is very expensive, there's a major performance gain in using changegroup.

I'm almost done, and I will need some testers on Windows :)

### comment:5 by Remy Blank, 11 years ago

A hook function has been added to TracMercurial in [10718]. It has been tested on Linux, and I need some volunteers to test on Windows. Copy the file hooks.py somewhere accessible, then configure the hooks in hgrc:

[hooks]

[trac]
env = C:\path\to\env

### follow-up:  7 comment:6 by Christian Boos, 11 years ago

Simple testing shows it works (on Windows), but it could still be improved.

First, using the chain of first children is not going to work in all cases, and using changectx.descendants as I first thought neither. Consider the following case, for the incoming sequence b, c, d where d is the merge of b and c (i.e. we have for example a -> b, a -> c, (b, c) -> d, with a being a changeset already present in the repository). If we start with node == b, we don't want to miss c even if it's not a descendant of b. Fortunately there's a much simpler way to get all the relevant changesets, because the hook is executed in a transaction (at least I verified for pull, I assume its the same for other means), so we have a guarantee that all changesets after node are part of the changegroup. So we'll need to use rev numbers here.

Second remark, using subprocess is one approach… and I understand you could sometimes want to notify environments with different versions of Trac for the same repository; in that situation using trac-admin is certainly the simpler way to go. But in the usual case where you have one version of Trac installed and reachable from the same version of Python used to run the hooks, you could as well call the relevant Trac Python code directly. I propose doing that when there's no trac-admin entry specified in the [trac] section of the .hgrc.

See follow-up patch, t10255-use-revs-r10718.patch.

Last edited 11 years ago by Christian Boos (previous) (diff)

### by Christian Boos, 11 years ago

add all revs for the changegroup and optionally use directly Trac's API for the notification

### in reply to:  6 ; follow-up:  8 comment:7 by Remy Blank, 11 years ago

First, using the chain of first children is not going to work in all cases, and using changectx.descendants as I first thought neither.

To be honest, I took the code from next_rev() of the plugin, because I wrongly assumed that it was used to make the changelog. Using revision numbers makes sense, and allows increasing the default number of changesets per call to trac-admin. I tried to find out how Mercurial implemented its log command, but the code combines many different cases (with a file specification, with revision specification, …) that I couldn't find an easy way.

Second remark, using subprocess is one approach… and I understand you could sometimes want to notify environments with different versions of Trac for the same repository; in that situation using trac-admin is certainly the simpler way to go.

It's not really due to different versions of Trac, but rather for the situation where you run Trac instances in virtualenvs. I actually wanted to add some functionality to handle this case (instead of requiring to write a wrapper script for trac-admin), but I was afraid that it would be difficult to make it cross-platform.

But in the usual case where you have one version of Trac installed and reachable from the same version of Python used to run the hooks, you could as well call the relevant Trac Python code directly. I propose doing that when there's no trac-admin entry specified in the [trac] section of the .hgrc.

I'm not sure I like the idea. It means that we load a complete Trac environment into the process executing the operation, which is fine when you run the hg executable (local operations, ssh) but not so nice when you run hgweb. All your hgweb processes end up taking much more memory. The upside is that you only pay once for environment startup, which can be nice if you push a lot.

### in reply to:  7 ; follow-up:  9 comment:8 by Christian Boos, 11 years ago

But in the usual case where you have one version of Trac installed and reachable from the same version of Python used to run the hooks, you could as well call the relevant Trac Python code directly. I propose doing that when there's no trac-admin entry specified in the [trac] section of the .hgrc.

I'm not sure I like the idea. It means that we load a complete Trac environment into the process executing the operation, which is fine when you run the hg executable (local operations, ssh) but not so nice when you run hgweb. All your hgweb processes end up taking much more memory.

It's probably worth to measure this cost … and verify the env gets properly gc'ed. If memory usage appears to be a real issue, it's easy to use the alternative way with trac-admin. But if think it's nice to have at least the option to bypass the call to trac-admin.

The upside is that you only pay once for environment startup, which can be nice if you push a lot.

Well, only if we add use_cache=True to open_environment(), which I didn't do here.

### in reply to:  8 ; follow-up:  10 comment:9 by Remy Blank, 11 years ago

It's probably worth to measure this cost … and verify the env gets properly gc'ed. If memory usage appears to be a real issue, it's easy to use the alternative way with trac-admin. But if think it's nice to have at least the option to bypass the call to trac-admin.

IIRC, Python is notoriously bad at releasing memory back to the OS, so even if the environment is garbage-collected, the process will probably keep the memory around. Measuring should indeed clarify the issue.

Well, only if we add use_cache=True to open_environment(), which I didn't do here.

Shame, then I see no upside at all :)

I mean, in the configuration where everything is installed globally, the trac-admin executable should be in PATH, so the default value of trac-admin should already be fine.

### in reply to:  9 ; follow-up:  11 comment:10 by Christian Boos, 11 years ago

It's probably worth to measure this cost … and verify the env gets properly gc'ed. If memory usage appears to be a real issue, it's easy to use the alternative way with trac-admin. But if think it's nice to have at least the option to bypass the call to trac-admin.

IIRC, Python is notoriously bad at releasing memory back to the OS, so even if the environment is garbage-collected, the process will probably keep the memory around.

Not on Windows ;-) But even without gc'ing the env, I'd be surprised if from one hook run to the next you'd have any noticeable memory increase.

Measuring should indeed clarify the issue.

Well, only if we add use_cache=True to open_environment(), which I didn't do here.

Shame, then I see no upside at all :)

For me the upside is to avoid having to launch an external process, pass a possibly long list of command line arguments, when the call could have been done instead directly in the current process, in one go. I indeed didn't think about long running processes, that's why I didn't bother using the cache. But if the memory impact is negligible, it might indeed be interesting to activate the cache.

I mean, in the configuration where everything is installed globally, the trac-admin executable should be in PATH, so the default value of trac-admin should already be fine.

Again, probably not on Windows.

### in reply to:  10 comment:11 by Remy Blank, 11 years ago

Not on Windows ;-) But even without gc'ing the env, I'd be surprised if from one hook run to the next you'd have any noticeable memory increase.

Not from one run to the next. You would get one increase on first run, and then constant usage (probably).

For me the upside is to avoid having to launch an external process, pass a possibly long list of command line arguments, when the call could have been done instead directly in the current process, in one go.

Ah, you've been lurking on Mercurial mailing list for too long, and are influenced by all the people moaning when they are told to call hg on the command-line to integrate Mercurial into their software.

In this particular case, I find the subprocess approach quite sensible: start a new process, use whatever memory is necessary, and release all of it to the OS on termination. Nice and clean.

But I don't mind having the option to call Trac directly. Choice is good ;) Patch applied in [10726].

Do you want me to add an option cache_env to allow using the environment cache?

### comment:12 by Remy Blank, 11 years ago

Added environment variable expansion in paths in [10727].

### comment:13 by Remy Blank, 11 years ago

Resolution: → fixed new → closed

Added the Mercurial option cache_env in [10731]. Please reopen if there's anything else to be done.

### Modify Ticket

Change Properties
Summary: Description: Setting up hooks in Mercurial to call trac-admin \$ENV changeset commands is reasonably simple on Unix, but quite difficult on Windows due to the absence of xargs. We could simplify the setup by providing Python functions for the commit and changegroup in MercurialPlugin. This would make for a very simple setup when the plugin is installed globally: {{{ [hooks] commit = python:tracext.hg.hooks:commit changegroup = python:tracext.hg.hooks:changegroup }}} Or, if the plugin is installed per environment, people could download the file hooks.py from here and use: {{{ [hooks] commit = python:/path/to/hooks.py:commit changegroup = python:/path/to/hooks.py:changegroup }}} Of course, the hooks should be cross-platform. The path to the trac-admin command should be configurable in .hgrc. You may use WikiFormatting here. defect enhancement task highest high normal low lowest 1.6 1.7.1 1.5.4 1.4.4 1.5.5 1.4.5 1.6.1 next-dev-1.7.x next-major-releases next-stable-1.4.x not applicable plugin - mercurial plugin - spam-filter topic-multiproject topic-wikiengine translations undecided unscheduled admin/console admin/web attachment contrib database backend general i18n notification plugin/git plugin/mercurial plugin/spamfilter project query system rendering report system roadmap search system ticket system timeline version control version control/browser version control/changeset view version control/log view web frontend web frontend/mod_python web frontend/mod_wsgi web frontend/tracd wiki system 1.3dev 1.2dev 1.1dev 1.0dev 1.0-stable 0.12dev 0.12-stable 1.5.3 1.4.3 1.5.2 1.2.6 1.0.20 1.4.2 1.5.1 1.4.1 1.0.19 1.4 1.2.5 1.3.6 1.0.18 1.3.5 1.3.4 1.2.4 1.2.3 1.3.3 1.0.17 0.12.8 1.3.2 1.2.2 1.0.15 1.0.14 1.2.1 1.3.1 1.2 1.0.13 1.0.12 1.0.11 1.0.10 1.0.9 1.0.8 1.1.6 1.0.7 0.12.7 1.0.6 1.1.5 1.1.4 1.0.5 1.0.4 1.1.3 1.0.3 1.1.2 1.0.2 0.12.6 1.0.1 0.12.5 1.1.1dev 1.0 0.12.4 1.0b1 0.12.3 0.12.2 0.12.2rc1 0.12.1 0.13dev 0.12 0.12rc1 0.12b1 0.11.7 0.11.6 0.11.5 0.10.5 0.10.4 0.10.1 0.9.6 0.9.3 0.8.4 blocker critical major normal minor trivial Set your email in Preferences
Action
as closed The owner will remain Remy Blank.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Remy Blank to the specified user.


text:
How do I set the certificate for k8s to more than 1 year?

I’m a novice k8s engineer.
Until now, I have managed the k8s certificate by manually renewing it once a year.

However, I became curious about how to set the certificates below for more than 1 year in the first place.

[[email protected] ~]# kubeadm alpha certs check-expiration

CERTIFICATE                EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
admin.conf                 May 21, 2022 01:29 UTC   311d            no
apiserver                  May 21, 2022 01:29 UTC   311d            no
apiserver-etcd-client      May 21, 2022 01:29 UTC   311d            no
apiserver-kubelet-client   May 21, 2022 01:29 UTC   311d            no
controller-manager.conf    May 21, 2022 01:29 UTC   311d            no
etcd-healthcheck-client    May 21, 2022 01:29 UTC   311d            no
etcd-peer                  May 21, 2022 01:29 UTC   311d            no
etcd-server                May 21, 2022 01:29 UTC   311d            no
front-proxy-client         May 21, 2022 01:29 UTC   311d            no
scheduler.conf             May 21, 2022 01:29 UTC   311d            no

Can these certificates be extended by more than a year? (nearly 10 years..?)

k8s version is 1.16

I can use a private certificate. It doesn’t matter which way.
(There is a limit to searching because I am not good at English.)

Please tell me how to renew the K8s certificate(api..) only once every 10 years!

Source: Docker Questions


text:
# Calculus

Calculus Level 3

Find the number of $$x \in \mathbb R$$ for which the equation below holds true.

$\large x^2-x \sin x-\cos x=0$

×

Problem Loading...

Note Loading...

Set Loading...


text:
# Hilbert space embeddings and metrics on probability measures

A Hilbert space embedding for probability measures has recently been proposed, with applications including dimensionality reduction, homogeneity testing, and independence testing. This embedding represents any probability measure as a mean element in a reproducing kernel Hilbert space (RKHS). A pseudometric on the space of probability measures can be defined as the distance between distribution embeddings: we denote this as $\gamma_k$, indexed by the kernel function $k$ that defines the inner product in the RKHS. We present three theoretical properties of $\gamma_k$. First, we consider the question of determining the conditions on the kernel $k$ for which $\gamma_k$ is a metric: such $k$ are denoted {\em characteristic kernels}. Unlike pseudometrics, a metric is zero only when two distributions coincide, thus ensuring the RKHS embedding maps all distributions uniquely (i.e., the embedding is injective). While previously published conditions may apply only in restricted circumstances (e.g. on compact domains), and are difficult to check, our conditions are straightforward and intuitive: bounded continuous strictly positive definite kernels are characteristic. Alternatively, if a bounded continuous kernel is translation-invariant on $\bb{R}^d$, then it is characteristic if and only if the support of its Fourier transform is the entire $\bb{R}^d$. Second, we show that there exist distinct distributions that are arbitrarily close in $\gamma_k$. Third, to understand the nature of the topology induced by $\gamma_k$, we relate $\gamma_k$ to other popular metrics on probability measures, and present conditions on the kernel $k$ under which $\gamma_k$ metrizes the weak topology.

PDF Abstract

## Code Add Remove Mark official

No code implementations yet. Submit your code now

## Datasets

Add Datasets introduced or used in this paper

## Results from the Paper Edit

Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.